Hi Nate Thanks for the response. It was indeed part of the problem. I needed collectl installed on the individual cluster instances.(job runners) Seems to be working properly now. I plan to work on using the metrics to dynamical & intelligently choose job destination of mapping jobs for example. Thanks Z On 26 Feb 2016 15:21, "Nate Coraor" <nate@bx.psu.edu> wrote:

On Wed, Feb 24, 2016 at 5:37 AM, Ziphozakhe Mashologu <zipho@sanbi.ac.za> wrote:

...

Hi All

I have enabled job metrics on a local galaxy install, with latest code from dev branch. I have "<collectl /> " uncommented in job_metrics_conf.xml file:

Following are the errors in the logs:

galaxy.jobs.metrics ERROR 2016-02-24 12:28:08,741 Failed to collect job properties for plugin <galaxy.jobs.metrics.instrumenters.collectl.CollectlPlugin object at 0x7ff0b6af3a50> Traceback (most recent call last): File "dev/galaxy/lib/galaxy/jobs/metrics/__init__.py", line 101, in collect_properties properties = plugin.job_properties( job_id, job_directory ) File "dev/galaxy/lib/galaxy/jobs/metrics/instrumenters/collectl.py", line 105, in job_properties raise Exception( message ) Exception: Failed to find collectl log in directory

It seems like collectl is looking for the following files:

['__instrument_collectl_pid', '__instrument_env_vars', 'galaxy.json', 'metadata_kwds_HistoryDatasetAssociation_2_1rh1WC', 'set_metadata_N05Q24.py', '__instrument_core_galaxy_slots', 'metadata_results_HistoryDatasetAssociation_2_33gTDj', '__instrument_core_epoch_start', 'galaxy_2.sh', 'metadata_out_HistoryDatasetAssociation_2_G5bZYP', '__instrument_core_epoch_end', '__instrument_meminfo_meminfo', '__instrument_uname_uname', 'metadata_override_HistoryDatasetAssociation_2_krldDM', 'set_metadata_lpIXS5.py', 'metadata_in_HistoryDatasetAssociation_2_iiYW0t', '__instrument_cpuinfo_cpuinfo', 'tool_script.sh', 'galaxy_2.ec']

Which are suppose to exist under galaxy/database/job_working_directory/000 job directory.

Has anybody faced anything similar issue or is it a configuration issue, or perhaps a known issue with job_metrics (collectl)?

BTW: I installed collectl packages on the host.

Hi Zipho,

If this Galaxy instance runs jobs on a cluster, you would also need collectl to be installed on the cluster.

--nate

...

Regards Zipho

___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/