Hi All
I have enabled job metrics on a local galaxy install, with latest code from dev branch. I have "<collectl /> " uncommented in job_metrics_conf.xml file:
Following are the errors in the logs:
galaxy.jobs.metrics ERROR 2016-02-24 12:28:08,741 Failed to collect job properties for plugin <galaxy.jobs.metrics.instrumenters.collectl.CollectlPlugin object at 0x7ff0b6af3a50> Traceback (most recent call last): File "dev/galaxy/lib/galaxy/jobs/metrics/__init__.py", line 101, in collect_properties properties = plugin.job_properties( job_id, job_directory ) File "dev/galaxy/lib/galaxy/jobs/metrics/instrumenters/collectl.py", line 105, in job_properties raise Exception( message ) Exception: Failed to find collectl log in directory
It seems like collectl is looking for the following files:
['__instrument_collectl_pid', '__instrument_env_vars', 'galaxy.json', 'metadata_kwds_HistoryDatasetAssociation_2_1rh1WC', 'set_metadata_N05Q24.py', '__instrument_core_galaxy_slots', 'metadata_results_HistoryDatasetAssociation_2_33gTDj', '__instrument_core_epoch_start', 'galaxy_2.sh', 'metadata_out_HistoryDatasetAssociation_2_G5bZYP', '__instrument_core_epoch_end', '__instrument_meminfo_meminfo', '__instrument_uname_uname', 'metadata_override_HistoryDatasetAssociation_2_krldDM', 'set_metadata_lpIXS5.py', 'metadata_in_HistoryDatasetAssociation_2_iiYW0t', '__instrument_cpuinfo_cpuinfo', 'tool_script.sh', 'galaxy_2.ec']
Which are suppose to exist under galaxy/database/job_working_directory/000 job directory.
Has anybody faced anything similar issue or is it a configuration issue, or perhaps a known issue with job_metrics (collectl)?
BTW: I installed collectl packages on the host.
Regards Zipho
On Wed, Feb 24, 2016 at 5:37 AM, Ziphozakhe Mashologu zipho@sanbi.ac.za wrote:
Hi All
I have enabled job metrics on a local galaxy install, with latest code from dev branch. I have "<collectl /> " uncommented in job_metrics_conf.xml file:
Following are the errors in the logs:
galaxy.jobs.metrics ERROR 2016-02-24 12:28:08,741 Failed to collect job properties for plugin <galaxy.jobs.metrics.instrumenters.collectl.CollectlPlugin object at 0x7ff0b6af3a50> Traceback (most recent call last): File "dev/galaxy/lib/galaxy/jobs/metrics/__init__.py", line 101, in collect_properties properties = plugin.job_properties( job_id, job_directory ) File "dev/galaxy/lib/galaxy/jobs/metrics/instrumenters/collectl.py", line 105, in job_properties raise Exception( message ) Exception: Failed to find collectl log in directory
It seems like collectl is looking for the following files:
['__instrument_collectl_pid', '__instrument_env_vars', 'galaxy.json', 'metadata_kwds_HistoryDatasetAssociation_2_1rh1WC', 'set_metadata_N05Q24.py', '__instrument_core_galaxy_slots', 'metadata_results_HistoryDatasetAssociation_2_33gTDj', '__instrument_core_epoch_start', 'galaxy_2.sh', 'metadata_out_HistoryDatasetAssociation_2_G5bZYP', '__instrument_core_epoch_end', '__instrument_meminfo_meminfo', '__instrument_uname_uname', 'metadata_override_HistoryDatasetAssociation_2_krldDM', 'set_metadata_lpIXS5.py', 'metadata_in_HistoryDatasetAssociation_2_iiYW0t', '__instrument_cpuinfo_cpuinfo', 'tool_script.sh', 'galaxy_2.ec']
Which are suppose to exist under galaxy/database/job_working_directory/000 job directory.
Has anybody faced anything similar issue or is it a configuration issue, or perhaps a known issue with job_metrics (collectl)?
BTW: I installed collectl packages on the host.
Hi Zipho,
If this Galaxy instance runs jobs on a cluster, you would also need collectl to be installed on the cluster.
--nate
Regards Zipho
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Nate
Thanks for the response.
It was indeed part of the problem. I needed collectl installed on the individual cluster instances.(job runners)
Seems to be working properly now. I plan to work on using the metrics to dynamical & intelligently choose job destination of mapping jobs for example.
Thanks Z On 26 Feb 2016 15:21, "Nate Coraor" nate@bx.psu.edu wrote:
On Wed, Feb 24, 2016 at 5:37 AM, Ziphozakhe Mashologu zipho@sanbi.ac.za wrote:
Hi All
I have enabled job metrics on a local galaxy install, with latest code from dev branch. I have "<collectl /> " uncommented in job_metrics_conf.xml file:
Following are the errors in the logs:
galaxy.jobs.metrics ERROR 2016-02-24 12:28:08,741 Failed to collect job properties for plugin <galaxy.jobs.metrics.instrumenters.collectl.CollectlPlugin object at 0x7ff0b6af3a50> Traceback (most recent call last): File "dev/galaxy/lib/galaxy/jobs/metrics/__init__.py", line 101, in collect_properties properties = plugin.job_properties( job_id, job_directory ) File "dev/galaxy/lib/galaxy/jobs/metrics/instrumenters/collectl.py", line 105, in job_properties raise Exception( message ) Exception: Failed to find collectl log in directory
It seems like collectl is looking for the following files:
['__instrument_collectl_pid', '__instrument_env_vars', 'galaxy.json', 'metadata_kwds_HistoryDatasetAssociation_2_1rh1WC', 'set_metadata_N05Q24.py', '__instrument_core_galaxy_slots', 'metadata_results_HistoryDatasetAssociation_2_33gTDj', '__instrument_core_epoch_start', 'galaxy_2.sh', 'metadata_out_HistoryDatasetAssociation_2_G5bZYP', '__instrument_core_epoch_end', '__instrument_meminfo_meminfo', '__instrument_uname_uname', 'metadata_override_HistoryDatasetAssociation_2_krldDM', 'set_metadata_lpIXS5.py', 'metadata_in_HistoryDatasetAssociation_2_iiYW0t', '__instrument_cpuinfo_cpuinfo', 'tool_script.sh', 'galaxy_2.ec']
Which are suppose to exist under galaxy/database/job_working_directory/000 job directory.
Has anybody faced anything similar issue or is it a configuration issue, or perhaps a known issue with job_metrics (collectl)?
BTW: I installed collectl packages on the host.
Hi Zipho,
If this Galaxy instance runs jobs on a cluster, you would also need collectl to be installed on the cluster.
--nate
Regards Zipho
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
galaxy-dev@lists.galaxyproject.org
-
Nate Coraor -
Ziphozakhe Mashologu