galaxy-dev May 2017

galaxy-dev@lists.galaxyproject.org

35 participants
34 discussions

location of history id
by Aarthi Mohan 15 May '17

15 May '17

Hello all, I have a local instance of galaxy version 15.03.1 (this is needed by IRIDA tool). I have retrieved datasets associated with a history using Bioblend. I want to confirm, if the history id used in get_histories(hid) or show_history(hid) is stored in the database? I have tried searching the entire database by mysqldump, and couldn't find such values in it. Where else is this id stored and will this be deleted periodically? Appreciate your help! Thanks, Aarthi ________________________________ Important: This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately; you should not copy or use it for any purpose, nor disclose its contents to any other person. Thank you.

2 1

Galaxy Start Up Error - Double Requirements, Tempita
by Yip, Miu ki 15 May '17

15 May '17

Hi all, When attempting to start up Galaxy, via the run.sh script, we’re getting the following error: galaxyd@genomics:/galaxy$ sh run.sh Activating virtualenv at .venv Requirement already satisfied: pip>=8.1 in ./.venv/lib/python2.7/site-packages Double requirement given: Tempita==0.5.3.dev0 (from -r requirements.txt (line 155)) (already in Tempita==0.5.3dev (from -r requirements.txt (line 57)), name='Tempita') This occurs when the start up script is run with the —no-create-venv and without the flag. It looks like there are two “Tempita” items in the requirements.txt, if one of them is removed, it leads to another error about “ethtools” galaxyd@genomics:/galaxy$ sh run.sh --daemon --no-create-venv Activating virtualenv at .venv Requirement already satisfied: pip>=8.1 in ./.venv/lib/python2.7/site-packages ... Collecting crypto==1.4.1 (from -r requirements.txt (line 81)) Using cached crypto-1.4.1-py2.py3-none-any.whl Requirement already satisfied: cycler==0.10.0 in ./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 82)) Collecting Cython==0.25.2 (from -r requirements.txt (line 83)) Using cached Cython-0.25.2-cp27-cp27mu-manylinux1_x86_64.whl Requirement already satisfied: docopt==0.6.2 in ./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 86)) Requirement already satisfied: drmaa==0.7.6 in ./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 88)) Collecting ethtool==0.8 (from -r requirements.txt (line 90)) Could not find a version that satisfies the requirement ethtool==0.8 (from -r requirements.txt (line 90)) (from versions: ) No matching distribution found for ethtool==0.8 (from -r requirements.txt (line 90)) We’re not sure if both versions of “Tempita” are required and if they are, how can the application be started up? Running both the run.sh and the run_reports.sh scripts lead to this error about “Tempita”. Has anyone encountered this error before? Thanks!

2 1

Pulsar - metadata staging rewrite problem
by Poole, Richard 13 May '17

13 May '17

Hi John and all, We are running an old version of Galaxy v15.10 and use Pulsar to stage certain jobs to our hpc cluster. This works great and we love it. However, we have noticed that for certain tools which require metadata to be staged, although the metadata file is actually staged, the Galaxy pulsar runner fails to rewrite the file path for the metadata correctly. Here is an example command from the latest Freebayes wrappers: ln -s -f '/cluster/galaxy/pulsar/files/staging/32728/inputs/dataset_54494.dat' 'b_0.bam' && ln -s -f '/Volumes/ngs/database/files/_metadata_files/002/metadata_2754.dat' 'b_0.bam.bai' && ln -s -f '/cluster/galaxy/pulsar/files/staging/32728/inputs/dataset_54496.dat' 'b_1.bam' && ln -s -f '/Volumes/ngs/database/files/_metadata_files/002/metadata_2755.dat' 'b_1.bam.bai' && samtools view -H b_0.bam | grep "^@SQ" | cut -f 2- | awk '{ gsub("^SN:","",$1); gsub("^LN:","",$2); print $1"\t0\t"$2; }' >> regions_all.bed && samtools view -H b_1.bam | grep "^@SQ" | cut -f 2- | awk '{ gsub("^SN:","",$1); gsub("^LN:","",$2); print $1"\t0\t"$2; }' >> regions_all.bed && sort -u regions_all.bed > regions_uniq.bed && mkdir vcf_output && mkdir failed_alleles && mkdir trace && for i in `cat regions_uniq.bed | awk '{print $1":"$2".."$3}'`; do echo " freebayes --region '$i' --bam 'b_0.bam' --bam 'b_1.bam' --fasta-reference '/cluster/galaxy/indexes/danRer7/sam_index/danRer7.fa' --vcf './vcf_output/part_$i.vcf' --standard-filters --min-coverage '3' "; done > freebayes_commands.sh && cat freebayes_commands.sh | parallel --no-notice -j ${GALAXY_SLOTS:-1} && grep "^#" "./vcf_output/part_$i.vcf" > header.txt && for i in `cat regions_uniq.bed | awk '{print $1":"$2".."$3}'`; do cat "./vcf_output/part_$i.vcf" | grep -v "^#" || true ; done | sort -k1,1 -k2,2n -k5,5 -u | cat header.txt - > '/cluster/galaxy/pulsar/files/staging/32728/outputs/dataset_54897.dat’ As you can see, the file path for the BAM file is correctly re-written but not the file path for the bam.bai index, which still has a file path local to Galaxy. Has anyone come across this problem before and is there a fix? We are very hesitant to upgrade our Galaxy server as everything else is working perfectly for us right now. Thanks, Richard

1 0

Pulsar - running as real DRMAA user problems
by Poole, Richard 13 May '17

13 May '17

Hey John, So I’ve been happily using Pulsar to send all my Galaxy server jobs to our cluster here at UCL for several months now (I love it!). I am now exploring the ‘run-as-real-user’ option for DRMAA submissions and have run into a problem. The files are correctly staged, correctly chowned, successfully submitted to the queue and the job runs. However, at job end (collection?) fails with the following error message in Pulsar: Exception happened during processing of request from (‘*.*.*.*', 54321) Traceback (most recent call last): File "/opt/rocks/lib/python2.6/site-packages/Paste-2.0.1-py2.6.egg/paste/httpserver.py", line 1072, in process_request_in_thread self.finish_request(request, client_address) File "/opt/rocks/lib/python2.6/SocketServer.py", line 322, in finish_request self.RequestHandlerClass(request, client_address, self) File "/opt/rocks/lib/python2.6/SocketServer.py", line 617, in __init__ self.handle() File "/opt/rocks/lib/python2.6/site-packages/Paste-2.0.1-py2.6.egg/paste/httpserver.py", line 446, in handle BaseHTTPRequestHandler.handle(self) File "/opt/rocks/lib/python2.6/BaseHTTPServer.py", line 329, in handle self.handle_one_request() File "/opt/rocks/lib/python2.6/site-packages/Paste-2.0.1-py2.6.egg/paste/httpserver.py", line 441, in handle_one_request self.wsgi_execute() File "/opt/rocks/lib/python2.6/site-packages/Paste-2.0.1-py2.6.egg/paste/httpserver.py", line 291, in wsgi_execute self.wsgi_start_response) File "/cluster/galaxy/pulsar/pulsar/web/framework.py", line 39, in __call__ return controller(environ, start_response, **request_args) File "/cluster/galaxy/pulsar/pulsar/web/framework.py", line 144, in controller_replacement result = self.__execute_request(func, args, req, environ) File "/cluster/galaxy/pulsar/pulsar/web/framework.py", line 124, in __execute_request result = func(**args) File "/cluster/galaxy/pulsar/pulsar/web/routes.py", line 82, in status return status_dict(manager, job_id) File "/cluster/galaxy/pulsar/pulsar/manager_endpoint_util.py", line 12, in status_dict job_status = manager.get_status(job_id) File "/cluster/galaxy/pulsar/pulsar/managers/stateful.py", line 95, in get_status proxy_status, state_change = self.__proxy_status(job_directory, job_id) File "/cluster/galaxy/pulsar/pulsar/managers/stateful.py", line 115, in __proxy_status proxy_status = self._proxied_manager.get_status(job_id) File "/cluster/galaxy/pulsar/pulsar/managers/queued_external_drmaa_original.py", line 62, in get_status external_status = super(ExternalDrmaaQueueManager, self)._get_status_external(external_id) File "/cluster/galaxy/pulsar/pulsar/managers/base/base_drmaa.py", line 31, in _get_status_external drmaa_state = self.drmaa_session.job_status(external_id) File "/cluster/galaxy/pulsar/pulsar/managers/util/drmaa/__init__.py", line 50, in job_status return self.session.jobStatus(str(external_job_id)) File "build/bdist.linux-x86_64/egg/drmaa/session.py", line 518, in jobStatus c(drmaa_job_ps, jobId, byref(status)) File "build/bdist.linux-x86_64/egg/drmaa/helpers.py", line 299, in c return f(*(args + (error_buffer, sizeof(error_buffer)))) File "build/bdist.linux-x86_64/egg/drmaa/errors.py", line 151, in error_check raise _ERRORS[code - 1](error_string) InvalidJobException: code 18: The job specified by the 'jobid' does not exist. With this corresponding error from my Galaxy server: galaxy.tools.actions INFO 2016-10-13 18:47:51,851 Handled output (279.421 ms) galaxy.tools.actions INFO 2016-10-13 18:47:52,093 Verified access to datasets (5.271 ms) galaxy.tools.execute DEBUG 2016-10-13 18:47:52,118 Tool [toolshed.g2.bx.psu.edu/repos/devteam/sam_to_bam/sam_to_bam/1.1.4] created job [25008<http://toolshed.g2.bx.psu.edu/repos/devteam/sam_to_bam/sam_to_bam/1.1.4]%20…>] (560.404 ms) galaxy.jobs DEBUG 2016-10-13 18:47:52,579 (25008) Working directory for job is: /Users/galaxy/galaxy-dist/database/job_working_directory/025/25008 galaxy.jobs.handler DEBUG 2016-10-13 18:47:52,591 (25008) Dispatching to pulsar runner galaxy.jobs DEBUG 2016-10-13 18:47:52,677 (25008) Persisting job destination (destination id: hpc_low) galaxy.jobs.runners DEBUG 2016-10-13 18:47:52,681 Job [25008] queued (90.231 ms) galaxy.jobs.handler INFO 2016-10-13 18:47:52,699 (25008) Job dispatched galaxy.tools.deps DEBUG 2016-10-13 18:47:53,138 Building dependency shell command for dependency 'samtools' galaxy.jobs.runners.pulsar INFO 2016-10-13 18:47:53,233 Pulsar job submitted with job_id 25008 galaxy.jobs DEBUG 2016-10-13 18:47:53,257 (25008) Persisting job destination (destination id: hpc_low) galaxy.datatypes.metadata DEBUG 2016-10-13 18:51:03,922 Cleaning up external metadata files galaxy.jobs.runners.pulsar ERROR 2016-10-13 18:51:03,945 failure finishing job 25008 Traceback (most recent call last): File "/Users/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pulsar.py", line 386, in finish_job run_results = client.full_status() File "/Users/galaxy/galaxy-dist/lib/pulsar/client/client.py", line 132, in full_status return self.raw_check_complete() File "/Users/galaxy/galaxy-dist/lib/pulsar/client/decorators.py", line 28, in replacement return func(*args, **kwargs) File "/Users/galaxy/galaxy-dist/lib/pulsar/client/decorators.py", line 13, in replacement response = func(*args, **kwargs) File "/Users/galaxy/galaxy-dist/lib/pulsar/client/client.py", line 146, in raw_check_complete check_complete_response = self._raw_execute("status", {"job_id": self.job_id}) File "/Users/galaxy/galaxy-dist/lib/pulsar/client/client.py", line 215, in _raw_execute return self.job_manager_interface.execute(command, args, data, input_path, output_path) File "/Users/galaxy/galaxy-dist/lib/pulsar/client/interface.py", line 96, in execute response = self.transport.execute(url, method=method, data=data, input_path=input_path, output_path=output_path) File "/Users/galaxy/galaxy-dist/lib/pulsar/client/transport/standard.py", line 34, in execute response = self._url_open(request, data) File "/Users/galaxy/galaxy-dist/lib/pulsar/client/transport/standard.py", line 20, in _url_open return urlopen(request, data) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open response = meth(req, response) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response 'http', request, response, code, msg, hdrs) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error return self._call_chain(*args) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 500: Internal Server Error I am running 15.10 and Python 2.7.10 on my iMac for the server and the cluster submission node is running Pulsar 0.5.0 and Python 2.7.12 For these tests I run Pulsar in an interactive window so I have not set the sudoers file up, but rather enter sudo password when requested by Pulsar (at the first step of chowning the staging directory). Also have rewrites set up in Galaxy’s pulsar_actions.yml and I am using remote_scp for the file transfers rather than http - although I have also tried switching back to http (as I noticed caching, which I am also testing, does not work with scp transfers) but get an identical set of error messages. As I say, I have no troubles using a regular queued_drmaa manager in pulsar. Any ideas what the problem may be? Cheers, Rich Richard J Poole PhD Wellcome Trust Fellow Department of Cell and Developmental Biology University College London 518 Rockefeller 21 University Street, London WC1E 6DE Office (518 Rockefeller): +44 20 7679 6577 (int. 46577) Lab (529 Rockefeller): +44 20 7679 6133 (int. 46133) https://www.ucl.ac.uk/cdb/academics/poole

2 5

Fwd: Galaxy Error while Using PBS
by Jennifer Hillman-Jackson 12 May '17

12 May '17

---------- Forwarded message ---------- From: Wenkai Wang (Kevin) <wenkai_wang(a)shbiochip.com> Date: Thu, May 11, 2017 at 8:01 PM Subject: Galaxy Error while Using PBS I am an engineer at a National Research and Engineering Center in Shanghai. Recently I am trying the outstanding Galaxy system developed by your team. The Galaxy system is great; however, there are some errors when Galaxy is connected to Torque PBS, about which errors I have to ask for help and advice from you. When I tried to execute a tool for output like "Hello Galaxy" from within the webpage, it was shown in webpage that "Unable to run this job due to a cluster error, please retry it later" (ref.: line 321 of file galaxy/lib/galaxy/jobs/runners/pbs.py). Meanwhile, the STDOUT of Linux shell displays information including, galaxy.jobs.runners.pbs DEBUG 2017-05-12 11:41:47,459 (25) submitting file /data3/wangwk/app/galaxy/database/pbs/25.sh galaxy.jobs.runners.pbs WARNING 2017-05-12 11:41:47,460 (25) pbs_submit failed (try 1/5), PBS error 15033: No free connections ...... galaxy.jobs.runners.pbs WARNING 2017-05-12 11:41:55,470 (25) pbs_submit failed (try 5/5), PBS error 15033: No free connections galaxy.jobs.runners.pbs ERROR 2017-05-12 11:41:57,473 (25) All attempts to submit job failed galaxy.model.metadata DEBUG 2017-05-12 11:41:57,574 Cleaning up external metadata files galaxy.model.metadata DEBUG 2017-05-12 11:41:57,604 Failed to cleanup MetadataTempFile temp files from /data3/wangwk/app/galaxy/ database/jobs_directory/000/25/metadata_out_HistoryDatasetAssociation_26_wrVwaf: No JSON object could be decoded Further information: (1) Linux version: RHEL 6.2, 2.6.32-220.el6.x86_64; (2) Torque PBS 4.2; (3) pbs-python: 4.4.1.2 (pbs-python 4.4.2.1 requires CentOS 7 with PBS Torque 5, and thus was not chosen); (4) glibc 2.12; (5) munge service has been installed, configured, and started on the master node and compute nodes of the cluster; (6) job_conf.xml file is attached with this email. Also, I tried sara_nodes.py from webpage https://oss.trac.surfsara.nl/ pbs_python/wiki/TorqueExamples . The error information is, Traceback (most recent call last): File "./sara_nodes.py", line 644, in <module> print_overview_normal(args.nodes) File "./sara_nodes.py", line 298, in print_overview_normal matched, rest = print_get_nodes(hosts) File "./sara_nodes.py", line 199, in print_get_nodes pbsq = PBSQuery.PBSQuery() File "/home/wangwk/app/anaconda2/lib/python2.7/site-packages/pbs/PBSQuery.py", line 137, in __init__ self.job_server_id = list(self.get_serverinfo())[0] IndexError: list index out of range The error above means function self.get_serverinfo() returns nothing and thus the size of list created based on it is zero. This might be useful for analysis. It would be deeply appreciated if you could offer some guide regarding how to further track or address this problem. Thanks a lot! Best Wishes, Wenkai (Kevin) ------------------------------ Wenkai Wang, National Research and Engineering Center for Biochip Email: wenkai_wang(a)shbiochip.com Email: ww288(a)cantab.net

1 0

Unable to run workflow from command-line
by Aarthi Mohan 12 May '17

12 May '17

Hi all, Aplogies for the long message! I am trying to execute a galaxy workflow from command-line using bioblend. The workflow invloves bwa, gatk, samtools mpileup, bcftools and works with dataset paired-end collections. The following is the script used to achieve it and I am unable to get the workflow to run > from bioblend.galaxy import GalaxyInstance > from bioblend.galaxy.histories import HistoryClient > from bioblend.galaxy.tools import ToolClient > from bioblend.galaxy.workflows import WorkflowClient > from bioblend.galaxy.datasets import DatasetClient > from bioblend.galaxy import dataset_collections as collections > import os,re > > url , key = "http://#.#.#.#:9090", "c5fe47950c99c52a95c6de2542599ddb" > galaxyInstance = GalaxyInstance(url=url, key=key) > historyClient = HistoryClient(galaxyInstance) > toolClient = ToolClient(galaxyInstance) > workflowClient = WorkflowClient(galaxyInstance) > datasetClient = DatasetClient(galaxyInstance) > workflow_list = workflowClient.get_workflows() > > bwa_workflow_id = [ i['id'] for i in workflow_list if > i['name'].startswith('bwa') ][0] #desired workflow id > bwa_workflow_details = workflowClient.show_workflow(bwa_workflow_id) > #workflow details like input, steps, etc., > bwa_workflow_input_id = bwa_workflow_details['inputs'].keys()[0] #workflow > input ids > created_history = historyClient.create_history(name='testing_bwa2bctools2') > print 'created a new history with name ' , created_history['name'] path = "/home/sphadmin/data/fastq_files/" > files = {i:os.path.join(path,i) for i in os.listdir(path) if > os.path.splitext(i)[1] == '.fastq'} > hda = {i:toolClient.upload_file(files[i], created_history['id'], > type='fastq',dbkey='H37Rv') for i in files} > files_dataset_id = {i:hda[i]['outputs'][0]['id'] for i in files} > print 'uploading fastq files to history' > > fwd = re.compile(r'R1') > paired_list = {} > for i in files: > if fwd.search(i): > prefix, file = i.split('_R1')[0] , i > if not prefix in paired_list: > paired_list[prefix] = {} > paired_list[prefix]['forward'] = file > else: > prefix, file = i.split('_R2')[0], i > if not prefix in paired_list: > paired_list[prefix] = {} > paired_list[prefix]['reverse'] = file > > paired_collection = > [collections.CollectionElement(name=i,type='paired',elements=[collections.HistoryDatasetElement(name=j,id=files_dataset_id[paired_list[i][j]]) > for j in paired_list[i] ]) for i in paired_list] > collectionDesc = collections.CollectionDescription(name='My collection > list', type='list:paired', elements=paired_collection) > dataset_collection = > historyClient.create_dataset_collection(history_id=created_history['id'], > collection_description=collectionDesc) > print 'creating dataset collection in history' > data_wstep_id = {bwa_workflow_input_id: {'id': dataset_collection['id'], > 'src':'hda'}} > data_wstep_index = {"0": {'id': dataset_collection['id'], 'src':'hda'}} > exec_wf = workflowClient.invoke_workflow(bwa_workflow_id, > inputs=data_wstep_index,history_id=created_history['id']) > print 'submitting workflow' I have tried using the step_id with run_workflow() and get the following error, workflowClient.run_workflow(bwa_workflow_id, data_wstep_id, > history_id=created_history['id']) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File > "/usr/local/lib/python2.7/dist-packages/bioblend/galaxy/workflows/__init__.py", > line 271, in run_workflow > return Client._post(self, payload) > File "/usr/local/lib/python2.7/dist-packages/bioblend/galaxy/client.py", > line 157, in _post > files_attached=files_attached) > File "/usr/local/lib/python2.7/dist-packages/bioblend/galaxyclient.py", > line 134, in make_post_request > body=r.text, status_code=r.status_code) > bioblend.ConnectionError: Unexpected HTTP status code: 500: {"err_msg": > "Uncaught exception in exposed API method:", "err_code": 0} step_order_index with invoke_workflow() does not produce any error, but workflow is executed. So 'exec_wf' returns with failed 'status'. {u'inputs': {}, u'update_time': u'2017-05-11T09:13:31', u'uuid': > u'199adb04-362a-11e7-90e9-1cc1de6d5ef4', u'history_id': > u'fd767e8b7355e21a', u'state': u'failed', u'workflow_id': > u'1fad1eaf5f4f1766', u'steps': [{u'update_time': u'2017-05-11T09:13:31', > u'job_id': None, u'state': None, u'order_index': 0, u'action': None, > u'model_class': u'WorkflowInvocationStep', u'workflow_step_id': > u'a73186a6d4577bbf', u'id': u'd882ad2044776478'}], u'model_class': > u'WorkflowInvocation', u'id': u'964b37715ec9bd22'} However, If I run the workflow from UI, everything goes fine. Upon checking the invocation details of the workflow run from UI, the 'inputs' is still empty (there is no associated dataset collection id) {u'inputs': {}, u'update_time': u'2017-05-09T03:44:31', u'uuid': > u'c495fef0-3466-11e7-90e9-1cc1de6d5ef4', u'history_id': > u'f158df71bb77bb86', u'state': u'scheduled', u'workflow_id': > u'1fad1eaf5f4f1766', u'steps': [{u'update_time': u'2017-05-09T03:22:47', > u'job_id': u'48885bbd1215db53', u'state': u'ok', u'order_index': 1, > u'action': None, u'model_class': u'WorkflowInvocationStep', > u'workflow_step_id': u'4df0bfb0de34905a', u'id': u'50807609e10e339d'}, > {u'update_time': u'2017-05-09T03:22:47', u'job_id': None, u'state': None, > u'order_index': 0, u'action': None, u'model_class': > u'WorkflowInvocationStep', u'workflow_step_id': u'a73186a6d4577bbf', u'id': > u'f5e1495a2ebc6080'}, {u'update_time': u'2017-05-09T03:22:47', u'job_id': > u'f1cb3a6ff6a93c01', u'state': u'ok', u'order_index': 1, u'action': None, > u'model_class': u'WorkflowInvocationStep', u'workflow_step_id': > u'4df0bfb0de34905a', u'id': u'1d972c3873de3f79'}, {u'update_time': > u'2017-05-09T03:22:50', u'job_id': u'822aeedf25d30c76', u'state': u'ok', > u'order_index': 2, u'action': None, u'model_class': > u'WorkflowInvocationStep', u'workflow_step_id': u'89e52cb393afc192', u'id': > u'89788b3f6c8e75f4'}, {u'update_time': u'2017-05-09T03:22:50', u'job_id': > u'4fb563a6a1060ab1', u'state': u'ok', u'order_index': 2, u'action': None, > u'model_class': u'WorkflowInvocationStep', u'workflow_step_id': > u'89e52cb393afc192', u'id': u'b4666ccc5542d171'}, {u'update_time': > u'2017-05-09T03:22:53', u'job_id': u'cab68a1ccca17e68', u'state': u'ok', > u'order_index': 3, u'action': None, u'model_class': > u'WorkflowInvocationStep', u'workflow_step_id': u'b6b7934af4ebec63', u'id': > u'3dfe330b26f78ced'}, {u'update_time': u'2017-05-09T03:22:53', u'job_id': > u'bb80841fd9d212dd', u'state': u'ok', u'order_index': 3, u'action': None, > u'model_class': u'WorkflowInvocationStep', u'workflow_step_id': > u'b6b7934af4ebec63', u'id': u'f89ec36d748e8f90'}, {u'update_time': > u'2017-05-09T03:22:57', u'job_id': u'02b1cfb59f716d61', u'state': u'ok', > u'order_index': 4, u'action': None, u'model_class': > u'WorkflowInvocationStep', u'workflow_step_id': u'dba3bb6dfae451b8', u'id': > u'dbc267d11443f9a4'}, {u'update_time': u'2017-05-09T03:22:57', u'job_id': > u'705cc5c1fdaa7d3e', u'state': u'ok', u'order_index': 4, u'action': None, > u'model_class': u'WorkflowInvocationStep', u'workflow_step_id': > u'dba3bb6dfae451b8', u'id': u'67ce804af6ec796b'}, {u'update_time': > u'2017-05-09T03:22:59', u'job_id': u'665a42f7453c0065', u'state': u'ok', > u'order_index': 5, u'action': None, u'model_class': > u'WorkflowInvocationStep', u'workflow_step_id': u'5935f39e477d88b2', u'id': > u'09d8568e61ebd1d9'}, {u'update_time': u'2017-05-09T03:22:59', u'job_id': > u'4488b53dbfdabfab', u'state': u'ok', u'order_index': 5, u'action': None, > u'model_class': u'WorkflowInvocationStep', u'workflow_step_id': > u'5935f39e477d88b2', u'id': u'b3e4e0c76fe49b64'}], u'model_class': > u'WorkflowInvocation', u'id': u'e89067bb68bee7a0'} Galaxy Version 15.03 & bioblend Version 0.8.0. Appreciate your inputs. Thanks, Aarthi

1 0

python environment on SGE cluster nodes
by David Simpson 11 May '17

11 May '17

Hi, We're close to having a working Galaxy job_conf.xml to submit galaxy jobs to an SGE cluster. At the moment a submitted job is failing with: python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory We'd like to solve this by sourcing the dev tools for all our jobs: source /opt/rh/python27 enable Please could you advise on how to proceed? regards David -- David Simpson Systems Engineer MRC Centre for Neuropsychiatric Genetics and Genomics Cardiff University School of Medicine Hadyn Ellis Building Maindy Road Cardiff CF24 4HQ ------------------ phone 44 29206 88426 mail SimpsonD4 at Cardiff.ac.uk

1 3

creating roles through the API
by Lemley, Rob 09 May '17

09 May '17

Is there a way to create roles with the Galaxy API? -- Rob Lemley Systems Administrator - CIRC University of Rochester

1 0

strange submit script
by Nikolaos Tur 04 May '17

04 May '17

Hi, Galaxy 16.04, sge as batch system. There is strange submit script from one job which replicates whole Galaxy home directory to the working directory for jobs and does this 2 times. Am I correct that tool_script.sh file is generated automatically by Galaxy or still user can add something there? If it generates automatically then is this a bug? cat tool_script.sh: #!/bin/bash # The following block can be used by the job system # to ensure this script is runnable before actually attempting # to run it. if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then exit 42 fi PACKAGE_BASE=/home/galaxy/tools/dependencies/samtools/1.2/iuc/package_samtools_1_2/f6ae3ba3f3c1; export PACKAGE_BASE; . /home/galaxy/tools/dependencies/samtools/1.2/iuc/package_samtools_1_2/f6ae3ba3f3c1/env.sh; MEM=`expr "${GALAXY_SLOTS:-16}" \* 48 - 16` ; workdir=`pwd`; echo "workdir is $workdir"; cd $LOCAL; echo "Running Trinity from `pwd`"; Trinity --max_memory "${MEM}G" --CPU "${GALAXY_SLOTS:-16}" --bflyHeapSpaceMax "32G" --bflyHeapSpaceInit "2G" --bflyGCThreads "6" --left /home/galaxy/database/files/000/dataset_81417.dat --right /home/galaxy/database/files/000/dataset_81418.dat --seqType fq --SS_lib_type RF --group_pairs_distance 500 > /home/galaxy/database/files/000/dataset_81515.dat 2>&1 || (ec=$? ; cp -pr . $workdir; cd $workdir; cat /home/galaxy/database/files/000/dataset_81515.dat >&2 ; mkdir -p trinity_out_dir ; touch trinity_out_dir/Trinity.fasta ; exit $ec); cp -pr . $workdir; cd $workdir Output is: workdir is /home/galaxy/database/jobs_directory/000/51718/working Running Trinity from /home/galaxy It means that in "cd $LOCAL" LOCAL variable is undefined. And then there is a bunch of copies from home directory. Thanx for the help, Nikolaos

1 0

Re: [galaxy-dev] unable to install tools
by Briand, Sheldon (NRC/CNRC) 03 May '17

03 May '17

Hi, Here are, I think, the relevant lines from my nginx.conf: location / { proxy_pass http://galaxy_app; proxy_set_header X-Forwarded-Host $host:$server_port; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-URL-SCHEME https; } >From galaxy.ini: nginx_x_accel_redirect_base = /_x_accel_redirect nginx_x_archive_files_base = /_x_accel_redirect I have checked the start-up logs and both the admin_toolshed controllers are enabled on start-up. I've also tried a fresh python venv environment which didn't help. I was able to install tools in the past. Has anyone run into this recently? Thanks, -Sheldon From: Briand, Sheldon (NRC/CNRC) Sent: Tuesday, May 02, 2017 5:30 PM To: 'galaxy-dev(a)lists.galaxyproject.org' <galaxy-dev(a)lists.galaxyproject.org> Subject: unable to install tools Hi, I've updated to Galaxy 17.01 from Galaxy 16.07. Today I am having trouble installing or updating tools. Whenever I pick the option to install a tool from the main toolshed I get a page that says: "NOT FOUND The requested url /admin_toolshed/prepare_to_install was not found on this server" Is there a setting that I need to change on my server? I do run through a nginx proxy server. I can provide it's configuration if it's relevant. Thanks, -Sheldon Sheldon Briand Computer Systems and Applications Analyst National Research Council/Government of Canada Sheldon.briand(a)canada.ca/<mailto:Sheldon.briand@canada.ca/> Tel: (902) 426-1677

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

galaxy-dev May 2017