Configuring Galaxy, Pulsar and Conda
/dev/null galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:23,885 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A19.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" Cache location: There are no tarballs to remove galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,291 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda
/dev/null 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:25,251 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove
/dev/null 156.148.160.117 - - [19/Jan/2017:14:50:59 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A55.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,967 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove galaxy.jobs.runners DEBUG 2017-01-19 14:51:00,351 executing external set_meta script for job 67: [ "$CONDA_DEFAULT_ENV" = "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env" ] || . /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/activate '/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env' conda_activate.log 2>&1 &&GALAXY_LIB="/home/galaxy/galaxy_test_pulsar/lib"; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH"; else PYTHONPATH="$GALAXY_LIB"; fi; export PYTHONPATH; fi; GALAXY_VIRTUAL_ENV="/home/galaxy/galaxy_test_pulsar/.venv"; if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate"; fi; python "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py" "/home/galaxy/galaxy_test_pulsar/database/tmp/tmpqS9WNY" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/working/galaxy.json" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_in_HistoryDatasetAssociation_71_TIRfAD,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_kwds_HistoryDatasetAssociation_71_drHgz4,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_results_HistoryDatasetAssociation_71_E1N138,/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_71.dat,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_override_HistoryDatasetAssociation_71_iGnttD" 5242880 Traceback (most recent call last): File "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py",
Hi, I'm tryng to configure a Pulsar server to resolve dependencies using Conda but unfortunately it does not work. When I try to execute, for example, bowtie2, the standard error in the Galaxy machine says: "/home/galaxy/pulsar/files/staging/67/command.sh: line 60: bowtie2-build: command not found" because Conda in the Pulsar machine does not download this dependency. Here is my app.yml: dependency_resolvers_config_file: dependency_resolvers_conf.xml My dependency_resolvers_conf.xml <dependency_resolvers> <conda auto_install="True" auto_init="True" /> </dependency_resolvers> The log of the Galaxy server: galaxy.tools DEBUG 2017-01-19 14:50:18,910 Validated and populated state for tool request (97.265 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,069 Handled output named output for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 (81.926 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,113 Added output datasets to history (43.950 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,179 Verified access to datasets for Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] (33.690 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,180 Setup for job Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] complete, ready to flush (66.261 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,240 Flushed transaction for job Job[id=67,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] (59.840 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,240 Tool [toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] created job [67] (296.972 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,266 Executed 1 job(s) for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 request: (355.402 ms) 156.148.160.117 - - [19/Jan/2017:14:50:18 +0200] "POST /api/tools HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/webhooks/tool HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T12%3A11%3A40.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs DEBUG 2017-01-19 14:50:19,912 (67) Working directory for job is: /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67 galaxy.jobs.handler DEBUG 2017-01-19 14:50:19,931 (67) Dispatching to pulsar runner galaxy.jobs DEBUG 2017-01-19 14:50:20,013 (67) Persisting job destination (destination id: remote_cluster) galaxy.jobs.runners DEBUG 2017-01-19 14:50:20,044 Job [67] queued (112.446 ms) galaxy.jobs.handler INFO 2017-01-19 14:50:20,090 (67) Job dispatched galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:20,205 pulsar_version is 0.7.3 galaxy.util.object_wrapper WARNING 2017-01-19 14:50:20,644 Unable to create dynamic subclass for <type 'instance'>, None: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases galaxy.tools.deps DEBUG 2017-01-19 14:50:20,820 Find dependency bowtie2 version 2.3.0 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency bowtie2 version 2.3.0 of type conda galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Find dependency samtools version 1.3.1 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency samtools version 1.3.1 of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:20,822 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __bowtie2@2.3.0 --export > ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2@2.3.0 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:21,220 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda create -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2@2.3.0 list --name __samtools@1.3.1 --export > ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools@1.3.1 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,672 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools@1.3.1 pulsar.client.client DEBUG 2017-01-19 14:50:25,904 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_65.dat] (action_type: [transfer]) pulsar.client.client DEBUG 2017-01-19 14:50:25,913 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_59.dat] (action_type: [transfer]) 156.148.160.117 - - [19/Jan/2017:14:50:28 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A24.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:32 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A28.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:37 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A32.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:41 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A37.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:45 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A41.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:50 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A45.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:55,332 Pulsar job submitted with job_id 67 galaxy.jobs DEBUG 2017-01-19 14:50:55,333 (67) Persisting job destination (destination id: remote_cluster) 156.148.160.117 - - [19/Jan/2017:14:50:55 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A50.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" pulsar.client.staging.down INFO 2017-01-19 14:50:57,558 collecting output COMMAND_VERSION with action FileAction[action_type=transfer] pulsar.client.staging.down DEBUG 2017-01-19 14:50:57,562 Cleaning up job (failed [False], cleanup_job [always]) galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Find dependency samtools version None galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Using dependency samtools version None of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,177 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools@_uv_ --export > ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,558 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() ImportError: No module named galaxy_ext.metadata.set_metadata galaxy.jobs.runners DEBUG 2017-01-19 14:51:01,013 execution of external set_meta for job 67 finished galaxy.jobs INFO 2017-01-19 14:51:03,230 Collecting metrics for Job 67 galaxy.jobs DEBUG 2017-01-19 14:51:03,261 job 67 ended (finish() executed in (2246.565 ms)) galaxy.model.metadata DEBUG 2017-01-19 14:51:03,272 Cleaning up external metadata files galaxy.model.metadata DEBUG 2017-01-19 14:51:03,317 Failed to cleanup MetadataTempFile temp files from /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP: No JSON object could be decoded 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A59.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:07 +0200] "GET /api/histories/1cd8e2f6b131e891/contents/datasets/fa6833a4eadf9064 HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:16 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:22 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" /home/galaxy/galaxy_test_pulsar/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/default_comparator.py:153: SAWarning: The IN-predicate on "request_type.id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance. 'strategies for improved performance.' % expr) 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin/center?status=done&message= HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:47 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:50 +0200] "GET /admin_toolshed/browse_tool_sheds HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:52 +0200] "GET /admin_toolshed/browse_tool_shed?tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://156.148.120.96:8080/admin_toolshed/browse_tool_sheds" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:17,048 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:16 +0200] "GET / HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/history-menu/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/datatypes?extension_only=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/genomes HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /welcome HTTP/1.1" 302 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /history/current_history_json HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:20,083 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/users/f597429621d6eb2b HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?limit=500&offset=0&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=deleted&q=purged&q=visible&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:21 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:23 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?keys=type_id%2Celement_count&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=history_content_type&q=deleted&q=purged&q=visible&qv=dataset_collection&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:24 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:27 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" The log of the Pulsar server: Sourcing file ./local_env.sh Starting pulsar with command [paster server "./server.ini" ""] 2017-01-19 14:49:36,769 INFO [pulsar.core][MainThread] Starting the Pulsar without a toolbox to white-list.Ensure this application is protected by firewall or a configured private token. 2017-01-19 14:49:36,793 INFO [pulsar.locks][MainThread] pylockfile module not found, skipping experimental lockfile handling. Starting server in PID 12234. serving on 0.0.0.0:8913 view at http://127.0.0.1:8913 2017-01-19 14:50:31,715 DEBUG [pulsar.web.routes][worker 0] Setup job with configuration: {'tool_id': u'toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0', 'metadata_directory': '/home/galaxy/pulsar/files/staging/67/metadata', 'tool_version': u'2.3.0', 'path_separator': '/', 'unstructured_files_directory': '/home/galaxy/pulsar/files/staging/67/unstructured', 'tools_directory': '/home/galaxy/pulsar/files/staging/67/tool_files', 'working_directory': '/home/galaxy/pulsar/files/staging/67/working', 'inputs_directory': '/home/galaxy/pulsar/files/staging/67/inputs', 'job_directory': '/home/galaxy/pulsar/files/staging/67', 'system_properties': {'galaxy_home': '/home/galaxy/galaxy_test_pulsar', 'separator': '/'}, 'outputs_directory': '/home/galaxy/pulsar/files/staging/67/outputs', 'job_id': '67', 'configs_directory': '/home/galaxy/pulsar/files/staging/67/configs', 'pulsar_version': '0.7.3'} 2017-01-19 14:51:06,843 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=67]] job_id: 67 - Checking authorization of command_line [bowtie2 --version > /home/galaxy/pulsar/files/staging/67/outputs/COMMAND_VERSION 2>&1; bowtie2-build --threads ${GALAXY_SLOTS:-4} '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome && ln -s -f '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome.fa && bowtie2 -p ${GALAXY_SLOTS:-4} -x 'genome' -U '/home/galaxy/pulsar/files/staging/67/inputs/dataset_65.dat' | samtools sort -O bam -o '/home/galaxy/pulsar/files/staging/67/outputs/dataset_71.dat'; return_code=$?; sh -c "exit $return_code"] 2017-01-19 14:51:07,826 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=67]] collecting output COMMAND_VERSION with action FileAction[action_type=transfer] 2017-01-19 14:51:07,827 INFO [pulsar.managers.stateful][[manager=_default_]-[action=postprocess]-[job=67]] Status of job [67] changed to [complete]. No callbacks enabled. I think I'm doing something wrong in the configuration of the Pulsar server so I need some help or suggestion. Thank you -- Ricardo Medda CRS4 - Biomedicina Phone +39 070 9250 ??? Loc. Pixina Manna Edificio 1 Pula 09010 (CA), Italy Email: medda@crs4.it Website: www.crs4.it
Hello, Thanks for working on this - Pulsar still hasn't quite caught up with Galaxy in terms of support for Conda but we are getting there. So I noticed two things that should help today when I was trying to recreate your problem - the first is that Conda support in Pulsar requires this PR (https://github.com/galaxyproject/pulsar/pull/126) which isn't part of 0.7.3 - the pulsar you are targeting. I did an update of Pulsar though so that should help - release 0.7.4 should be available as of today. Additionally, I noticed that the default for the Pulsar client option "dependency_resolution" in Galaxy was broken. So to get this to work at all - you will need to update your job_conf.xml and specify that Pulsar should resolve dependencies and not Galaxy. This means adding <param id="dependency_resolution">remote</param> to your Pulsar destination. I opened a Pull Request here for the Galaxy change - so hopefully this will be the default by the time 17.01 is released. The logs should make it clear if there are additional problems after those changes I think. One think I would recommend is adding a versionless Conda resolver also - so you may want to make that dependency resolvers file look like this instead: <dependency_resolvers> <conda auto_install="True" auto_init="True" /> <conda auto_install="True" auto_init="True" versionless="True" /> </dependency_resolvers> This may not be strictly needed based on the tools you are attempting to get this to work for though. Thanks for your patience and let me know how it goes. -John On Fri, Feb 3, 2017 at 6:18 AM, ric <medda@crs4.it> wrote:
Hi, I'm tryng to configure a Pulsar server to resolve dependencies using Conda but unfortunately it does not work. When I try to execute, for example, bowtie2, the standard error in the Galaxy machine says: "/home/galaxy/pulsar/files/staging/67/command.sh: line 60: bowtie2-build: command not found" because Conda in the Pulsar machine does not download this dependency.
Here is my app.yml:
dependency_resolvers_config_file: dependency_resolvers_conf.xml
My dependency_resolvers_conf.xml
<dependency_resolvers> <conda auto_install="True" auto_init="True" /> </dependency_resolvers>
The log of the Galaxy server:
/dev/null galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:23,885 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A19.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" Cache location: There are no tarballs to remove galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,291 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools@1.3.1 --export > ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools@1.3.1 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,672 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools@1.3.1 /dev/null 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:25,251 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove
/dev/null 156.148.160.117 - - [19/Jan/2017:14:50:59 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A55.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,967 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove galaxy.jobs.runners DEBUG 2017-01-19 14:51:00,351 executing external set_meta script for job 67: [ "$CONDA_DEFAULT_ENV" = "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env" ] || . /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/activate '/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env' > conda_activate.log 2>&1 &&GALAXY_LIB="/home/galaxy/galaxy_test_pulsar/lib"; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH"; else PYTHONPATH="$GALAXY_LIB"; fi; export PYTHONPATH; fi; GALAXY_VIRTUAL_ENV="/home/galaxy/galaxy_test_pulsar/.venv"; if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate"; fi; python "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py" "/home/galaxy/galaxy_test_pulsar/database/tmp/tmpqS9WNY" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/working/galaxy.json" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_in_HistoryDatasetAssociation_71_TIRfAD,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_kwds_HistoryDatasetAssociation_71_drHgz4,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_results_HistoryDatasetAssociation_71_E1N138,/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_71.dat,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_override_HistoryDatasetAssociation_71_iGnttD" 5242880 Traceback (most recent call last): File "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py",
galaxy.tools DEBUG 2017-01-19 14:50:18,910 Validated and populated state for tool request (97.265 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,069 Handled output named output for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 (81.926 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,113 Added output datasets to history (43.950 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,179 Verified access to datasets for Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] (33.690 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,180 Setup for job Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] complete, ready to flush (66.261 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,240 Flushed transaction for job Job[id=67,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] (59.840 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,240 Tool [toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] created job [67] (296.972 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,266 Executed 1 job(s) for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 request: (355.402 ms) 156.148.160.117 - - [19/Jan/2017:14:50:18 +0200] "POST /api/tools HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/webhooks/tool HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T12%3A11%3A40.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs DEBUG 2017-01-19 14:50:19,912 (67) Working directory for job is: /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67 galaxy.jobs.handler DEBUG 2017-01-19 14:50:19,931 (67) Dispatching to pulsar runner galaxy.jobs DEBUG 2017-01-19 14:50:20,013 (67) Persisting job destination (destination id: remote_cluster) galaxy.jobs.runners DEBUG 2017-01-19 14:50:20,044 Job [67] queued (112.446 ms) galaxy.jobs.handler INFO 2017-01-19 14:50:20,090 (67) Job dispatched galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:20,205 pulsar_version is 0.7.3 galaxy.util.object_wrapper WARNING 2017-01-19 14:50:20,644 Unable to create dynamic subclass for <type 'instance'>, None: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases galaxy.tools.deps DEBUG 2017-01-19 14:50:20,820 Find dependency bowtie2 version 2.3.0 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency bowtie2 version 2.3.0 of type conda galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Find dependency samtools version 1.3.1 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency samtools version 1.3.1 of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:20,822 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __bowtie2@2.3.0 --export > ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2@2.3.0 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:21,220 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda create -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2@2.3.0 pulsar.client.client DEBUG 2017-01-19 14:50:25,904 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_65.dat] (action_type: [transfer]) pulsar.client.client DEBUG 2017-01-19 14:50:25,913 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_59.dat] (action_type: [transfer]) 156.148.160.117 - - [19/Jan/2017:14:50:28 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A24.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:32 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A28.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:37 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A32.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:41 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A37.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:45 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A41.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:50 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A45.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:55,332 Pulsar job submitted with job_id 67 galaxy.jobs DEBUG 2017-01-19 14:50:55,333 (67) Persisting job destination (destination id: remote_cluster) 156.148.160.117 - - [19/Jan/2017:14:50:55 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A50.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" pulsar.client.staging.down INFO 2017-01-19 14:50:57,558 collecting output COMMAND_VERSION with action FileAction[action_type=transfer] pulsar.client.staging.down DEBUG 2017-01-19 14:50:57,562 Cleaning up job (failed [False], cleanup_job [always]) galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Find dependency samtools version None galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Using dependency samtools version None of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,177 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools@_uv_ --export > ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,558 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() ImportError: No module named galaxy_ext.metadata.set_metadata galaxy.jobs.runners DEBUG 2017-01-19 14:51:01,013 execution of external set_meta for job 67 finished galaxy.jobs INFO 2017-01-19 14:51:03,230 Collecting metrics for Job 67 galaxy.jobs DEBUG 2017-01-19 14:51:03,261 job 67 ended (finish() executed in (2246.565 ms)) galaxy.model.metadata DEBUG 2017-01-19 14:51:03,272 Cleaning up external metadata files galaxy.model.metadata DEBUG 2017-01-19 14:51:03,317 Failed to cleanup MetadataTempFile temp files from /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP: No JSON object could be decoded 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A59.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:07 +0200] "GET /api/histories/1cd8e2f6b131e891/contents/datasets/fa6833a4eadf9064 HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:16 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:22 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" /home/galaxy/galaxy_test_pulsar/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/default_comparator.py:153: SAWarning: The IN-predicate on "request_type.id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance. 'strategies for improved performance.' % expr) 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin/center?status=done&message= HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:47 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:50 +0200] "GET /admin_toolshed/browse_tool_sheds HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:52 +0200] "GET /admin_toolshed/browse_tool_shed?tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://156.148.120.96:8080/admin_toolshed/browse_tool_sheds" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:17,048 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:16 +0200] "GET / HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/history-menu/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/datatypes?extension_only=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/genomes HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /welcome HTTP/1.1" 302 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /history/current_history_json HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:20,083 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/users/f597429621d6eb2b HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?limit=500&offset=0&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=deleted&q=purged&q=visible&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:21 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:23 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?keys=type_id%2Celement_count&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=history_content_type&q=deleted&q=purged&q=visible&qv=dataset_collection&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:24 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:27 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"
The log of the Pulsar server:
Sourcing file ./local_env.sh Starting pulsar with command [paster server "./server.ini" ""] 2017-01-19 14:49:36,769 INFO [pulsar.core][MainThread] Starting the Pulsar without a toolbox to white-list.Ensure this application is protected by firewall or a configured private token. 2017-01-19 14:49:36,793 INFO [pulsar.locks][MainThread] pylockfile module not found, skipping experimental lockfile handling. Starting server in PID 12234. serving on 0.0.0.0:8913 view at http://127.0.0.1:8913 2017-01-19 14:50:31,715 DEBUG [pulsar.web.routes][worker 0] Setup job with configuration: {'tool_id': u'toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0', 'metadata_directory': '/home/galaxy/pulsar/files/staging/67/metadata', 'tool_version': u'2.3.0', 'path_separator': '/', 'unstructured_files_directory': '/home/galaxy/pulsar/files/staging/67/unstructured', 'tools_directory': '/home/galaxy/pulsar/files/staging/67/tool_files', 'working_directory': '/home/galaxy/pulsar/files/staging/67/working', 'inputs_directory': '/home/galaxy/pulsar/files/staging/67/inputs', 'job_directory': '/home/galaxy/pulsar/files/staging/67', 'system_properties': {'galaxy_home': '/home/galaxy/galaxy_test_pulsar', 'separator': '/'}, 'outputs_directory': '/home/galaxy/pulsar/files/staging/67/outputs', 'job_id': '67', 'configs_directory': '/home/galaxy/pulsar/files/staging/67/configs', 'pulsar_version': '0.7.3'} 2017-01-19 14:51:06,843 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=67]] job_id: 67 - Checking authorization of command_line [bowtie2 --version > /home/galaxy/pulsar/files/staging/67/outputs/COMMAND_VERSION 2>&1; bowtie2-build --threads ${GALAXY_SLOTS:-4} '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome && ln -s -f '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome.fa && bowtie2 -p ${GALAXY_SLOTS:-4} -x 'genome' -U '/home/galaxy/pulsar/files/staging/67/inputs/dataset_65.dat' | samtools sort -O bam -o '/home/galaxy/pulsar/files/staging/67/outputs/dataset_71.dat'; return_code=$?; sh -c "exit $return_code"] 2017-01-19 14:51:07,826 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=67]] collecting output COMMAND_VERSION with action FileAction[action_type=transfer] 2017-01-19 14:51:07,827 INFO [pulsar.managers.stateful][[manager=_default_]-[action=postprocess]-[job=67]] Status of job [67] changed to [complete]. No callbacks enabled.
I think I'm doing something wrong in the configuration of the Pulsar server so I need some help or suggestion.
Thank you
-- Ricardo Medda CRS4 - Biomedicina Phone +39 070 9250 ??? Loc. Pixina Manna Edificio 1 Pula 09010 (CA), Italy Email: medda@crs4.it Website: www.crs4.it
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi John, thank you for your help. Now it works! Il 07/02/2017 21:20, John Chilton ha scritto:
Hello,
Thanks for working on this - Pulsar still hasn't quite caught up with Galaxy in terms of support for Conda but we are getting there. So I noticed two things that should help today when I was trying to recreate your problem - the first is that Conda support in Pulsar requires this PR (https://github.com/galaxyproject/pulsar/pull/126) which isn't part of 0.7.3 - the pulsar you are targeting. I did an update of Pulsar though so that should help - release 0.7.4 should be available as of today.
Additionally, I noticed that the default for the Pulsar client option "dependency_resolution" in Galaxy was broken. So to get this to work at all - you will need to update your job_conf.xml and specify that Pulsar should resolve dependencies and not Galaxy. This means adding <param id="dependency_resolution">remote</param> to your Pulsar destination. I opened a Pull Request here for the Galaxy change - so hopefully this will be the default by the time 17.01 is released.
The logs should make it clear if there are additional problems after those changes I think. One think I would recommend is adding a versionless Conda resolver also - so you may want to make that dependency resolvers file look like this instead:
<dependency_resolvers> <conda auto_install="True" auto_init="True" /> <conda auto_install="True" auto_init="True" versionless="True" /> </dependency_resolvers>
This may not be strictly needed based on the tools you are attempting to get this to work for though.
Thanks for your patience and let me know how it goes.
-John
On Fri, Feb 3, 2017 at 6:18 AM, ric <medda@crs4.it> wrote:
Hi, I'm tryng to configure a Pulsar server to resolve dependencies using Conda but unfortunately it does not work. When I try to execute, for example, bowtie2, the standard error in the Galaxy machine says: "/home/galaxy/pulsar/files/staging/67/command.sh: line 60: bowtie2-build: command not found" because Conda in the Pulsar machine does not download this dependency.
Here is my app.yml:
dependency_resolvers_config_file: dependency_resolvers_conf.xml
My dependency_resolvers_conf.xml
<dependency_resolvers> <conda auto_install="True" auto_init="True" /> </dependency_resolvers>
The log of the Galaxy server:
/dev/null galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:23,885 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A19.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" Cache location: There are no tarballs to remove galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,291 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools@1.3.1 --export > ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools@1.3.1 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,672 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools@1.3.1 /dev/null 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:25,251 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove
/dev/null 156.148.160.117 - - [19/Jan/2017:14:50:59 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A55.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,967 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove galaxy.jobs.runners DEBUG 2017-01-19 14:51:00,351 executing external set_meta script for job 67: [ "$CONDA_DEFAULT_ENV" = "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env" ] || . /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/activate '/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env' > conda_activate.log 2>&1 &&GALAXY_LIB="/home/galaxy/galaxy_test_pulsar/lib"; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH"; else PYTHONPATH="$GALAXY_LIB"; fi; export PYTHONPATH; fi; GALAXY_VIRTUAL_ENV="/home/galaxy/galaxy_test_pulsar/.venv"; if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate"; fi; python "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py" "/home/galaxy/galaxy_test_pulsar/database/tmp/tmpqS9WNY" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/working/galaxy.json" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_in_HistoryDatasetAssociation_71_TIRfAD,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_kwds_HistoryDatasetAssociation_71_drHgz4,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_results_HistoryDatasetAssociation_71_E1N138,/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_71.dat,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_override_HistoryDatasetAssociation_71_iGnttD" 5242880 Traceback (most recent call last): File "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py",
galaxy.tools DEBUG 2017-01-19 14:50:18,910 Validated and populated state for tool request (97.265 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,069 Handled output named output for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 (81.926 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,113 Added output datasets to history (43.950 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,179 Verified access to datasets for Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] (33.690 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,180 Setup for job Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] complete, ready to flush (66.261 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,240 Flushed transaction for job Job[id=67,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] (59.840 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,240 Tool [toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] created job [67] (296.972 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,266 Executed 1 job(s) for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 request: (355.402 ms) 156.148.160.117 - - [19/Jan/2017:14:50:18 +0200] "POST /api/tools HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/webhooks/tool HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T12%3A11%3A40.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs DEBUG 2017-01-19 14:50:19,912 (67) Working directory for job is: /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67 galaxy.jobs.handler DEBUG 2017-01-19 14:50:19,931 (67) Dispatching to pulsar runner galaxy.jobs DEBUG 2017-01-19 14:50:20,013 (67) Persisting job destination (destination id: remote_cluster) galaxy.jobs.runners DEBUG 2017-01-19 14:50:20,044 Job [67] queued (112.446 ms) galaxy.jobs.handler INFO 2017-01-19 14:50:20,090 (67) Job dispatched galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:20,205 pulsar_version is 0.7.3 galaxy.util.object_wrapper WARNING 2017-01-19 14:50:20,644 Unable to create dynamic subclass for <type 'instance'>, None: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases galaxy.tools.deps DEBUG 2017-01-19 14:50:20,820 Find dependency bowtie2 version 2.3.0 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency bowtie2 version 2.3.0 of type conda galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Find dependency samtools version 1.3.1 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency samtools version 1.3.1 of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:20,822 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __bowtie2@2.3.0 --export > ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2@2.3.0 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:21,220 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda create -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2@2.3.0 pulsar.client.client DEBUG 2017-01-19 14:50:25,904 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_65.dat] (action_type: [transfer]) pulsar.client.client DEBUG 2017-01-19 14:50:25,913 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_59.dat] (action_type: [transfer]) 156.148.160.117 - - [19/Jan/2017:14:50:28 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A24.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:32 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A28.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:37 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A32.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:41 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A37.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:45 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A41.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:50 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A45.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:55,332 Pulsar job submitted with job_id 67 galaxy.jobs DEBUG 2017-01-19 14:50:55,333 (67) Persisting job destination (destination id: remote_cluster) 156.148.160.117 - - [19/Jan/2017:14:50:55 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A50.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" pulsar.client.staging.down INFO 2017-01-19 14:50:57,558 collecting output COMMAND_VERSION with action FileAction[action_type=transfer] pulsar.client.staging.down DEBUG 2017-01-19 14:50:57,562 Cleaning up job (failed [False], cleanup_job [always]) galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Find dependency samtools version None galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Using dependency samtools version None of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,177 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools@_uv_ --export > ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,558 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() ImportError: No module named galaxy_ext.metadata.set_metadata galaxy.jobs.runners DEBUG 2017-01-19 14:51:01,013 execution of external set_meta for job 67 finished galaxy.jobs INFO 2017-01-19 14:51:03,230 Collecting metrics for Job 67 galaxy.jobs DEBUG 2017-01-19 14:51:03,261 job 67 ended (finish() executed in (2246.565 ms)) galaxy.model.metadata DEBUG 2017-01-19 14:51:03,272 Cleaning up external metadata files galaxy.model.metadata DEBUG 2017-01-19 14:51:03,317 Failed to cleanup MetadataTempFile temp files from /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP: No JSON object could be decoded 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A59.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:07 +0200] "GET /api/histories/1cd8e2f6b131e891/contents/datasets/fa6833a4eadf9064 HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:16 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:22 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" /home/galaxy/galaxy_test_pulsar/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/default_comparator.py:153: SAWarning: The IN-predicate on "request_type.id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance. 'strategies for improved performance.' % expr) 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin/center?status=done&message= HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:47 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:50 +0200] "GET /admin_toolshed/browse_tool_sheds HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:52 +0200] "GET /admin_toolshed/browse_tool_shed?tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://156.148.120.96:8080/admin_toolshed/browse_tool_sheds" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:17,048 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:16 +0200] "GET / HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/history-menu/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/datatypes?extension_only=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/genomes HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /welcome HTTP/1.1" 302 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /history/current_history_json HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:20,083 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/users/f597429621d6eb2b HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?limit=500&offset=0&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=deleted&q=purged&q=visible&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:21 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:23 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?keys=type_id%2Celement_count&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=history_content_type&q=deleted&q=purged&q=visible&qv=dataset_collection&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:24 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:27 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"
The log of the Pulsar server:
Sourcing file ./local_env.sh Starting pulsar with command [paster server "./server.ini" ""] 2017-01-19 14:49:36,769 INFO [pulsar.core][MainThread] Starting the Pulsar without a toolbox to white-list.Ensure this application is protected by firewall or a configured private token. 2017-01-19 14:49:36,793 INFO [pulsar.locks][MainThread] pylockfile module not found, skipping experimental lockfile handling. Starting server in PID 12234. serving on 0.0.0.0:8913 view at http://127.0.0.1:8913 2017-01-19 14:50:31,715 DEBUG [pulsar.web.routes][worker 0] Setup job with configuration: {'tool_id': u'toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0', 'metadata_directory': '/home/galaxy/pulsar/files/staging/67/metadata', 'tool_version': u'2.3.0', 'path_separator': '/', 'unstructured_files_directory': '/home/galaxy/pulsar/files/staging/67/unstructured', 'tools_directory': '/home/galaxy/pulsar/files/staging/67/tool_files', 'working_directory': '/home/galaxy/pulsar/files/staging/67/working', 'inputs_directory': '/home/galaxy/pulsar/files/staging/67/inputs', 'job_directory': '/home/galaxy/pulsar/files/staging/67', 'system_properties': {'galaxy_home': '/home/galaxy/galaxy_test_pulsar', 'separator': '/'}, 'outputs_directory': '/home/galaxy/pulsar/files/staging/67/outputs', 'job_id': '67', 'configs_directory': '/home/galaxy/pulsar/files/staging/67/configs', 'pulsar_version': '0.7.3'} 2017-01-19 14:51:06,843 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=67]] job_id: 67 - Checking authorization of command_line [bowtie2 --version > /home/galaxy/pulsar/files/staging/67/outputs/COMMAND_VERSION 2>&1; bowtie2-build --threads ${GALAXY_SLOTS:-4} '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome && ln -s -f '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome.fa && bowtie2 -p ${GALAXY_SLOTS:-4} -x 'genome' -U '/home/galaxy/pulsar/files/staging/67/inputs/dataset_65.dat' | samtools sort -O bam -o '/home/galaxy/pulsar/files/staging/67/outputs/dataset_71.dat'; return_code=$?; sh -c "exit $return_code"] 2017-01-19 14:51:07,826 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=67]] collecting output COMMAND_VERSION with action FileAction[action_type=transfer] 2017-01-19 14:51:07,827 INFO [pulsar.managers.stateful][[manager=_default_]-[action=postprocess]-[job=67]] Status of job [67] changed to [complete]. No callbacks enabled.
I think I'm doing something wrong in the configuration of the Pulsar server so I need some help or suggestion.
Thank you
-- Ricardo Medda CRS4 - Biomedicina Phone +39 070 9250 ??? Loc. Pixina Manna Edificio 1 Pula 09010 (CA), Italy Email: medda@crs4.it Website: www.crs4.it
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Ricardo Medda CRS4 - Biomedicina Phone +39 070 9250 ??? Loc. Pixina Manna Edificio 1 Pula 09010 (CA), Italy Email: medda@crs4.it Website: www.crs4.it
participants (2)
-
John Chilton -
ric