- galaxy-dev - lists.galaxyproject.org

Relabeling dataset pairs in 'list:paired' collection
by Peter Briggs 14 Feb '17

14 Feb '17

Dear Developers Is there an existing tool or mechanism that can be used to duplicate a "list of pairs" dataset collection, keeping the paired datasets the same but relabeling each pair with a new identifier taken from a user supplied file or list? I've cobbled together my own tool to try and do something like this: https://github.com/pjbriggs/Amplicon_analysis-galaxy/blob/77340d8bb2470a646… and while it works, it doesn't feel like a good solution as it creates duplicates of the datasets from the first collection and consumes additional disk/quota space unnecessarily. (This is particularly undesirable as we expect that the input collections might be relatively large numbers of FASTQ pairs e.g. 30 or more.) Looking at some of the 'Collection Operations' tools that come with Galaxy, it appears that these are able to create new collections without making duplicate datasets, which seems much better. But these tools work by directly invoking Python classes from the Galaxy core, so I don't know if a similar approach could be used in a non-core tool. Any advice or suggestions are very welcome! Thanks Best wishes Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

2 2

Error in fresh galaxy install
by evan clark 13 Feb '17

13 Feb '17

I am setting up a new galaxy instance and upon intial setup and user login I get this error. The database is brand new and there is no data in any history as I am the sole user at this moment. "onLine": true, "version": "17.05", "xhr": { "readyState": 4, "responseText": "{\"err_msg\": \"History is not accessible by user\", \"err_code\": 403002}", "responseJSON": { "err_msg": "History is not accessible by user", "err_code": 403002 }, "status": 403, "statusText": "Forbidden" }, "options": { "parse": true, "keys": "type_id,element_count", "filters": { "history_content_type": "dataset_collection" }, "remove": false, "traditional": true, "data": { "keys": "type_id,element_count", "order": "hid", "v": "dev", "q": [ "history_content_type", "deleted", "purged", "visible" ], "qv": [ "dataset_collection", "False", "False", "True" ] }, "emulateHTTP": false, "emulateJSON": false, "textStatus": "error", "errorThrown": "Forbidden" },

1 0

Tools using Galaxy's Python library?
by Peter Cock 13 Feb '17

13 Feb '17

Hi all, A few of my tools have for a long time used Galaxy's own parsing functionality in order to avoid an external dependency. Lately this has stopped working on my TravisCI testing with planemo using the Galaxy dev branch (the stable master branch is fine): e.g. https://travis-ci.org/peterjc/pico_galaxy/builds/196655736 The tools fail with things like: | from galaxy_utils.sequence.fasta import fastaReader, fastaWriter | ImportError: No module named galaxy_utils.sequence.fasta or: | from galaxy_utils.sequence.fastq import fastqReader | ImportError: No module named galaxy_utils.sequence.fastq Is this a temporary regression in Galaxy, or a deliberate change? Do the tools need to do something to explicit have access to the Galaxy Python library, or are they now considered private? If so, I can update these tools to use an explicit dependency for parsing FASTA and FASTQ (e.g. Biopython). Thanks, Peter

4 8

regarding tool dependencies
by Nir Shachaf 09 Feb '17

09 Feb '17

Dear Galaxy-dev team, As a research scientist in the dept. of Plant Sciences I would like to contribute a tool to the 'Metabolomics' tool-box under Galaxy. The tool is principally written in R, with some dependencies (namely: R>=3.x, and several R packages). Following the documentation in: https://wiki.galaxyproject.org/Admin/Tools/ToolDependencies https://wiki.galaxyproject.org/Admin/Config/ToolDependencies I admit I get confused by the numerous possibilities offered by the framework. My main concern is to be able to link the Galaxy server to tools already installed on my development environment, and NOT to depend on the tools available in the tool-shed repository (as many are not there, and there are specific versions requirements). Could you kindly direct me to the relevant solution, or a user's-list post, if applicable? Many thanks in advance, Nir Shachaf, Plant Sciences Dept., Weizmann Inst. of Science, Rehovot, Israel.

2 1

Configuring Galaxy, Pulsar and Conda
by ric 09 Feb '17

09 Feb '17

Hi, I'm tryng to configure a Pulsar server to resolve dependencies using Conda but unfortunately it does not work. When I try to execute, for example, bowtie2, the standard error in the Galaxy machine says: "/home/galaxy/pulsar/files/staging/67/command.sh: line 60: bowtie2-build: command not found" because Conda in the Pulsar machine does not download this dependency. Here is my app.yml: dependency_resolvers_config_file: dependency_resolvers_conf.xml My dependency_resolvers_conf.xml <dependency_resolvers> <conda auto_install="True" auto_init="True" /> </dependency_resolvers> The log of the Galaxy server: galaxy.tools DEBUG 2017-01-19 14:50:18,910 Validated and populated state for tool request (97.265 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,069 Handled output named output for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 (81.926 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,113 Added output datasets to history (43.950 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,179 Verified access to datasets for Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/… (33.690 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,180 Setup for job Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/… complete, ready to flush (66.261 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,240 Flushed transaction for job Job[id=67,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.… (59.840 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,240 Tool [toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] created job [67] (296.972 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,266 Executed 1 job(s) for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 request: (355.402 ms) 156.148.160.117 - - [19/Jan/2017:14:50:18 +0200] "POST /api/tools HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/webhooks/tool HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T12%3A11%3A40.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs DEBUG 2017-01-19 14:50:19,912 (67) Working directory for job is: /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67 galaxy.jobs.handler DEBUG 2017-01-19 14:50:19,931 (67) Dispatching to pulsar runner galaxy.jobs DEBUG 2017-01-19 14:50:20,013 (67) Persisting job destination (destination id: remote_cluster) galaxy.jobs.runners DEBUG 2017-01-19 14:50:20,044 Job [67] queued (112.446 ms) galaxy.jobs.handler INFO 2017-01-19 14:50:20,090 (67) Job dispatched galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:20,205 pulsar_version is 0.7.3 galaxy.util.object_wrapper WARNING 2017-01-19 14:50:20,644 Unable to create dynamic subclass for <type 'instance'>, None: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases galaxy.tools.deps DEBUG 2017-01-19 14:50:20,820 Find dependency bowtie2 version 2.3.0 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency bowtie2 version 2.3.0 of type conda galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Find dependency samtools version 1.3.1 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency samtools version 1.3.1 of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:20,822 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __bowtie2(a)2.3.0 --export > ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2(a)2.3.0 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:21,220 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda create -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2(a)2.3.0 > /dev/null galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:23,885 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A19.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" Cache location: There are no tarballs to remove galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,291 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools(a)1.3.1 --export > ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools(a)1.3.1 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,672 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools(a)1.3.1 > /dev/null 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:25,251 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove pulsar.client.client DEBUG 2017-01-19 14:50:25,904 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_65.dat] (action_type: [transfer]) pulsar.client.client DEBUG 2017-01-19 14:50:25,913 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_59.dat] (action_type: [transfer]) 156.148.160.117 - - [19/Jan/2017:14:50:28 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A24.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:32 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A28.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:37 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A32.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:41 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A37.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:45 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A41.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:50 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A45.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:55,332 Pulsar job submitted with job_id 67 galaxy.jobs DEBUG 2017-01-19 14:50:55,333 (67) Persisting job destination (destination id: remote_cluster) 156.148.160.117 - - [19/Jan/2017:14:50:55 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A50.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" pulsar.client.staging.down INFO 2017-01-19 14:50:57,558 collecting output COMMAND_VERSION with action FileAction[action_type=transfer] pulsar.client.staging.down DEBUG 2017-01-19 14:50:57,562 Cleaning up job (failed [False], cleanup_job [always]) galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Find dependency samtools version None galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Using dependency samtools version None of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,177 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools@_uv_ --export > ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,558 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ > /dev/null 156.148.160.117 - - [19/Jan/2017:14:50:59 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A55.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,967 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove galaxy.jobs.runners DEBUG 2017-01-19 14:51:00,351 executing external set_meta script for job 67: [ "$CONDA_DEFAULT_ENV" = "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env" ] || . /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/activate '/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env' > conda_activate.log 2>&1 &&GALAXY_LIB="/home/galaxy/galaxy_test_pulsar/lib"; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH"; else PYTHONPATH="$GALAXY_LIB"; fi; export PYTHONPATH; fi; GALAXY_VIRTUAL_ENV="/home/galaxy/galaxy_test_pulsar/.venv"; if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate"; fi; python "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py" "/home/galaxy/galaxy_test_pulsar/database/tmp/tmpqS9WNY" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/working/galaxy.json" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_in_HistoryDatasetAssociation_71_TIRfAD,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_kwds_HistoryDatasetAssociation_71_drHgz4,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_results_HistoryDatasetAssociation_71_E1N138,/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_71.dat,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_override_HistoryDatasetAssociation_71_iGnttD" 5242880 Traceback (most recent call last): File "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py", line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() ImportError: No module named galaxy_ext.metadata.set_metadata galaxy.jobs.runners DEBUG 2017-01-19 14:51:01,013 execution of external set_meta for job 67 finished galaxy.jobs INFO 2017-01-19 14:51:03,230 Collecting metrics for Job 67 galaxy.jobs DEBUG 2017-01-19 14:51:03,261 job 67 ended (finish() executed in (2246.565 ms)) galaxy.model.metadata DEBUG 2017-01-19 14:51:03,272 Cleaning up external metadata files galaxy.model.metadata DEBUG 2017-01-19 14:51:03,317 Failed to cleanup MetadataTempFile temp files from /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP: No JSON object could be decoded 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A59.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:07 +0200] "GET /api/histories/1cd8e2f6b131e891/contents/datasets/fa6833a4eadf9064 HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:16 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:22 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" /home/galaxy/galaxy_test_pulsar/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/default_comparator.py:153: SAWarning: The IN-predicate on "request_type.id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance. 'strategies for improved performance.' % expr) 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin/center?status=done&message= HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:47 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:50 +0200] "GET /admin_toolshed/browse_tool_sheds HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:52 +0200] "GET /admin_toolshed/browse_tool_shed?tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://156.148.120.96:8080/admin_toolshed/browse_tool_sheds" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:17,048 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:16 +0200] "GET / HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/history-menu/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/datatypes?extension_only=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/genomes HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /welcome HTTP/1.1" 302 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /history/current_history_json HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:20,083 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/users/f597429621d6eb2b HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?limit=500&offset=0&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=deleted&q=purged&q=visible&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:21 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:23 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?keys=type_id%2Celement_count&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=history_content_type&q=deleted&q=purged&q=visible&qv=dataset_collection&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:24 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:27 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" The log of the Pulsar server: Sourcing file ./local_env.sh Starting pulsar with command [paster server "./server.ini" ""] 2017-01-19 14:49:36,769 INFO [pulsar.core][MainThread] Starting the Pulsar without a toolbox to white-list.Ensure this application is protected by firewall or a configured private token. 2017-01-19 14:49:36,793 INFO [pulsar.locks][MainThread] pylockfile module not found, skipping experimental lockfile handling. Starting server in PID 12234. serving on 0.0.0.0:8913 view at http://127.0.0.1:8913 2017-01-19 14:50:31,715 DEBUG [pulsar.web.routes][worker 0] Setup job with configuration: {'tool_id': u'toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0', 'metadata_directory': '/home/galaxy/pulsar/files/staging/67/metadata', 'tool_version': u'2.3.0', 'path_separator': '/', 'unstructured_files_directory': '/home/galaxy/pulsar/files/staging/67/unstructured', 'tools_directory': '/home/galaxy/pulsar/files/staging/67/tool_files', 'working_directory': '/home/galaxy/pulsar/files/staging/67/working', 'inputs_directory': '/home/galaxy/pulsar/files/staging/67/inputs', 'job_directory': '/home/galaxy/pulsar/files/staging/67', 'system_properties': {'galaxy_home': '/home/galaxy/galaxy_test_pulsar', 'separator': '/'}, 'outputs_directory': '/home/galaxy/pulsar/files/staging/67/outputs', 'job_id': '67', 'configs_directory': '/home/galaxy/pulsar/files/staging/67/configs', 'pulsar_version': '0.7.3'} 2017-01-19 14:51:06,843 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=67]] job_id: 67 - Checking authorization of command_line [bowtie2 --version > /home/galaxy/pulsar/files/staging/67/outputs/COMMAND_VERSION 2>&1; bowtie2-build --threads ${GALAXY_SLOTS:-4} '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome && ln -s -f '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome.fa && bowtie2 -p ${GALAXY_SLOTS:-4} -x 'genome' -U '/home/galaxy/pulsar/files/staging/67/inputs/dataset_65.dat' | samtools sort -O bam -o '/home/galaxy/pulsar/files/staging/67/outputs/dataset_71.dat'; return_code=$?; sh -c "exit $return_code"] 2017-01-19 14:51:07,826 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=67]] collecting output COMMAND_VERSION with action FileAction[action_type=transfer] 2017-01-19 14:51:07,827 INFO [pulsar.managers.stateful][[manager=_default_]-[action=postprocess]-[job=67]] Status of job [67] changed to [complete]. No callbacks enabled. I think I'm doing something wrong in the configuration of the Pulsar server so I need some help or suggestion. Thank you -- Ricardo Medda CRS4 - Biomedicina Phone +39 070 9250 ??? Loc. Pixina Manna Edificio 1 Pula 09010 (CA), Italy Email: medda(a)crs4.it Website: www.crs4.it

2 2

Create a new datatype svg
by Jochen Bick 09 Feb '17

09 Feb '17

Hi all, I'm working on a tool that will have svg as an output file. Is this data type already established? If not how do I start? Cheers Jochen

4 9

Pulsar Metadata
by Joseph Brent Greer 07 Feb '17

07 Feb '17

Hi All, We've been using Pulsar to send jobs to our cluster on campus. Generally we send a dataset list of files over. We're wondering if there is a way to get the metadata for the files after they've been sent to the cluster. We're specifically trying to find the filepaths of the files before they were through Pulsar to the cluster. Any guidance would be great. Thanks, Joe

2 1

GalaxyAdmins Feb online meetup (Enis Afgan)
by David Kovalic 07 Feb '17

07 Feb '17

Enis, Could you send the call-in information please? David On Tue, Feb 7, 2017 11:00 AM, galaxy-dev-request(a)lists.galaxyproject.org wrote: Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.galaxyproject.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.galaxyproject.org/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.galaxyproject.org You can reach the person managing the list at galaxy-dev-owner(a)lists.galaxyproject.org When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative.Today's Topics: 1. Re: Request more details from Nikolaos Tur about pyBamTools & pyBamParser dependency packages message (Peter Briggs) 2. GalaxyAdmins Feb online meetup (Enis Afgan) 3. Create a new datatype svg (Jochen Bick) 4. Re: Create a new datatype svg (Björn Grüning) 5. Re: Create a new datatype svg (Jochen Bick) 6. Re: Request more details from Nikolaos Tur about pyBamTools & pyBamParser dependency packages message (Jennifer Hsieh) 7. SLURM configuration problem (Leonor Palmeira) 8. Re: Issue when using xsendfile with galaxy release_16.07 (SAPET, Frederic) _______________________________________________ galaxy-dev mailing list galaxy-dev(a)lists.galaxyproject.org https://lists.galaxyproject.org/listinfo/galaxy-dev To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

1 0

Vote for the Training you would like to see at GCC2017
by Dave Clements 07 Feb '17

07 Feb '17

Hello potential GCC2017 Attendees! The 2017 Galaxy Community Conference (GCC2017) <https://gcc2017.sciencesconf.org/> will be held 26-30 June, in Montpellier, France. GCC2017 training <https://gcc2017.sciencesconf.org/> will be held 27-28 June. *We need your vote to determine what to offer at GCC2017 training.* If there is even a remote chance that you will attend GCC2017 (it's in the South of France; it will be summer; don't let that influence your decision), then *please vote on which of the **34 nominated topics you would like to see offered at GCC2017.* *Vote Now! <https://docs.google.com/forms/d/e/1FAIpQLSdpkLoRuCGZOO3_T6dgRwFeDxVjb_zW-tl…>* 27 June will be single track and focus on how to use Galaxy for research. 28 June will have multiple tracks, and offer the full spectrum of Galaxy training. Vote for as many topics as you like, but please note that the more topics you vote for, the less your vote for each single topic will count. *Voting closes on 8 February.* Once that happens the organisers will scramble to confirm instructors and set a schedule. The training schedule will be posted before registration opens in February. The nominated topics this year are: - *Using Galaxy Topic Nominations <https://gcc2017.sciencesconf.org/resource/page/id/8#using>* 1. Galaxy 101 - A gentle introduction to Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#galaxy_101> 2. Galaxy at scale <https://gcc2017.sciencesconf.org/resource/page/id/8#galaxy_at_scale> 3. Visualisation of BIG DATA in Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#visualisation_of_big_da…> 4. GSuite HyperBrowser: integrative analysis of dataset collections across the genome and epigenome <https://gcc2017.sciencesconf.org/resource/page/id/8#gsuite_hyperbrowser> 5. ChIPseq analysis using DeepTools and MACS <https://gcc2017.sciencesconf.org/resource/page/id/8#chipseq> 6. RADseq Data Analysis Through STACKS on Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#radseq> 7. RNAseq analysis in Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#rnaseq> 8. Flow Cytometry Analysis with ImmPort Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#flow_cytometry> 9. Metabolomics Analysis with Galaxy on W4M infrastructure <https://gcc2017.sciencesconf.org/resource/page/id/8#metabolomics> 10. How to analyze metagenomic and amplicon data in Galaxy? <https://gcc2017.sciencesconf.org/resource/page/id/8#metagenomics> 11. Galaxy for Proteomics <https://gcc2017.sciencesconf.org/resource/page/id/8#proteomics> 12. Processing and Analysis of Hi-C data using HiCExplorer <https://gcc2017.sciencesconf.org/resource/page/id/8#hic> 13. SNP analysis and GWAS using SNiPlay <https://gcc2017.sciencesconf.org/resource/page/id/8#snp> 14. Small genome de novo assembly using Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#assembly> 15. Scripting Galaxy using the API and BioBlend <https://gcc2017.sciencesconf.org/resource/page/id/8#scripting> - *Deploying, Administering, and Extending Galaxy Topics <https://gcc2017.sciencesconf.org/resource/page/id/8#admin>* 1. Introduction to Galaxy admin: Setting up a Galaxy instance as a service <https://gcc2017.sciencesconf.org/resource/page/id/8#intro_admin> 2. Advanced customisation of a Galaxy instance <https://gcc2017.sciencesconf.org/resource/page/id/8#custom> 3. Advanced accelerated Galaxy admin <https://gcc2017.sciencesconf.org/resource/page/id/8#aaga> 4. How to use Galaxy Ansible Playbooks <https://gcc2017.sciencesconf.org/resource/page/id/8#ansible> 5. Testing Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#testing> 6. Using and deploying your application to the cloud <https://gcc2017.sciencesconf.org/resource/page/id/8#cloud> 7. Galaxy Architecture <https://gcc2017.sciencesconf.org/resource/page/id/8#arch> 8. Writing & Publishing Galaxy Tools <https://gcc2017.sciencesconf.org/resource/page/id/8#tools_basic> 9. Advanced Topics in Galaxy Tool Development <https://gcc2017.sciencesconf.org/resource/page/id/8#tools_advanced> 10. Visualisation Development in Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#viz_dep> 11. Adding support for new external sources <https://gcc2017.sciencesconf.org/resource/page/id/8#data_sources> 12. Conda and Containers for Tool Dependencies - A Developers Perspective <https://gcc2017.sciencesconf.org/resource/page/id/8#conda_dev> 13. Conda and Containers for Tool Dependencies - A Deployers Perspective <https://gcc2017.sciencesconf.org/resource/page/id/8#conda_dep> 14. Introduction to Galaxy Interactive Environments <https://gcc2017.sciencesconf.org/resource/page/id/8#ie_intro> 15. Advanced topics in Galaxy Interactive Environments <https://gcc2017.sciencesconf.org/resource/page/id/8#ie_adv> 16. Galaxy and Tripal - coupling Galaxy to a community database <https://gcc2017.sciencesconf.org/resource/page/id/8#tripal> 17. The Galaxy Genome Annotation project: Dockerized genome annotation environment based on Galaxy and GMOD tools <https://gcc2017.sciencesconf.org/resource/page/id/8#gap> 18. Using Hadoop with Galaxy <https://gcc2017.sciencesconf.org/resource/page/id/8#hadoop> - *Trainer Training Topics <https://gcc2017.sciencesconf.org/resource/page/id/8#trainer_training>* 1. Galaxy for Training and Education <https://gcc2017.sciencesconf.org/resource/page/id/8#galaxy_for_training> Please vote <https://docs.google.com/forms/d/e/1FAIpQLSdpkLoRuCGZOO3_T6dgRwFeDxVjb_zW-tl…>! Your participation helps make Galaxy Community Conferences the successful and community focused events that they are. And, thanks to everyone who nominated topics and/or contributed topic descriptions. Hope to see you in Montpellier! Dave Clements, on behalf of the GCC2017 Organising Committee -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 1

Request more details from Nikolaos Tur about pyBamTools & pyBamParser dependency packages message
by Jennifer Hsieh 07 Feb '17

07 Feb '17

Hi, I've installed a local instance of Galaxy (on Ubuntu 16.04, python 2.7.12) and have run in to this exact problem (posted by Nikolaos Tur https://www.mail-archive.com/galaxy-dev@lists.galaxyproject.org/msg03955.ht… ) while trying to add naive_variant_caller tool. Naive_variant_caller can't install because these two dependency packages can't be installed. I basically get exactly the same error message that ends with "IOError: Could not build the egg." Unfortunately I'm very new to both Galaxy and Python (Taking Coursera Galaxy course at the moment) and I don't really understand the solution Nikolaos described about modifying the setup.py to use "setuptools" instead of "distribute". I don't even know where to find setup.py... Can anyone help me with step-by-step details as to how to solve this dependency problem? Much Thanks! -- Jennifer Hsieh PhD Neuroscience (student) Department of Psychiatry and Mental Health University of Cape Town Tell: +27 21 447 3157 Cell: +27 74 841 4761

4 7