- galaxy-dev - lists.galaxyproject.org

Galaxy Restart & Jobs finishing
by Yip, Miu ki 14 Feb '17

14 Feb '17

Hi all, Is there a way to restart Galaxy without having to wait till all the outstanding jobs are finished running? I’ve noticed that whenever Galaxy daemon is stopped and then started, the UI does not return back online until all the jobs that were running are finished or killed because they are all running on the same port as the original program. Is there a configuration that needs to be changed so that the application can restart right away without having to wait or kill jobs? Thanks!

1 0

How to build a fresh Docker image from the latest Galaxy
by Md. Rezaul Karim 14 Feb '17

14 Feb '17

Hi there, I was wondering how to build a fresh Docker image/container from the latest version of Galaxy with all the tools available from the tool shed? Please note that I can build a Docker image for Galaxy by using the following Dockerfile: # VERSION 0.1 FROM bgruening/galaxy-stable:dev MAINTAINER Asif Karim<asif.karim.csedu"gmail.com> ENV GALAXY_CONFIG_BRAND BioOpener RUN curl -sL https://github.com/bgruening/galaxytools/archive/master.tar.gz | tar xz && cp -r galaxytools-master/visualisations/* config/plugins/visualizations/ && rm -rf ./galaxytools-master RUN install-biojs msa RUN install-repository \ "--url https://toolshed.g2.bx.psu.edu/ -o iuc --name package_numpy_1_7" \ "--url https://toolshed.g2.bx.psu.edu/ -o iuc --name package_perl_pdf_api2_2_023" \ "--url https://toolshed.g2.bx.psu.edu/ -o bgruening --name infernal --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o biopython --name package_biopython_1_61 --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o iuc --name package_trnascan_1_3_1 --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o iuc --name package_aragorn_1_2_36 --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o bgruening --name trna_prediction --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o lionelguy --name rnammer --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o bgruening --name rnaz --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name blockclust --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name cofold --panel-section-name RNATools" RUN install-repository \ "--url https://toolshed.g2.bx.psu.edu/ -o iuc --name package_vienna_rna_1_8" \ "--url https://toolshed.g2.bx.psu.edu/ -o devteam --name package_bowtie_0_12_7" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name vienna_rna --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o iuc --name package_squid_1_9g " \ "--url https://toolshed.g2.bx.psu.edu/ -o iuc --name package_randfold_2_0 " \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name mirdeep2_mapper --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name mirdeep2_quantifier --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name mirdeep2 --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name rnashapes --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name suite_mirdeep_2_0" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name antarna --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name dorina --panel-section-id getext" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name compalignp --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name rnabob --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name mafft --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name rnacode --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name gotohscan --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name kinwalker --panel-section-name RNATools" RUN install-repository \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name mea --panel-section-name RNATools" \ "--url https://toolshed.g2.bx.psu.edu/ -o rnateam --name sortmerna --panel-section-name RNATools" The Docker image works fine on my machine. However, I'm not sure if I image is built with the latest features of Galaxy. My second question is can I specify non-shed tools while building the Galaxy Docker image? Regards, _________________________________ *Md. Rezaul Karim*, BSc, MSc PhD Researcher, INSIGHT Centre for Data Analytics National University of Ireland, Galway IDA Business Park, Dangan, Galway, Ireland Web: http://www.reza-analytics.eu/index.html <http://139.59.184.114/index.html>

1 0

Relabeling dataset pairs in 'list:paired' collection
by Peter Briggs 14 Feb '17

14 Feb '17

Dear Developers Is there an existing tool or mechanism that can be used to duplicate a "list of pairs" dataset collection, keeping the paired datasets the same but relabeling each pair with a new identifier taken from a user supplied file or list? I've cobbled together my own tool to try and do something like this: https://github.com/pjbriggs/Amplicon_analysis-galaxy/blob/77340d8bb2470a646… and while it works, it doesn't feel like a good solution as it creates duplicates of the datasets from the first collection and consumes additional disk/quota space unnecessarily. (This is particularly undesirable as we expect that the input collections might be relatively large numbers of FASTQ pairs e.g. 30 or more.) Looking at some of the 'Collection Operations' tools that come with Galaxy, it appears that these are able to create new collections without making duplicate datasets, which seems much better. But these tools work by directly invoking Python classes from the Galaxy core, so I don't know if a similar approach could be used in a non-core tool. Any advice or suggestions are very welcome! Thanks Best wishes Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

2 2

Error in fresh galaxy install
by evan clark 13 Feb '17

13 Feb '17

I am setting up a new galaxy instance and upon intial setup and user login I get this error. The database is brand new and there is no data in any history as I am the sole user at this moment. "onLine": true, "version": "17.05", "xhr": { "readyState": 4, "responseText": "{\"err_msg\": \"History is not accessible by user\", \"err_code\": 403002}", "responseJSON": { "err_msg": "History is not accessible by user", "err_code": 403002 }, "status": 403, "statusText": "Forbidden" }, "options": { "parse": true, "keys": "type_id,element_count", "filters": { "history_content_type": "dataset_collection" }, "remove": false, "traditional": true, "data": { "keys": "type_id,element_count", "order": "hid", "v": "dev", "q": [ "history_content_type", "deleted", "purged", "visible" ], "qv": [ "dataset_collection", "False", "False", "True" ] }, "emulateHTTP": false, "emulateJSON": false, "textStatus": "error", "errorThrown": "Forbidden" },

1 0

Tools using Galaxy's Python library?
by Peter Cock 13 Feb '17

13 Feb '17

Hi all, A few of my tools have for a long time used Galaxy's own parsing functionality in order to avoid an external dependency. Lately this has stopped working on my TravisCI testing with planemo using the Galaxy dev branch (the stable master branch is fine): e.g. https://travis-ci.org/peterjc/pico_galaxy/builds/196655736 The tools fail with things like: | from galaxy_utils.sequence.fasta import fastaReader, fastaWriter | ImportError: No module named galaxy_utils.sequence.fasta or: | from galaxy_utils.sequence.fastq import fastqReader | ImportError: No module named galaxy_utils.sequence.fastq Is this a temporary regression in Galaxy, or a deliberate change? Do the tools need to do something to explicit have access to the Galaxy Python library, or are they now considered private? If so, I can update these tools to use an explicit dependency for parsing FASTA and FASTQ (e.g. Biopython). Thanks, Peter

4 8

regarding tool dependencies
by Nir Shachaf 09 Feb '17

09 Feb '17

Dear Galaxy-dev team, As a research scientist in the dept. of Plant Sciences I would like to contribute a tool to the 'Metabolomics' tool-box under Galaxy. The tool is principally written in R, with some dependencies (namely: R>=3.x, and several R packages). Following the documentation in: https://wiki.galaxyproject.org/Admin/Tools/ToolDependencies https://wiki.galaxyproject.org/Admin/Config/ToolDependencies I admit I get confused by the numerous possibilities offered by the framework. My main concern is to be able to link the Galaxy server to tools already installed on my development environment, and NOT to depend on the tools available in the tool-shed repository (as many are not there, and there are specific versions requirements). Could you kindly direct me to the relevant solution, or a user's-list post, if applicable? Many thanks in advance, Nir Shachaf, Plant Sciences Dept., Weizmann Inst. of Science, Rehovot, Israel.

2 1

Configuring Galaxy, Pulsar and Conda
by ric 09 Feb '17

09 Feb '17

Hi, I'm tryng to configure a Pulsar server to resolve dependencies using Conda but unfortunately it does not work. When I try to execute, for example, bowtie2, the standard error in the Galaxy machine says: "/home/galaxy/pulsar/files/staging/67/command.sh: line 60: bowtie2-build: command not found" because Conda in the Pulsar machine does not download this dependency. Here is my app.yml: dependency_resolvers_config_file: dependency_resolvers_conf.xml My dependency_resolvers_conf.xml <dependency_resolvers> <conda auto_install="True" auto_init="True" /> </dependency_resolvers> The log of the Galaxy server: galaxy.tools DEBUG 2017-01-19 14:50:18,910 Validated and populated state for tool request (97.265 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,069 Handled output named output for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 (81.926 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,113 Added output datasets to history (43.950 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,179 Verified access to datasets for Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/… (33.690 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,180 Setup for job Job[unflushed,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/… complete, ready to flush (66.261 ms) galaxy.tools.actions INFO 2017-01-19 14:50:19,240 Flushed transaction for job Job[id=67,tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.… (59.840 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,240 Tool [toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0] created job [67] (296.972 ms) galaxy.tools.execute DEBUG 2017-01-19 14:50:19,266 Executed 1 job(s) for tool toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0 request: (355.402 ms) 156.148.160.117 - - [19/Jan/2017:14:50:18 +0200] "POST /api/tools HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/webhooks/tool HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:19 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T12%3A11%3A40.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs DEBUG 2017-01-19 14:50:19,912 (67) Working directory for job is: /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67 galaxy.jobs.handler DEBUG 2017-01-19 14:50:19,931 (67) Dispatching to pulsar runner galaxy.jobs DEBUG 2017-01-19 14:50:20,013 (67) Persisting job destination (destination id: remote_cluster) galaxy.jobs.runners DEBUG 2017-01-19 14:50:20,044 Job [67] queued (112.446 ms) galaxy.jobs.handler INFO 2017-01-19 14:50:20,090 (67) Job dispatched galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:20,205 pulsar_version is 0.7.3 galaxy.util.object_wrapper WARNING 2017-01-19 14:50:20,644 Unable to create dynamic subclass for <type 'instance'>, None: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases galaxy.tools.deps DEBUG 2017-01-19 14:50:20,820 Find dependency bowtie2 version 2.3.0 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency bowtie2 version 2.3.0 of type conda galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Find dependency samtools version 1.3.1 galaxy.tools.deps DEBUG 2017-01-19 14:50:20,821 Using dependency samtools version 1.3.1 of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:20,822 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __bowtie2(a)2.3.0 --export > ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2(a)2.3.0 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:21,220 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda create -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsUaJpU60c3b56cd60a211bacfadd5c45280e5adba95113fa75f9e47b09cad555f49ae2b/__bowtie2(a)2.3.0 > /dev/null galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:23,885 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A19.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" Cache location: There are no tarballs to remove galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,291 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools(a)1.3.1 --export > ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools(a)1.3.1 galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:24,672 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsGUOeaPf795a38a38496af1ff491d0cf89e7e5d9fe737e5c36da3dd56c71ad608bb9b15/__samtools(a)1.3.1 > /dev/null 156.148.160.117 - - [19/Jan/2017:14:50:24 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:25,251 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove pulsar.client.client DEBUG 2017-01-19 14:50:25,904 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_65.dat] (action_type: [transfer]) pulsar.client.client DEBUG 2017-01-19 14:50:25,913 Uplodaing path [/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_59.dat] (action_type: [transfer]) 156.148.160.117 - - [19/Jan/2017:14:50:28 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A24.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:32 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A28.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:37 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A32.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:41 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A37.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:45 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A41.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:50:50 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A45.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.jobs.runners.pulsar INFO 2017-01-19 14:50:55,332 Pulsar job submitted with job_id 67 galaxy.jobs DEBUG 2017-01-19 14:50:55,333 (67) Persisting job destination (destination id: remote_cluster) 156.148.160.117 - - [19/Jan/2017:14:50:55 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A50.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" pulsar.client.staging.down INFO 2017-01-19 14:50:57,558 collecting output COMMAND_VERSION with action FileAction[action_type=transfer] pulsar.client.staging.down DEBUG 2017-01-19 14:50:57,562 Cleaning up job (failed [False], cleanup_job [always]) galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Find dependency samtools version None galaxy.tools.deps DEBUG 2017-01-19 14:50:59,176 Using dependency samtools version None of type conda galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,177 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda list --name __samtools@_uv_ --export > ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,558 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda install -y --unknown --offline --prefix /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env --copy --file ./database/tmp/jobdepsgV6rJH373e7bbee36e032187809f77033895b78158b1575365caf0cada674b44d010b8/__samtools@_uv_ > /dev/null 156.148.160.117 - - [19/Jan/2017:14:50:59 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A55.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.tools.deps.conda_util DEBUG 2017-01-19 14:50:59,967 Executing command: /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/conda clean --tarballs -y Cache location: There are no tarballs to remove galaxy.jobs.runners DEBUG 2017-01-19 14:51:00,351 executing external set_meta script for job 67: [ "$CONDA_DEFAULT_ENV" = "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env" ] || . /home/galaxy/galaxy_test_pulsar/database/dependencies/_conda/bin/activate '/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/conda-env' > conda_activate.log 2>&1 &&GALAXY_LIB="/home/galaxy/galaxy_test_pulsar/lib"; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH"; else PYTHONPATH="$GALAXY_LIB"; fi; export PYTHONPATH; fi; GALAXY_VIRTUAL_ENV="/home/galaxy/galaxy_test_pulsar/.venv"; if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate"; fi; python "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py" "/home/galaxy/galaxy_test_pulsar/database/tmp/tmpqS9WNY" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/working/galaxy.json" "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_in_HistoryDatasetAssociation_71_TIRfAD,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_kwds_HistoryDatasetAssociation_71_drHgz4,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_results_HistoryDatasetAssociation_71_E1N138,/home/galaxy/galaxy_test_pulsar/database/files/000/dataset_71.dat,/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_override_HistoryDatasetAssociation_71_iGnttD" 5242880 Traceback (most recent call last): File "/home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/set_metadata_6JXZtD.py", line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() ImportError: No module named galaxy_ext.metadata.set_metadata galaxy.jobs.runners DEBUG 2017-01-19 14:51:01,013 execution of external set_meta for job 67 finished galaxy.jobs INFO 2017-01-19 14:51:03,230 Collecting metrics for Job 67 galaxy.jobs DEBUG 2017-01-19 14:51:03,261 job 67 ended (finish() executed in (2246.565 ms)) galaxy.model.metadata DEBUG 2017-01-19 14:51:03,272 Cleaning up external metadata files galaxy.model.metadata DEBUG 2017-01-19 14:51:03,317 Failed to cleanup MetadataTempFile temp files from /home/galaxy/galaxy_test_pulsar/database/jobs_directory/000/67/metadata_out_HistoryDatasetAssociation_71_AQg1tP: No JSON object could be decoded 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-01-19T13%3A50%3A59.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:04 +0200] "POST /api/tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0/build HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:07 +0200] "GET /api/histories/1cd8e2f6b131e891/contents/datasets/fa6833a4eadf9064 HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:16 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:22 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" /home/galaxy/galaxy_test_pulsar/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/default_comparator.py:153: SAWarning: The IN-predicate on "request_type.id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance. 'strategies for improved performance.' % expr) 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:45 +0200] "GET /admin/center?status=done&message= HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:47 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:50 +0200] "GET /admin_toolshed/browse_tool_sheds HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:51:52 +0200] "GET /admin_toolshed/browse_tool_shed?tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://156.148.120.96:8080/admin_toolshed/browse_tool_sheds" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:17,048 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:16 +0200] "GET / HTTP/1.1" 200 - "http://156.148.120.96:8080/admin" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/history-menu/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/datatypes?extension_only=False HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/webhooks/masthead/all HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /api/genomes HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /welcome HTTP/1.1" 302 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:19 +0200] "GET /history/current_history_json HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" galaxy.managers.users WARNING 2017-01-19 14:59:20,083 {u'RepositoryInstallationGrid.filter': u'{}', u'RepositoryInstallationGrid.sort_key': u'"-create_time"'} 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/users/f597429621d6eb2b HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:20 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?limit=500&offset=0&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=deleted&q=purged&q=visible&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:21 +0200] "GET /api/histories/1cd8e2f6b131e891?keys=size%2Cnon_ready_jobs%2Ccontents_active%2Chid_counter HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:23 +0200] "GET /api/histories/1cd8e2f6b131e891/contents?keys=type_id%2Celement_count&details=b847e822bdc195d0%2C40876639881ca029%2C915ae9a80309f157%2Cfd767e8b7355e21a%2C9ce617df390851fc%2C3cc0effd29705aa3%2Cb701da857886499b%2Cfa6833a4eadf9064&order=hid&v=dev&q=history_content_type&q=deleted&q=purged&q=visible&qv=dataset_collection&qv=False&qv=False&qv=True HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:24 +0200] "GET /datasets/fa6833a4eadf9064/show_params HTTP/1.1" 200 - "http://156.148.120.96:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 156.148.160.117 - - [19/Jan/2017:14:59:27 +0200] "GET /datasets/fa6833a4eadf9064/stderr HTTP/1.1" 200 - "http://156.148.120.96:8080/datasets/fa6833a4eadf9064/show_params" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" The log of the Pulsar server: Sourcing file ./local_env.sh Starting pulsar with command [paster server "./server.ini" ""] 2017-01-19 14:49:36,769 INFO [pulsar.core][MainThread] Starting the Pulsar without a toolbox to white-list.Ensure this application is protected by firewall or a configured private token. 2017-01-19 14:49:36,793 INFO [pulsar.locks][MainThread] pylockfile module not found, skipping experimental lockfile handling. Starting server in PID 12234. serving on 0.0.0.0:8913 view at http://127.0.0.1:8913 2017-01-19 14:50:31,715 DEBUG [pulsar.web.routes][worker 0] Setup job with configuration: {'tool_id': u'toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.3.0', 'metadata_directory': '/home/galaxy/pulsar/files/staging/67/metadata', 'tool_version': u'2.3.0', 'path_separator': '/', 'unstructured_files_directory': '/home/galaxy/pulsar/files/staging/67/unstructured', 'tools_directory': '/home/galaxy/pulsar/files/staging/67/tool_files', 'working_directory': '/home/galaxy/pulsar/files/staging/67/working', 'inputs_directory': '/home/galaxy/pulsar/files/staging/67/inputs', 'job_directory': '/home/galaxy/pulsar/files/staging/67', 'system_properties': {'galaxy_home': '/home/galaxy/galaxy_test_pulsar', 'separator': '/'}, 'outputs_directory': '/home/galaxy/pulsar/files/staging/67/outputs', 'job_id': '67', 'configs_directory': '/home/galaxy/pulsar/files/staging/67/configs', 'pulsar_version': '0.7.3'} 2017-01-19 14:51:06,843 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=67]] job_id: 67 - Checking authorization of command_line [bowtie2 --version > /home/galaxy/pulsar/files/staging/67/outputs/COMMAND_VERSION 2>&1; bowtie2-build --threads ${GALAXY_SLOTS:-4} '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome && ln -s -f '/home/galaxy/pulsar/files/staging/67/inputs/dataset_59.dat' genome.fa && bowtie2 -p ${GALAXY_SLOTS:-4} -x 'genome' -U '/home/galaxy/pulsar/files/staging/67/inputs/dataset_65.dat' | samtools sort -O bam -o '/home/galaxy/pulsar/files/staging/67/outputs/dataset_71.dat'; return_code=$?; sh -c "exit $return_code"] 2017-01-19 14:51:07,826 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=67]] collecting output COMMAND_VERSION with action FileAction[action_type=transfer] 2017-01-19 14:51:07,827 INFO [pulsar.managers.stateful][[manager=_default_]-[action=postprocess]-[job=67]] Status of job [67] changed to [complete]. No callbacks enabled. I think I'm doing something wrong in the configuration of the Pulsar server so I need some help or suggestion. Thank you -- Ricardo Medda CRS4 - Biomedicina Phone +39 070 9250 ??? Loc. Pixina Manna Edificio 1 Pula 09010 (CA), Italy Email: medda(a)crs4.it Website: www.crs4.it

2 2

Create a new datatype svg
by Jochen Bick 09 Feb '17

09 Feb '17

Hi all, I'm working on a tool that will have svg as an output file. Is this data type already established? If not how do I start? Cheers Jochen

4 9

Pulsar Metadata
by Joseph Brent Greer 07 Feb '17

07 Feb '17

Hi All, We've been using Pulsar to send jobs to our cluster on campus. Generally we send a dataset list of files over. We're wondering if there is a way to get the metadata for the files after they've been sent to the cluster. We're specifically trying to find the filepaths of the files before they were through Pulsar to the cluster. Any guidance would be great. Thanks, Joe

2 1

GalaxyAdmins Feb online meetup (Enis Afgan)
by David Kovalic 07 Feb '17

07 Feb '17

Enis, Could you send the call-in information please? David On Tue, Feb 7, 2017 11:00 AM, galaxy-dev-request(a)lists.galaxyproject.org wrote: Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.galaxyproject.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.galaxyproject.org/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.galaxyproject.org You can reach the person managing the list at galaxy-dev-owner(a)lists.galaxyproject.org When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative.Today's Topics: 1. Re: Request more details from Nikolaos Tur about pyBamTools & pyBamParser dependency packages message (Peter Briggs) 2. GalaxyAdmins Feb online meetup (Enis Afgan) 3. Create a new datatype svg (Jochen Bick) 4. Re: Create a new datatype svg (Björn Grüning) 5. Re: Create a new datatype svg (Jochen Bick) 6. Re: Request more details from Nikolaos Tur about pyBamTools & pyBamParser dependency packages message (Jennifer Hsieh) 7. SLURM configuration problem (Leonor Palmeira) 8. Re: Issue when using xsendfile with galaxy release_16.07 (SAPET, Frederic) _______________________________________________ galaxy-dev mailing list galaxy-dev(a)lists.galaxyproject.org https://lists.galaxyproject.org/listinfo/galaxy-dev To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

1 0