November 2015 - galaxy-dev - lists.galaxyproject.org

workflow API: step_order vs step_id in bioblend
by Jorrit Boekel 20 Nov '15

20 Nov '15

Hi all, Here’s a mail for heads up and googleable error message in case someone finds a similar error and scratches her/his head. So (some time) after the very nice API class we had at GCC2015 I am now trying my hand at running workflows using Bioblend. I had some frustration trying to invoke a simple WF, but now found out that the inputs parameter to invoke_workflow is not as I thought it would be. I’m on the latest galaxy-dist and using Bioblend 0.7 in Python3.4. So I thought I’d call e.g.: gi.workflows.invoke_workflow(workflow_id, inputs={‘9678’: {’src’: ‘hda’, ‘id’: ‘abcdef12345’}} , history_id=‘abc1234’) Where the keys in inputs dict represent the ids from: gi.workflows.get_workflow(workflow_id)[‘inputs’] But apparently the new standard is to use the step_order (so 0 for the first step) instead of the step_id, as shown in the code at lib/galaxy/workflow/run_request.py So this gave me the HTTP 400 error "Workflow cannot be run because an expected input step '84' has no input dataset." I have reverted to using legacy code with run_workflow and dataset_map, which circumvents the problem: gi.workflows.run_workflow(workflow_id, dataset_map={‘9678’: {’src’: ‘hda’, ‘id’: ‘abcdef12345’}} , history_id=‘abc1234’) Is there any way to specify inputs_by in the payload or am I on the wrong bioblend version? Otherwise I can file a request on the Bioblend github. cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm, Sweden

2 3

ProFTPD with SQLite
by Mic 19 Nov '15

19 Nov '15

Hello, I installed all tools from the toolshed which I needed for Galaxy (15.10) and noticed that I can not upload bigger files than 2 GB to Galaxy without a FTP server. I installed on Ubuntu 14.04 ProFTPD in the following way: *sudo aptitude install ProFTPD proftpd-mod-sqlite sqlite3* Unfortunately, I do not know how to connect the ProFTPD with Galaxy's SQLite. Thank you in advance. Mic

2 4

rgFastQC.py error - cannot find executable /fastqc
by Mic 19 Nov '15

19 Nov '15

Hello, I installed FastQC (~/shed_tools/toolshed.g2.bx.psu.edu/repos/*devteam* /fastqc/*2d094334f61e*/ *FastQC ).* However, I got the following error while running it: *Fatal error: Exit code 1 () Traceback (most recent call last): File "/home/lorencm/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastqc/2d094334f61e/fastqc/rgFastQC.py <http://toolshed.g2.bx.psu.edu/repos/devteam/fastqc/2d094334f61e/fastqc/rgFa…>", line 157, in <module> assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##rgFastQC.py error - cannot find executable /fastqc* The Fastqc package seems to be correctly installed: *~/galaxy/tool_dependency/FastQC/0.11.2/devteam/package_fastqc_0_11_2/4b65f6e39cb0 > ls -1 -ahltotal 824Kdrwxrwxr-x 8 lorencm lorencm 4.0K Nov 4 02:21 .drwxrwxr-x 3 lorencm lorencm 4.0K Nov 4 02:21 ..drwxrwxr-x 2 lorencm lorencm 4.0K Nov 4 02:21 Configuration-rwxrwxr-x 1 lorencm lorencm 251 Nov 4 02:21 env.sh-rwxr-xr-x 1 lorencm lorencm 13K Nov 4 02:21 fastqc-rw-rw-r-- 1 lorencm lorencm 2.2K Nov 4 02:21 fastqc_icon.icodrwxrwxr-x 5 lorencm lorencm 4.0K Nov 4 02:21 Help-rw-rw-r-- 1 lorencm lorencm 5.5K Nov 4 02:21 INSTALL.txt-rw-rw-r-- 1 lorencm lorencm 49K Nov 4 02:21 jbzip2-0.9.jar-rw-rw-r-- 1 lorencm lorencm 35K Nov 4 02:21 LICENSE.txtdrwxrwxr-x 3 lorencm lorencm 4.0K Nov 4 02:21 netdrwxrwxr-x 3 lorencm lorencm 4.0K Nov 4 02:21 org-rw-rw-r-- 1 lorencm lorencm 2.1K Nov 4 02:21 README.txt-rw-rw-r-- 1 lorencm lorencm 32K Nov 4 02:21 RELEASE_NOTES.txt-rw-rw-r-- 1 lorencm lorencm 101 Nov 4 02:21 run_fastqc.bat-rw-rw-r-- 1 lorencm lorencm 630K Nov 4 02:21 sam-1.103.jardrwxrwxr-x 3 lorencm lorencm 4.0K Nov 4 02:21 Templatesdrwxrwxr-x 3 lorencm lorencm 4.0K Nov 4 02:21 uk* What do I miss out? Thank you in advance. Mic

3 5

API key change manually
by Mic 19 Nov '15

19 Nov '15

Hello, After I migrated Galaxy installation to Postgresql my API key got lost. I still know it and would like to add it to the database, but I am not quite sure how to do it. How is it possible to add an API key to the database. Thank you in advance. Mic

3 3

Error installing tools from toolshed
by Scott Szakonyi 19 Nov '15

19 Nov '15

Hello all, I'm having issues installing Trimmomatic and Picard tools from the toolshed. The errors are of the following form (Trimmomatic used for the example). Trimmomatic 0.32 package shows an installation error. Error details: File "/vectorbase/web/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 142, in install_and_build_package_via_fabric tool_dependency = self.install_and_build_package( tool_shed_repository, tool_dependency, actions_dict ) File "/vectorbase/web/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 136, in install_and_build_package dir = tmp_dir File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__ self.gen.throw(type, value, traceback) File "/vectorbase/web/Galaxy/galaxy-dist/eggs/Fabric-1.7.0-py2.6.egg/fabric/context_managers.py", line 142, in _setenv yield File "/vectorbase/web/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 100, in install_and_build_package initial_download=True ) File "/vectorbase/web/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 32, in execute_step initial_download=initial_download ) File "/vectorbase/web/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 685, in execute_step dir = self.url_download( work_dir, downloaded_filename, url, extract=True, checksums=checksums ) File "/vectorbase/web/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 223, in url_download extraction_path = archive.extract( install_dir ) File "/vectorbase/web/Galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 89, in extract os.chmod( absolute_filepath, unix_permissions ) [Errno 2] No such file or directory: '/data/galaxy/tmp/tmp-toolshed-mtdeqJvts/Trimmomatic-0.32/Trimmomatic-0.32/' I've tried several version of the tool with no success. Any thoughts? I've searched project website but haven't come up with anything definitive. Thanks for any help you can offer, -- Scott B. Szakonyi Research Programmer *Center for Research Computing* 107 Information Technology Center Notre Dame, IN 46556 http://crc.nd.edu

2 1

Cloudman vs StarCluster
by Ryan G 19 Nov '15

19 Nov '15

Hi all - Our organization requires us to use a specific AMI (with a specific flavor of linux) that isn't supported by Cloudman. Its taking us a bit of time to get this combination working. Its taken more than 3 weeks and we still don't have Galaxy / Cloudman running. I could probably do the same thing with StarCluster and be up and running within a day. I'm very comfortable with using StarCluster and am wondering if there is any benefit to using Cloudman over StarCluster for node management with Galaxy. Thoughts?

2 1

Tool code for symlinking a data collection from input to output?
by Dooley, Damion 19 Nov '15

19 Nov '15

Has anyone done a tool part of which simply takes in a data collection and provides it (symlinked) in a corresponding output data collection? We have a quality control tool that enables us to stop workflow, preventing subsequent jobs to be run if a problem has been detected. Currently our tool only sym-links through an individual dataset. It would be great if it could work on a whole data collection of any sort. I see there’s a <collections> tag now – is there an easy solution using this – and that doesn’t loose metadata? <collection name=“genericCollectionSymlink" label=“Workflow datasets”> <discover_datasets pattern=???????????? visible="true" ......................????????? </collection> At moment we use parameters for format_source="workflow_files" metadata_source=“workflow_files” to pass through all the info on an input dataset... <inputs> … <param name="workflow_files" type="data" optional="True" multiple=“False" label="Workflow data" help="Select dataset(s) that subsequent workflow stages can consume if report status is not fail'." /> </inputs> <outputs> <data name="workflow_files_pass" format_source="workflow_files" label="Workflow datasets" metadata_source="workflow_files” > <filter>(workflow_files)</filter> </data> </outputs> Much obliged! Damion Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

4 6

Dataset tags lost when copying or sharing history
by Brian Claywell 18 Nov '15

18 Nov '15

One of our users is using dataset tags to manage the datasets in an RNA-seq experiment, and has reported that these tags are lost when making a copy of a history to run different analyses, or when the history is shared with another user so they can run their own analyses. Is this a bug or intended behavior? If it's the latter, where would be a good place to suggest that dataset tags be maintained when copying or sharing histories? Cheers, Brian -- Brian Claywell | programmer/analyst Matsen Group | matsen.fredhutch.org <http://matsen.fhcrc.org/> Fred Hutchinson Cancer Research Center

2 1

galaxy folder tree permissions
by lejeczek 17 Nov '15

17 Nov '15

hi everybody I'd like to ask if you think it's worthwhile is pursuing finely grained tree permissions? Would this improve security to leave out everything but only files/folders necessary for writing - to galaxy user what needs to write everything else root? Or just full perms to galaxy user on whole tree is the only way? many thanks.

3 4

list collection output - format set from input
by Alexander Vowinkel 17 Nov '15

17 Nov '15

Hi, I have an input, that can be fasta,fastqsanger,fastqillumina: <param name="fastq_input1" type="data" format="fasta,fastqsanger,fastqillumina" label="Select the fastq file" help="Specify fastq file with reads"/> I have multiple outputfiles - bundled in a list collection: <collection name="split_output" type="list" label="@OUTPUT_NAME_PREFIX@ on ${on_string} (Fastq Collection)" format_source="fastq_input1"> <discover_datasets pattern="__name_and_ext__" directory="splits" /> </collection> The format_source parameter doesn't work - the files in the list (extension fq) are of format "fq" How can I make it possible that they are fasta,fastqsanger,fastqillumina depending on fastq_input1? Thanks, Alexander

4 6