- galaxy-dev - lists.galaxyproject.org

Galaxy Update Problem
by 孙兆朕 05 Mar '15

05 Mar '15

Hi all, I installed my Galaxy instance from https://bitbucket.org/galaxy/galaxy-dist/. But I did not use Mercurial to install Galaxy at that time, because I was not farmiliar with Mercurial and failed severed times. The instance imported many tool kits later, as it would be very difficult to restart everthing for me. I tried to use hg command: hg clone https://bitbucket.org/galaxy/galaxy-dist/ cd galaxy-dist hg update stable to make a new copy and copied the ".hg" directory to the former instance and used the "hg update stable" command. Then it reports that there were not files to change. I wonder if there could be any way to update the instance. Thanks, Zhaozhen

2 1

RFC: new param attribute for Galaxy tool XML
by Björn Grüning 04 Mar '15

04 Mar '15

Hi all, we are planning to work on a project to implement a Galaxy fuse based shell. Probably starting with the work from Clare [1]. Next to our Galaxy IPython integration it should attract more bioinformaticians and should offer a new way to interact with Galaxy. This includes moving, deleting datasets, but also executing tools and workflows. For the latter I would like to have some sort of bash auto-completion. Type in your tool/workflow and you will see all the parameters you can/should modify, in addition to your normal help page. Currently the parameter identifiers (<param name="foo">) do not need to be meaningful. In fact many tools invent their own unique names, to identify a parameter in the cheetah section. This name is always unique but it's hard to guess it's meaning from just the name and are also not mappable to the original parameter name. This makes tool execution from the API sometimes hard. I would like to propose a new attribute for all <param> tags. This should specify a unique value that 100% matches the original parameter name of the underlying tool. This attribute could be used to: * automatically enhance the help text of each parameter. Currently best practise it to include this parameter in brackets at the end of each help text. We can do this now automatically, or only show it by mouse over etc ... * In a Galaxy shell, the user could just type in a normal command (-i history1/foo.bam -o history1/bar.sam) and Galaxy shell would be able to map this parameter to the correct <param> tag in Galaxy I greatly appreciate any comments! Thanks, Bjoern [1] https://github.com/claresloggett/gvl_commandline_utilities/blob/master/gala…

3 3

Instances Working with Merged Ephemerals
by Stephen Rosen 04 Mar '15

04 Mar '15

Hi Dina, Alex, I have ngxbio1 working and merging ephemerals on the workers as desired. We were missing a line in cloudinit to install lvm, which is needed to merge disks. Because of the instance type requested, it only has two ephemeral volumes maximum, even though we request four ephemerals. Different types have different limits, so this isn't a code issue at all. I assume this is fine, but if not we'll have to look at the instance types we're requesting for different workloads. I'm going to propagate these changes to ngxbio[2-4] now. Mail to follow. Best, -Stephen

3 3

Re: [galaxy-dev] Galaxy tool shed alert for new repository named blast_reporting
by Peter Cock 03 Mar '15

03 Mar '15

Hi Damion, I see your blast_reporting tool is now on the Main Tool Shed. https://toolshed.g2.bx.psu.edu/view/damion/blast_reporting I am a little confused about your tabular output options: <option value="std" selected="True">Standard 12 columns</option> <option value="std+seqs">Standard 12 columns + sequences</option> <option value="ext">Extended 24 columns</option> <option value="ext+">Extended 26 columns</option> <option value="custom">Only field selections below</option> In comparison my blastxml_to_tabular currently offers: <option value="std" selected="True">Tabular (standard 12 columns)</option> <option value="ext">Tabular (extended 25 columns)</option> <option value="cols">Tabular (select columns to output)</option> For the moment my pick-you-own columns choices are limited here to the 25 columns (unlike the BLAST+ wrappers which offers more). See: https://github.com/peterjc/galaxy_blast/tree/master/tools/ncbi_blast_plus In ncbi_blast_plus the extended output used to be 24 columns, my 25th column is the description (BLAST+ field salltitles, added in BLAST+ 2.2.28). https://github.com/peterjc/galaxy_blast/commit/0b161baa7f87d514c94fd1b3976a… https://github.com/peterjc/galaxy_blast/commit/5234e32a277595044ec6d2a7002d… http://blastedbio.blogspot.co.uk/2012/05/blast-tabular-missing-descriptions… Your in-tool help and the blast_reporting.py comments show you've picked different extra columns: Column 25 pcov Percentage coverage Column 26 sallseqdescr All subject Seq-descr(s), separated by a ',' I'm guessing these names map to NCBI BLAST+ fields qcovs or qcovhsp, and salltitles? Could you check and adopt the NCBI names? It is unfortunately the choice of fields has diverged. Could you switch your columns 25 and 26 round in your "ext+" output so that we agree on the first 25 columns? And expand your "ext" output to add the descriptions and so match the 25 column extended output in ncbi_blast_plus too? Thanks, Peter P.S. There is a problem in your markdown for the README.md file, shown as a "Unexpected indentation." error when viewed on the Tool Shed. https://toolshed.g2.bx.psu.edu/view/damion/blast_reporting On Tue, Mar 3, 2015 at 1:46 AM, <galaxy-no-reply(a)montana.galaxyproject.org> wrote: > > Sharable link: https://toolshed.g2.bx.psu.edu/view/damion/blast_reporting > Repository name: blast_reporting > Revision: 0:7db7ecc78ad6 > Change description: > Uploaded > > Uploaded by: damion > Date content uploaded: 2015-03-02 > > > > ----------------------------------------------------------------------------- > This change alert was sent from the Galaxy tool shed hosted on the server > "toolshed.g2.bx.psu.edu" > ----------------------------------------------------------------------------- > You received this alert because you registered to receive email when > new repositories were created in the Galaxy tool shed named "toolshed.g2.bx.psu.edu". > -----------------------------------------------------------------------------

2 2

join bam results in one file
by Roberto Alonso CIPF 03 Mar '15

03 Mar '15

Hello, I am playing with Galaxy "splitters" capabilities. After some cases that you help me out to solve I am facing a new issue, this is maybe due to my tool configuration file, but in any case I tell you what I've done. What I would like to do exactly, is to split paired fastq, map them and then join them. This is my configuration file: <tool id="bwa_mio" name="map with bwa"> <description>map with bwa</description> <parallelism method="basic" split_size="3" split_mode="number_of_parts" merge_outputs="output"></parallelism> <command> bwa mem -R '@RG\tID:foo\tSM:bar' /home/ralonso/BiB/Galaxy/data/Cclementina_v1.0_scaffolds.fa $input > temporary_bam_file.sam 2>/dev/null ; samtools view -Sb temporary_bam_file.sam > temporary_bam_file.bam ; samtools sort temporary_bam_file.bam $output ; </command> <inputs> <param format="fastqsanger" name="input" type="data" label="fastq"/> </inputs> <outputs> <data format="bam" name="output" /> </outputs> <help> bwa </help> </tool> My problem of this configuration is that generates an empty file. So, after seeing the code, I discover that when it tries to join the several bam files it goes to the first parent: *class Data( object )*, to the method merge: *def merge( split_files, output_file). *So I may be wrong, but I think binary.bam class should override this method, is this right? if this is the case, I would like to implement this method, I have couple of basic ideas, like merge them with samtools. What do you think? On ther other hand, is this related with the last email of John Chilton and the 15.03 release? Best Regards -- Roberto Alonso Functional Genomics Unit Bioinformatics and Genomics Department Prince Felipe Research Center (CIPF) C./Eduardo Primo Yúfera (Científic), nº 3 (junto Oceanografico) 46012 Valencia, Spain Tel: +34 963289680 Ext. 1021 Fax: +34 963289574 E-Mail: ralonso(a)cipf.es

1 0

Tool tests failing on Main Tool Shed with "Exception: Job in error state."
by Peter Cock 03 Mar '15

03 Mar '15

Hello all, Good news - automated testing on the Galaxy Tool Shed seems to be mostly working again (after a period of trouble in the New Year). Thanks Dave B. et al, I'm sure this has been a lot of effort behind the scenes. I'm looking forward to getting a clean sweep of passing tests again... I currently have 18 repositories listed with no failing tests :) The bad news is I currently have four Tool Shed repositories with unexpectedly failing tests (all of which I expected to pass based on past testing, including locally and on TravisCI). I will need some assistance to debug the first three which all fail with the cryptic message "Exception: Job in error state." (Where a tool has more than one test, I've only shown the first error) https://toolshed.g2.bx.psu.edu/view/peterjc/blast2go Test runs 2015-01-29 05:12:50 Automated test environment Tests that failed Tool id: blast2go Tool version: blast2go Test: test_tool_000000 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/peterjc/bl…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 40, in do_it self._verify_outputs( testdef, test_history, jobs, shed_tool_id, data_list, galaxy_interactor ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 80, in _verify_outputs galaxy_interactor.verify_output( history, jobs, output_data, output_testdef=output_testdef, shed_tool_id=shed_tool_id, maxseconds=maxseconds ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 62, in verify_output self.wait_for_job( job[ 'id' ], history_id, maxseconds ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 119, in wait_for_job self.twill_test_case.wait_for( lambda: not self.__job_ready( job_id, history_id ), maxseconds=maxseconds) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/twilltestcase.py", line 2448, in wait_for result = func() File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 119, in <lambda> self.twill_test_case.wait_for( lambda: not self.__job_ready( job_id, history_id ), maxseconds=maxseconds) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 277, in __job_ready return self._state_ready( state, error_msg="Job in error state." ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 346, in _state_ready raise Exception( error_msg ) Exception: Job in error state. http://toolshed.g2.bx.psu.edu/view/peterjc/clinod 2015-01-29 05:07:17 Automated test environment Tests that failed Tool id: clinod Tool version: clinod Test: test_tool_000000 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/peterjc/cl…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 40, in do_it self._verify_outputs( testdef, test_history, jobs, shed_tool_id, data_list, galaxy_interactor ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 80, in _verify_outputs galaxy_interactor.verify_output( history, jobs, output_data, output_testdef=output_testdef, shed_tool_id=shed_tool_id, maxseconds=maxseconds ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 62, in verify_output self.wait_for_job( job[ 'id' ], history_id, maxseconds ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 119, in wait_for_job self.twill_test_case.wait_for( lambda: not self.__job_ready( job_id, history_id ), maxseconds=maxseconds) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/twilltestcase.py", line 2448, in wait_for result = func() File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 119, in <lambda> self.twill_test_case.wait_for( lambda: not self.__job_ready( job_id, history_id ), maxseconds=maxseconds) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 277, in __job_ready return self._state_ready( state, error_msg="Job in error state." ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 346, in _state_ready raise Exception( error_msg ) Exception: Job in error state. https://toolshed.g2.bx.psu.edu/view/peterjc/effectivet3 2015-01-29 05:11:05 Automated test environment Tests that failed Tool id: effectiveT3 Tool version: effectiveT3 Test: test_tool_000000 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/peterjc/ef…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 40, in do_it self._verify_outputs( testdef, test_history, jobs, shed_tool_id, data_list, galaxy_interactor ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 80, in _verify_outputs galaxy_interactor.verify_output( history, jobs, output_data, output_testdef=output_testdef, shed_tool_id=shed_tool_id, maxseconds=maxseconds ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 62, in verify_output self.wait_for_job( job[ 'id' ], history_id, maxseconds ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 119, in wait_for_job self.twill_test_case.wait_for( lambda: not self.__job_ready( job_id, history_id ), maxseconds=maxseconds) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/twilltestcase.py", line 2448, in wait_for result = func() File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 119, in <lambda> self.twill_test_case.wait_for( lambda: not self.__job_ready( job_id, history_id ), maxseconds=maxseconds) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 277, in __job_ready return self._state_ready( state, error_msg="Job in error state." ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 346, in _state_ready raise Exception( error_msg ) Exception: Job in error state. The fourth and final failure on the main Tool Shed this morning is from: https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad This is different, "Converting local (test-data) bam to sam failed", already reported (27 Jan): https://lists.galaxyproject.org/pipermail/galaxy-dev/2015-January/021383.ht… http://dev.list.galaxyproject.org/test-base-twilltestcase-py-Converting-loc… Regards, Peter

1 1

Re: [galaxy-dev] Tool param type "library_data" (Dooley, Damion)
by Dooley, Damion 02 Mar '15

02 Mar '15

I understand a new tool form module is rolling out in the next few weeks. Will anything change in the HTML <form> hidden input submit fields? Right now tool forms have the equivalent of: <form id="tool_form" name="tool_form" action="../../../tool_runner" target="galaxy_main" method="post" enctype="application/x-www-form-urlencoded"> <input type="hidden" name="refresh" value="refresh"/> <input type="hidden" name="tool_id" value="bccdcSelectSubsets"/> <input type="hidden" name="tool_state" value="3865326638 .... 1323033303764"> <input type="hidden" name="input" value="4246"/> <input type="hidden" name="incl_excl" value="1"/> ... and then on to the visible fields. Currently one of my galaxy tools "blast_reporting" generates an html report (form) which users can view and select items and submit - which calls another tool with just the right parameters, as above, including tool_state value which is copied once from viewing source of the live target tool form. 2nd tool immediately generates the desired dataset. I know html form approach is a hack but it works and it keeps things simple (its never part of a workflow). However, a nice feature for the new tool form engine would be to have the tool state accept an empty value="" for initial submission of forms. Is that possible? That way I wouldn't have to bother with one other yucky installation step for my blast_reporting tool where one has to manually view source of an instance of the target tool (bccdcSelectSubsets above) and retrieve the generic tool_state field value setting. d.

1 0

The March 2015 Galaxy News
by Dave Clements 02 Mar '15

02 Mar '15

Hello all, As always, it's a busy time in the Galaxy <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03>. The March 2015 Galaxy News <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03> is out, and it's chock full of information. Here are some of the highlights. *UseGalaxy.org Update <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#UseGalaxy.org_Update>:* A refresh of UseGalaxy.org <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#UseGalaxy.org_Update> is underway. Find out what's changing in a new screencast. *2015 Galaxy Community Conference (GCC2015) <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#GCC2015:_6-8_July.2C_N…>* [image: Sponsor GCC2015] <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#GCC2015:_6-8_July.2C_N…> The event schedule for GCC2015 has been released, and it features a few new events as well as all of the familiar ones: - Code Hackathon <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Code_Hackathon>, 4-5 July - Data Wrangling Hackathon <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Data_Wrangling_Hackath…>, 4-5 July (*new*) - Training SunDay <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Training_SunDay>, 5 July (*new*) - Training Day <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Training_.28Mon.29Day>, 6 July - GCC Meeting <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Abstract_Submission_is…>, 7-8 July In addition, the workshop schedules for the *new* Training SunDay <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Training_SunDay>, and the returning Training Day <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Training_.28Mon.29Day> have been finalised. A record 15 topics will be covered in 18 sessions over two days of training. There are topics on using Galaxy, interacting with it programmatically, and deploying, administering, and extending Galaxy. No matter what you do with Galaxy, there are workshops for you. Abstract submission for oral and poster presentations is now open <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Abstract_Submission_is…>. We again have an arrangement with GigaScience that any oral or poster presentation will also be eligible for consideration for publication in the GigaScience "Galaxy: Data Intensive and Reproducible Research" series (first announced for GCC2013). Finally, registration is *not* open yet, but the list of registration packages and their costs <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Early_Registration_ope…> have been. Early registration is very affordable and starts at less than £40 per day for students and postdocs. *Other Events <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Other_Events>* And there is no shortage of other upcoming events either, including RNA-Seq and ChIP-Seq Analysis with Galaxy, UC Davis, March 23-26 <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#RNA-Seq_and_ChIP-Seq_A…>, and at least 12 other events in the next 2 months <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Other_Events>. *Lots of other news <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03>* In addition, there is news on: - 82 new publications <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#New_Papers>, including 7 highlighted ones - Summary of the Fall 2014 Galaxy Community Questionnaires <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Fall_2014_Questionnair…> - An introduction to Gravity <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Gravity>, a process manager (supervisor) for Galaxy servers. - Who's Hiring <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Who.27s_Hiring> - Whale Shark! <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Whale_Shark> - Planemo 0.5.0 released <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Planemo_0.5.0> - Two new Community Log Board entries <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Galaxy_Community_Hubs> - *Galaxy on GitHub! <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Galaxy_GitHub>* Come and join us in developing Galaxy! - And, other news <https://wiki.galaxyproject.org/GalaxyUpdates/2015_03#Other_News> as well. Dave Clements and the Galaxy Team <https://wiki.galaxyproject.org/GalaxyTeam> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

Interested in GSOC idea "Docker Swarm and Compose" & ask for advices.
by afe.young＠gmail.com 02 Mar '15

02 Mar '15

Hi I'm a first-year student for my Master Degree in China. I'm really interested in GSOC and viewed a lot of 2015 ideas recently. "Docker Swarm and Compose" is the most attractive one for me. I've some experience in opensource and have developed a lot of projects with python. I read some books about docker recently and have deployed galaxy with https://github.com/bgruening/docker-galaxy-stable/ . I wondering what else I can do to prepare for the idea "Docker Swarm and Compose". It will be awesome to me to work with you. So I'm here asking for advices. Thanks in advance.

3 2

gatk tool config file
by Roberto Alonso CIPF 02 Mar '15

02 Mar '15

Hello, I would like to add a very simple GATK wrapper to galaxy, but I am facing one problem. When you want to use UnifiedGenotyper tool, the extension of the bam file must be .bam, and when you upload something to galaxy it changes its extension,is there any way to change this extension? or is it anyway of solving this issue? This is my conf file. <tool id="gatk_mio" name="calling with gatk"> <description>calling with gatk</description> <command> java -jar /home/ralonso/software/GenomeAnalysisTK-3.3-0/GenomeAnalysisTK.jar -T UnifiedGenotyper -R /home/ralonso/BiB/Galaxy/data/Cclementina_v1.0_scaffolds.fa -I $input -o $output </command> <inputs> <param format="bam" name="input" type="data" label="bam"/> </inputs> <outputs> <data format="vcf" name="output" /> </outputs> <help> gatk </help> </tool> Best regards -- Roberto Alonso Functional Genomics Unit Bioinformatics and Genomics Department Prince Felipe Research Center (CIPF) C./Eduardo Primo Yúfera (Científic), nº 3 (junto Oceanografico) 46012 Valencia, Spain Tel: +34 963289680 Ext. 1021 Fax: +34 963289574 E-Mail: ralonso(a)cipf.es

2 1