March 2014 - galaxy-dev - lists.galaxyproject.org

ENA SRA data retrieval with Cloudman
by Mohammad Heydarian 02 Apr '14

02 Apr '14

Hi Cloudman Development Team, I noticed on the most recent Cloudman release (revision 11219:5c789ab4144a<http://bitbucket.org/galaxy/galaxy-dist/changesets/5c789ab4144a>) that there is no option to "Get Data" from the ENA SRA. We've found that retrieving large (fastq) data sets from public databases into Galaxy/Cloudman is the most convenient from the ENA SRA option in the "Get Data" menu. Will this option be coming back, or is the ENA SRA option permanently removed? Cheers, Mo Heydarian

2 4

Bowtie2 and tophat2 data library set up
by Huayan Gao 01 Apr '14

01 Apr '14

Hi, I wanted to use bowtie2 and tophat2 locally. I remembered you have instructions about how to set up the library here, but it is not accessible any more. https://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup Can you direct me to the right page or send me the documents please? Thanks, Huayan

2 5

Managing large database lists in tool wrappers?
by Smithies, Russell 01 Apr '14

01 Apr '14

Does anyone have a workable solution to managing and displaying large lists in tool wrappers? eg. a database list containing hundreds of enteries in the blast+ tool? (we have 1,056 different blast databases). As a work-around we've split our blast databases into species (eg cow, sheep, fungus, clover, fish, rice, etc) then faked a multi-select to pull in those sub-database files with conditional selectors like this: <conditional name="db_opts"> <param name="db_opts_selector" type="select" label="Nucleotide Database Type"> <option value="pub" selected="True">Pub Databases</option> <option value="cattle">Cattle Databases</option> <option value="sheep">Sheep Databases</option> <option value="deer">Deer Databases</option> <option value="ruminant">Ruminant Databases</option> <option value="dog">Dog Databases</option> <option value="fungi">Fungi Databases</option> <option value="invertebrates">Invertebrates Databases</option> <option value="microarrays">Microarrays Databases</option> <option value="marsupials">Marsupials Databases</option> <option value="rodent">Rodent Databases</option> <option value="viruses">Viruses Databases</option> <option value="human">Human Databases</option> <option value="microarrays">Microarrays Databases</option> </param> <when value="ruminant"> <param name="database" type="select" label="Ruminant Nucleotide BLAST database" help=""> <options from_file="blastdb.ruminant.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when> <when value="misc"> <param name="database" type="select" label="Misc Nucleotide BLAST database" help=""> <options from_file="blastdb.misc.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when> We end up with a lot of duplication between tool wrappers (I've never managed to get an <include> tag working) and these aren't exactly simple to maintain so does anyone have a better solution? I had thought about tacking an extra field onto the .loc files with a "species" and doing some grouping with that but haven't progressed that idea very far yet. Any suggestions greatly appreciated :) Thanx, --Russell

3 6

compressed file datatype
by Sanka, Ravi 31 Mar '14

31 Mar '14

Greetings, I am trying to add a new datatype in our Galaxy install to represent a TAR file (we are trying to add a custom tool that produces a tarball of its output). Is it possible to create a datatype of a TAR file (or ZIP, BZIP, GZIP, or any other type of compressed file)? If so, what settings need to be used with creating the XML instance of the new datatype in datatypes_conf.xml. ---------------------------------------------- Ravi Sanka ICS – Sr. Bioinformatics Engineer J. Craig Venter Institute 301-795-7743 ----------------------------------------------

1 0

April 2014 Galaxy Update Newsletter: GCC2014 Talk Abstracts due this Friday
by Dave Clements 31 Mar '14

31 Mar '14

1 0

Re: [galaxy-dev] . Output file to history (Sebastian Luna Valero)
by Jun Fan 31 Mar '14

31 Mar '14

Hi Sebastian, I am not an expert in galaxy developing. So the following content is just my personal opinion. I do not think it is possible to meet you requirement. I reckon Galaxy searches the certain folder, defined in the universal_wsgi.ini by a parameter called #collect_outputs_from = new_file_path,job_working_directory. On top of that, a job id is required, which is the reason in the dynamic multiple output job id is part of the file name. If your script.py cannot follow the rules, Galaxy for sure cannot recognize the outputs. Normally in this kind of case, I write an additional script to execute the script.py with its original parameter and do the file manipulation afterwards. Of course this is not ideal, but I do nothing about python. It may be possible to write some complicated python code within the command tag of your wrapper file to do this. I am here also waiting for experts' opinions/comments. Best regards! Jun On 3/21/14 9:28 AM, Sebastian Luna Valero wrote: > > Dear All, > > I am trying to add a new tool in Galaxy and I have the following problem. > > Let me explain a simplified example. Let's imagine that my script > works from CLI as follows: > > python script.py --input "input-file" --pattern "output-pattern" > > After processing "input-file", the script writes several output files > and their name is given according to "output-pattern". > > The number of output files depends on the content of "input-file". The > output files are written to the current working directory where > script.py is located. > > In the simplest scenario, I get only one output file in the working > directory. > > My problem is that I would like to see the output files in Galaxy's > history without modifying the CLI interface. > > To solve this problem, I have looked at the wiki page: > > https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#N > umber_of_Output_datasets_cannot_be_determined_until_tool_run > > and the email here: > > http://dev.list.galaxyproject.org/Multiple-output-files-do-not-appear- > in-history-td4660470.html > > However, I think that mine is a different scenario. I do not want to > add new output parameters in the CLI. I would like Galaxy to bring > these output files to my history without modifying the CLI options, is > that possible? > > Many thanks in advance for your help! > > Best regards, > Sebastian.

3 2

Debugging DRMAA interface: what to do when job
by Luca Toldo 31 Mar '14

31 Mar '14

Dear Galaxians, I have galaxy running fine on a 8cpu machine, however I need to have it submitting jobs to a linux cluster using PBS. I got universe and job_conf.xml setup and as far as I use the local runner, everything is fine. However, whenever I try (e.g. clustalw) to use the DRMAAJobRunner, the UI shows "this job is waiting to run" and nothing literally happens. My question is "what to do to test what's going on". I searched for a tutorial but coulnd't find it... Looking forward your advice and thankyou so much ! Luca

2 1

trackster is not working on the vrelease_2014.02.10--2--29ce93a13ac7
by Shu-Yi Su 31 Mar '14

31 Mar '14

Hi all, We have recently updated our local Galaxy installation to vrelease_2014.02.10--2--29ce93a13ac7 (database version is 118). I found that the Trackster is not working. I have checked the latest commits related to Trackster bugs. So i have updated theses files: ./static/scripts/viz/trackster.js (commits date: 2014-02-28) ./static/scripts/viz/trackster_ui.js (commits date: 2014-02-28) ./static/scripts/viz/trackster/tracks.js (commits date: 2014-03-16) ./static/scripts/utils/utils.js (commits date: 2014-03-19) ./static/scripts/utils/config.js (commits date: 2014-03-15) But, it is still not working. I have tried different format…bam, bed, sam….all are not working. I looked into all possible files I can think about that might cause the problems but still don't have any clues. I also looked into the log, and there is no error, but i noticed that there is difference from the log where the trackster was working in previous installation. The log where the trackster is not working: 1. "GET /galaxy-dev/visualization/trackster?dataset_id=f04ecb1d3d259245&hda_ldda=hda The log where the trackster is working (from the old installation): 1. "GET /galaxy/visualization/trackster?dataset_id=5d38b380cba2f4a6&hda_ldda=hda…… 2. "GET /galaxy/api/datasets/5d38b380cba2f4a6?hda_ldda=hda&data_type=converted_datasets_state…. 3. "GET /galaxy/api/datasets/5d38b380cba2f4a6?data_type=data&chrom=chr2L….. It looks like that the trackster in new version of Galaxy does not execute the second and the third steps. Here is the screen shot when i clicked "View in new visualization" (there is nothing shown up---blank) Any ideas or suggestions are appreciated. Thank you very much. Best, Shu-Yi

4 11

Fetch Taxonomic Representation : Fails : terminated by signal11
by Nilaksha Neththikumara 31 Mar '14

31 Mar '14

I'm running a local instance of Galaxy and have installed all the tool dependencies and generated dependent files for Taxonomy Fetch. After running the tool for like 20 minutes I get the following error. *Dataset 43: Fetch taxonomic representation on data 37* Tool execution generated the following error message: Execution of taxBuilder terminated by signal 11 Any help would be much appreciated.

1 0

homebrew python
by Joshua Udall 28 Mar '14

28 Mar '14

For various reasons, I installed a Homebrew of python instead of the system version on OSX 10.9.2. Now, when galaxy initializes, it isn't looking in the right location for eggs (or they aren't placed in the right spot on my system). I was able to manually install several of the eggs and the galaxy startup would move to the next egg until here. Some eggs are out of date, attempting to fetch... Warning: MarkupSafe (a dependent egg of WebHelpers) cannot be fetched Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 37, in <module> c.resolve() # Only fetch eggs required by the config File "/Users/galaxy/galaxy-old3/lib/galaxy/eggs/__init__.py", line 345, in resolve egg.resolve() File "/Users/galaxy/galaxy-old3/lib/galaxy/eggs/__init__.py", line 195, in resolve return self.version_conflict( e.args[0], e.args[1] ) File "/Users/galaxy/galaxy-old3/lib/galaxy/eggs/__init__.py", line 226, in version_conflict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 588, in resolve The `plugin_env` should be an ``Environment`` instance that contains pkg_resources.DistributionNotFound: numpy==1.6.0 Fetch failed. I know numpy, WebHelpers, MarkupSafe are installed and they are current (maybe too current?) ... what would be a good way to resolve the conflict? The manual download below fails on a dozen packages or so. sudo python ./scripts/make_egg_packager.py py2.7-macosx-10.9-x86_64-ucs2 Using Python interpreter at /usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python, Version 2.7.6 (default, Mar 24 2014, 14:39:40) [GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] This platform is 'py2.7-macosx-10.9-x86_64-ucs2' Override with: make_egg_packager.py <forced-platform> Completed packager is 'egg_packager-py2.7-macosx-10.9-x86_64-ucs2.py'. To fetch eggs, please copy this file to a system with internet access and run with: python egg_packager-py2.7-macosx-10.9-x86_64-ucs2.py -- Joshua Udall Assistant Professor 295 WIDB Plant and Wildlife Science Dept. Brigham Young University Provo, UT 84602 801-422-9307 Fax: 801-422-0008 USA

3 2