November 2011 - galaxy-dev - lists.galaxyproject.org

Quota will not decrease with permanent delete
by Mary Anne Alliegro 19 Jan '12

19 Jan '12

Hi Galaxy Users, I have permanently deleted numerous files. My usage % has decreased, but this is NOT reflected in my Gb report (upper right)- it remains the same. Am I missing some phantom trash bin? If not, will Galaxy recalculate my Gb usage so that I may proceed with my project? Thank you, Mary Anne

5 6

rgenetics & galaxy
by Ivan Merelli 17 Jan '12

17 Jan '12

Hi, I'm trying to use rgenetics in galaxy, but I have some problem in installting rpy-1.0.3 with R-2.13.1. I found this guide, which is about the same problem of mine: http://gmod.827538.n3.nabble.com/Rpy-1-0-3-and-R-2-13-1-td3359590.html but didn't work for me (although the part about the modulefile is quite obscure for me). The problem seems to be correlated to an header file (Rdevices.h) which is no more present in the latest R distribution. Any hints to solve this problem? May I use rpy2? Best, Ivan

3 2

Re: [galaxy-dev] Tool shed and datatypes
by Jim Johnson 05 Jan '12

05 Jan '12

Greg, It would be great if there were a way to expand upon the core datatypes using the ToolShed. Would it be possible to have a separate datatype repository within the ToolShed? Datatype name="" description="" datatype_dependencies=[] definition=<python code> The tool config could be expanded to have requirement for datatypes. <requirement type="datatype">ssmap</requirement> Table datatype Column | Type | Modifiers -------------+-----------------------------+--------------------------------------------------- id | integer | not null default nextval('datatype_id_seq'::regclass) name | character varying(255) | version | character varying(40) | description | text | definition | text | UNIQUE (name) Table datatype_datatype_association Column | Type | Modifiers -------------+-----------------------------+--------------------------------------------------- id | integer | not null default nextval('datatype_id_seq'::regclass) datatype_id | integer | requires_id | integer | FOREIGN KEY (datatype_id) REFERENCES datatype(id) FOREIGN KEY (requires_id) REFERENCES datatype(id) Then for my mothur metagenomics tools I could define: name="ssmap" description="Secondary Structure Map" version="1.0" datatype_dependencies=[tabular] definition= from galaxy.datatypes.tabular import Tabular class SecondaryStructureMap(Tabular): file_ext = 'ssmap' def __init__(self, **kwd): """Initialize secondary structure map datatype""" Tabular.__init__( self, **kwd ) self.column_names = ['Map'] def sniff( self, filename ): """ Determines whether the file is a secondary structure map format A single column with an integer value which indicates the row that this row maps to. check you make sure is structMap[10] = 380 then structMap[380] = 10. """ ... Then the align.check.xml tool_config could require the 'ssmap' datatype: <tool id="mothur_align_check" name="Align.check" version="1.19.0"> <description>Calculate the number of potentially misaligned bases</description> <requirements> <requirement type="binary">mothur</requirement> <requirement type="datatype">ssmap</requirement> </requirements> > John, > > I've been following this message thread, and it seems it's gone in a direction that differs from your initial question about the possibility for Galaxy to handle automatic editing of the datatypes_conf.xml file when certain Galaxy tool shed tools are automatically installed. There are some complexities to consider in attempting this. One of the issues to consider is that the work for adding support for a new datatype to Galaxy lies outside of the intended function of the tool shed. If new support is added to the Galaxy code base, an entry for that new datatype should be manually added to the table at the same time. There may be benefits to enabling automatic changes to datatype entries that already exist in the file (e.g., adding a new converter for an existing datatype entry), but perhaps adding a completely new datatype to the file may not be appropriate. I'll continue to think about this - send additional thought and feedback, as doing so is always helpful > > Thanks! > > Greg > > > On Oct 5, 2011, at 11:48 PM, Duddy, John wrote: > >> One of the things we’re facing is the sheer size of a whole human genome at 30x coverage. An effective way to deal with that is by compressing the FASTQ files. That works for BWA and our ELAND, which can directly read a compressed FASTQ, but other tools crash when reading compressed FASTQ filesfiles. One way to address that would be to introduce a new type, for example “CompressedFastQ”, with a conversion to FASTQ defined. BWA could take both types as input. This would allow the best of both worlds – efficient storage and use by all existing tools. >> >> Another example would be adding the CASAVA tools to Galaxy. Some of the statistics generation tools use custom file formats. To be able to make the use of those tools optional and configurable, they should be separate from the aligner, but that would require that Galaxy be made aware of the custom file formats – we’d have to add a datatype. >> >> John Duddy >> Sr. Staff Software Engineer >> Illumina, Inc. >> 9885 Towne Centre Drive >> San Diego, CA 92121 >> Tel: 858-736-3584 >> E-mail: jduddy at illumina.com >> >> From: Greg Von Kuster [mailto:greg at bx.psu.edu] >> Sent: Wednesday, October 05, 2011 6:25 PM >> To: Duddy, John >> Cc: galaxy-dev at lists.bx.psu.edu >> Subject: Re: [galaxy-dev] Tool shed and datatypes >> >> Hello John, >> >> The Galaxy tool shed currently is not enabled to automatically edit the datatypes_conf.xml file, although I could add this feature if the need exists. Can you elaborate on what you are looking to do regarding this? >> >> Thanks! >> >> >> On Oct 5, 2011, at 1:52 PM, Duddy, John wrote: >> >> >> Can we introduce new file types via tools in the tool shed? It seems Galaxy can load them if they are in the datatypes configuration file. Does tool installation automate the editing of that file? >> >> >> John Duddy >> Sr. Staff Software Engineer >> Illumina, Inc. >> 9885 Towne Centre Drive >> San Diego, CA 92121 >> Tel: 858-736-3584 >> E-mail: jduddy at illumina.com >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ >> >> Greg Von Kuster >> Galaxy Development Team >> greg at bx.psu.edu >>

7 15

Tools accessing web services
by Amelia Ireland 12 Dec '11

12 Dec '11

Apologies in advance for the newbie question. I have a script that uses various UniProt web services. The script sends off a request to the UniProt server, waits for it to be returned, and then writes the results to a file. This works fine as a standalone script, but when plugged into Galaxy, I get this error: ⇝ KeyError: 'GALAXY_URL' URL: http://127.0.0.1:8080/tool_runner/index Does Galaxy "intervene" in the operation of all tools that access outside web services, and if so, do I need to rewrite the tool as one that acts as a data source (like the UCSC browser, biomart, etc.)? The data sources documentation did not mention whether this was necessary or not. Thanks! -- Amelia Ireland Scientific Curator, Gene Ontology project http://www.berkeleybop.org || http://www.geneontology.org Preventing Euthanasia Through Rescue: http://petrescueinc.org

2 2

Multiple file box in Galaxy XML
by Roger Ngo 09 Dec '11

09 Dec '11

Hi all, I was wondering if it was possible for a user to specify the number of files to upload in Galaxy and have multiple file input boxes to upload updated on the wrapper page accordingly. A user will have from one to many files needed to be uploaded from a job. Right now I can see that galaxy does have the option for 1 file with the param "file". With an options tagset, I can have the user specify a range. The easiest solution I can think of right now is to have a 1-N option tag set and create 1-N file upload boxes corresponding to the option number. Is there a better way already built in to Galaxy to do this? Thanks!

4 5

Version String
by SHAUN WEBB 09 Dec '11

09 Dec '11

Hi. I am using the tag: <version_string>path to tool --version</version_string> In my xml file. I would expect the tool version to be printed in the information window for each dataset but the version field is blank. I am running the latest Galaxy dist, is there anything else I need to do? I have tried using <version_command> also. Thanks Shaun -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

3 2

SICER bug in Workflow
by Susan Newman 07 Dec '11

07 Dec '11

I'm working on a SOLiD ChIP-Seq workflow in which I want to use SICER for peak detection. In attempting to add SICER to the workflow, I've run into some difficulty: 1) My input is an interval file which I've created using Convert SAM. In order to connect to the SICER module, I had to use a trick by setting up the connection with the Convert SAM set to BED. The SICER algorithm fails to run when tested with the bed file. If I then change the connected Convert SAM to interval, SICER will run. 2) I need to change the default run parameters but they are not saved in the workflow. There is always reversion to the default values. I get a Server error notice. This workflow also contains Bowtie and MACS. Parameter changes for these algorithms are saved correctly. 3) I would like to run SICER twice in the workflow with different parameters. Is it allowable to load SICER a second time? May I run from the same interval conversion file? I tried this and again could not get the input file recognized as a valid connection. Thanks in advance, Susan Susan S. Newman, Laboratory Manager Genomics Core Facility, L2012 Pennington Biomedical Research Center 6400 Perkins Road Baton Rouge, LA 70808 225-763-0255 (o) 225-763-0257 (lab) http://gcf.pbrc.edu

2 1

Re: [galaxy-dev] [Internal - Galaxy-dev #2159] (New) 2 questions about Galaxy's functional testing support
by Greg Von Kuster 06 Dec '11

06 Dec '11

Hello Eric, > Submitted by epaniagu(a)cshl.edu > Hi all, > > I've read the Wiki page on Writing Functional Tests (http://wiki.g2.bx.psu.edu/Admin/Tools/Writing%20Tests) and I've been looking through test/base and test/functional and I am left with two questions: > > * Is it possible to write a test to validate metadata directly on an (optionally composite) output dataset? I'm sure this is possible, but it would require enhancements to the current functional test framework. > Everything described on the above page is file oriented. I see that there is TwillTestCase.check_metadata_for_string, but as far as I can tell this is a bit nonspecific since it appears to just do a text search on the Edit page. This is correct. > I don't yet fully understand the context in which tests run, but is there some way to access a "live" dataset's metadata directly, either as a dictionary or just as attributes? Or even to get the actual dataset object? Not with the current functional test framework. Doing this would require enhancements to the framework. > > * Does the test harness support retaining output files only for failed tests? Ideally with a cap on how much output data to save. If not, would this be difficult to configure? I'm not sure what you mean by "output files" in your question. If you mean output datasets that result from running a functional test for a tool, then I believe there is no difference if the test passed or failed. > > Thanks, > Eric > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

3 6

Can't view data files from published histories, only imported histories
by Liisa Koski 06 Dec '11

06 Dec '11

Hi, I found a weird bug. I am trying to view data files by clicking on the 'eye' icon from a published history on my local galaxy installation. When I click on the eye I get a 'Server Error' and in the log file I get the following. Error - <type 'exceptions.NameError'>: global name 'data' is not defined URL: http://domain:8080/u/user/d/8bdb720fee635874 File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File 'galaxy_dist/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File 'galaxy_dist/lib/galaxy/web/controllers/dataset.py', line 693 in display_by_username_and_slug trans.response.set_content_type( data.get_mime() ) NameError: global name 'data' is not defined If I import the history into my own history pane I can click on the eye and see the data with no errors. Any ideas how I can see the data from the published histories? Thanks, Liisa Liisa Koski Bioinformatics Programmer

2 6

GATK / R local install configuration
by Carlos Borroto 05 Dec '11

05 Dec '11

Hi, I'm testing the GATK pipeline and I ran into a problem with "Variant Recalibrator" tool. It seems I don't have correctly configure R and GATK on my instance, as this tool is failing with this error: mv: /Volumes/Data/Users/cjavier/galaxy_central/database/files/000/dataset_393.dat.pdf: No such file or directory I see this PDF is builded with R and I also see this in the log file: INFO 17:11:44,260 VariantRecalibrator - Executing: Rscript /Volumes/Data/Users/cjavier/galaxy_central/database/files/000/dataset_393.dat WARN 17:11:48,407 RScriptExecutor - RScript exited with 1. Run with -l DEBUG for more info. In the testing server this tool does work: http://test.g2.bx.psu.edu/u/cjav/h/variant-recalibrator---tutorial At least after making sure the input VCF file has matching annotations string to the one you want to select in Galaxy. Any help on what need to be done to get this configuration right. BTW I already have R and Rpy correctly configured and I can run tools like "Statistics/Summary Statistics". Thanks! Carlos

2 2