August 2010 - galaxy-dev - lists.galaxyproject.org

Fwd: [galaxy-user] Tool Integration: SOAPaligner/soap2
by Kelly Vincent 18 Aug '10

18 Aug '10

Begin forwarded message: > From: Kelly Vincent <kpvincent(a)bx.psu.edu> > Date: August 18, 2010 11:46:32 AM EDT > To: btimm(a)wisc.edu > Cc: galaxy-user(a)bx.psu.edu > Subject: Re: [galaxy-user] [galaxy-dev] Tool Integration: > SOAPaligner/soap2 > > Branden, > > Are you currently calling a script file (Python Perl, etc.) in your > command tag, or calling the tool directly? If you are running it > only with an XML file, you cannot do what you want here. The command > that will be run is generated by converting the variable values you > put in the <command> tag to the names of the actual input and output > files that Galaxy creates when the tool is run. > > However, this is not particularly difficult if you are calling a > script in the <command> tag, though it's a little convoluted. There > are several tools that deal with this situation (the NGS tools > again, in particular). There are a couple of basic situations. If > you just need to have a single file with a certain extension, simply > create a temp file with the extension specified (in Python > "tempfile.mkstemp( suffix='.ext' )" will do it). It's slightly more > complicated if you need associated files in the same directory (for > instance, Samtools expects a fasta file like "hg19.fa" plus its > index "hg19.fa.fai" to be in the same place. When you call the > indexing command with the name of the fasta file, it automatically > creates the index file with that name plus ".fai"). In this case, > you should symlink the original file to a temp file (with or without > a specified extension, as appropriate) and then get the name of that > temp file. The name of the other file is that name PLUS the > appropriate extension. For example, sam_to_bam.py does this in lines > 73 through 76. > > Note that the reason for symlinking to a temp file is that otherwise > the tool will create a new file in a database/files subdirectory, > which is not the right way to do things. So in the Samtools case, if > the uploaded fasta file is "database/files/000/dataset_23.dat", a > new file "database/files/000/dataset_23.dat.fai" will be created, > but the database won't know about it and it will not be deleted > after the tool finishes (but it still won't be available to future > use, either). > > The paster.log contains the command as generated by the <command> > tag, but it's not possible to print to that log from a tool script. > You can simply print the relevant command to sdtout or stderr from > the script and when you run it in the browser, you will see it in > the info section. > > Hopefully this makes sense--let us know if you need further > assistance. > > Regards, > Kelly > > > On Aug 17, 2010, at 1:21 PM, Branden Timm wrote: > >> Just a follow up here ... if the <command> tag needs to list all >> output files, but the tool itself does not specify output files on >> the command line, does that mean I need to write a wrapper for my >> tool that accepts each output file as a command-line parameter? >> >> Currently I've tried specifying all of the output files on the >> <command> line, where each variable corresponds to a <data> line >> under outputs, but then SOAPaligner/soap2 tries to read those non- >> existent files as FASTA files. >> >> As a more general question, is there a way to see the exact >> command(s) that Galaxy is dispatching on my behalf when I execute a >> tool? I checked paster.log but it wasn't there. It seems like >> that would be a great debugging feature. >> >> Branden >> >> On Aug 17, 2010, at 10:06 AM, Hans-Rudolf Hotz wrote: >> >>> Hi Branden >>> >>>> Hello, >>>> I'm very new to Galaxy, and trying to use SOAPaligner/soap2 as a >>>> test >>>> integration case. >>>> >>>> soap2 includes two executables, 2bwt-builder and soap. 2bwt-builder >>>> takes a FASTA files and generates a set of 13 different index >>>> files, >>>> which soap needs in order to do it's alignment. >>>> >>>> I have started by just creating the tool XML configuration for >>>> 2bwt-builder. The configuration follows: >>>> >>>> <tool id="2bwt-builder" name="2bwt-Builder"> >>>> <description>build index files for the SOAPaligner/soap2</ >>>> description> >>>> <command>2bwt-builder $input</command> >>> >>> the "command line" needs all output files listed, see: >>> http://bitbucket.org/galaxy/galaxy-central/wiki/AddToolTutorial >>> >>> >>> However, in your case: Do you really want to make an extra tool >>> for the indexing step? Wouldn't it make more sense to have the >>> indices pre-built for some genomes? >>> >>> Your soap galaxy tool can then re-use the indices again and again. >>> This is also much more space efficient, as all the user share the >>> same index files. >>> >>> Regards, Hans >>> >>> >>>> <inputs> >>>> <param type="data" format="fasta" name="input" label="Source >>>> file"/> >>>> </inputs> >>>> >>>> <outputs> >>>> <data format="tabular" name=".amb Index File"/> >>>> <data format="tabular" name=".ann Index File"/> >>>> <data format="tabular" name=".bwt Index File"/> >>>> <data format="tabular" name=".fmv Index File"/> >>>> <data format="tabular" name=".hot Index File"/> >>>> <data format="tabular" name=".lkt Index File"/> >>>> <data format="tabular" name=".pac Index File"/> >>>> <data format="tabular" name=".rev.bwt Index File"/> >>>> <data format="tabular" name=".rev.fmv Index File"/> >>>> <data format="tabular" name=".rev.lkt Index File"/> >>>> <data format="tabular" name=".rev.pac Index File"/> >>>> <data format="tabular" name=".sa Index File"/> >>>> <data format="tabular" name=".sai Index File"/> >>>> </outputs> >>>> </tool> >>>> >>>> I've used the tabular data type for the output files, which I'm >>>> not sure >>>> is correct. When the script runs, it generates 13 output files in >>>> my >>>> history, but they are all empty according to galaxy. When I look at >>>> galaxy_dist/database/files/.../, the output files have been >>>> generated >>>> correctly and are non-empty. >>>> >>>> Where am I going wrong? Thank you in advance for any advice. >>>> >>>> -- >>>> Branden Timm >>>> System Administrator >>>> Great Lakes Bioenergy Research Center >>>> University of Wisconsin >>>> btimm(a)glbrc.wisc.edu >>>> _______________________________________________ >>>> galaxy-dev mailing list >>>> galaxy-dev(a)lists.bx.psu.edu >>>> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> _______________________________________________ >> galaxy-dev mailing list >> galaxy-dev(a)lists.bx.psu.edu >> http://lists.bx.psu.edu/listinfo/galaxy-dev > > _______________________________________________ > galaxy-user mailing list > galaxy-user(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-user

1 0

Writing a tool, need wisdom on <output>
by Reid, Robert 17 Aug '10

17 Aug '10

Hi, We have begun dabbling with a local instance of Galaxy (amazingly impressive !!). I have 2 questions: 1. I have written a tool to implement Lucy (http://bioinformatics.oxfordjournals.org/cgi/content/short/17/12/1093) I can figure out how to handle output when the command line has a -o option to explicitly state the output file name. Or when the file output is always the same. I am trying to make a tool where the java executable is launched and makes an output file. The output file is named the same as the input file (input.txt) but has a new extension added to the end of it (.cal), i.e., inputfilename.txt.cal I can't for the life of me figure out the XML syntax to read this file back into galaxy as an output. Is the best strategy to write a wrapper in another language (perl, python, java), that will feed in $output, then read in the ".cal" output and spew out a file to satisfy $output for the XML ? 2. For testing, what is the best way to do the testing? I have written a few java based executable jars and the corresponding XML docs to go with them. I'd like to share with the world but I am not sure the best way to go testing. Thank you very much for your attention, Rob Reid Rob Reid, Ph.D. | Research Associate UNC Charlotte | Dept. of Bioinformatics and Genomics 9201 University City Blvd. | Charlotte, NC 28223 Phone: 704-250-5766 rreid2(a)uncc.edu | http://www.uncc.edu --------------------------------------------------------------------------- If you are not the intended recipient of this transmission or a person responsible for delivering it to the intended recipient, any disclosure, copying, distribution, or other use of any of the information in this transmission is strictly prohibited.

2 1

Tool Integration: SOAPaligner/soap2
by Branden Timm 17 Aug '10

17 Aug '10

Hello, I'm very new to Galaxy, and trying to use SOAPaligner/soap2 as a test integration case. soap2 includes two executables, 2bwt-builder and soap. 2bwt-builder takes a FASTA files and generates a set of 13 different index files, which soap needs in order to do it's alignment. I have started by just creating the tool XML configuration for 2bwt-builder. The configuration follows: <tool id="2bwt-builder" name="2bwt-Builder"> <description>build index files for the SOAPaligner/soap2</description> <command>2bwt-builder $input</command> <inputs> <param type="data" format="fasta" name="input" label="Source file"/> </inputs> <outputs> <data format="tabular" name=".amb Index File"/> <data format="tabular" name=".ann Index File"/> <data format="tabular" name=".bwt Index File"/> <data format="tabular" name=".fmv Index File"/> <data format="tabular" name=".hot Index File"/> <data format="tabular" name=".lkt Index File"/> <data format="tabular" name=".pac Index File"/> <data format="tabular" name=".rev.bwt Index File"/> <data format="tabular" name=".rev.fmv Index File"/> <data format="tabular" name=".rev.lkt Index File"/> <data format="tabular" name=".rev.pac Index File"/> <data format="tabular" name=".sa Index File"/> <data format="tabular" name=".sai Index File"/> </outputs> </tool> I've used the tabular data type for the output files, which I'm not sure is correct. When the script runs, it generates 13 output files in my history, but they are all empty according to galaxy. When I look at galaxy_dist/database/files/.../, the output files have been generated correctly and are non-empty. Where am I going wrong? Thank you in advance for any advice. -- Branden Timm System Administrator Great Lakes Bioenergy Research Center University of Wisconsin btimm(a)glbrc.wisc.edu

3 5

S.Cerevisiae Bowtie pre built index
by SHAUN WEBB 17 Aug '10

17 Aug '10

Just a word of warning to everyone. I have recently been trying to map S.Cerevisiae sequencing reads to a reference genome. The pre built index supplied by bowtie is built on the CYGD assembly and not the commonly used SGD assembly (strain S288C). I'm unsure if CYGD use a different strain or an older assembly but I have noticed differences (mainly SNPs) in the sequence files. Shaun Webb -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 3

Galaxy on python 2.7 ?
by Assaf Gordon 17 Aug '10

17 Aug '10

Hi all, Did anyone try running Galaxy with python-2.7 ? I'm in a need to upgrade from python-2.5, and was wondering if I should use the latest stable python (2.7) or upgrade to a not-so-recent version (2.6.5?). Thanks, -gordon

3 2

Tool Integration: SOAPaligner/soap2
by Branden Timm 17 Aug '10

17 Aug '10

Hello, Sorry if this is a double post, I tried to send a message earlier today and accidentally sent it incomplete. I am working on integrating SOAPaligner/soap2 into Galaxy. soap2 includes two executables, 2bwt-builder which takes a FASTA file and generates 13 index files, and soap which uses these index files for alignment. Here is my configuration for 2bwt-builder: <tool id="2bwt-builder" name="2bwt-Builder"> <description>build index files for soap</description> <command>2bwt-builder $input</command> <inputs> <param type="data" format="fasta" name="input" label="Source file"/> </inputs> <outputs> <data format="tabular" name=".amb Index File"/> <data format="tabular" name=".ann Index File"/> <data format="tabular" name=".bwt Index File"/> <data format="tabular" name=".fmv Index File"/> <data format="tabular" name=".hot Index File"/> <data format="tabular" name=".lkt Index File"/> <data format="tabular" name=".pac Index File"/> <data format="tabular" name=".rev.bwt Index File"/> <data format="tabular" name=".rev.fmv Index File"/> <data format="tabular" name=".rev.lkt Index File"/> <data format="tabular" name=".rev.pac Index File"/> <data format="tabular" name=".sa Index File"/> <data format="tabular" name=".sai Index File"/> </outputs> </tool> The tool runs in Galaxy, but when it finishes it puts 13 empty data files in my history. I can see in database/files/.../ that the output of 2bwt-builder is correct and the indexes are non-empty files. I'm guessing that tabular may be the wrong format for these output files, but I'm not really sure what other datatype they would be. Perhaps I need to define my own. Where am I going wrong? Cheers -- Branden Timm System Administrator Great Lakes Bioenergy Research Center University of Wisconsin btimm(a)glbrc.wisc.edu

1 0

Re: [galaxy-dev] [galaxy-user] Static content fails to render
by Nate Coraor 17 Aug '10

17 Aug '10

Andrew Stewart wrote: > Just installed galaxy, but the static content is failing to render and I'm not sure why. > > Screenshot of browser: http://i38.tinypic.com/34rgjlv.png > > > My httpd.conf contains the following: > > RewriteEngine on > RewriteRule ^/galaxy$ /galaxy/ [R] > RewriteRule ^/galaxy/static/style/(.*) /usr/local/src/galaxy_dist/static/june_2007_style/blue/$1 [L] > RewriteRule ^/galaxy/static/scripts/(.*) /usr/local/src/galaxy_dist/static/scripts/packed/$1 [L] > RewriteRule ^/galaxy/static/(.*) /usr/local/src/galaxy_dist/static/$1 [L] > RewriteRule ^/galaxy/favicon.ico /usr/local/src/galaxy_dist/static/favicon.ico [L] > RewriteRule ^/galaxy/robots.txt /usr/local/src/galaxy_dist/static/robots.txt [L] > RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] Hi Andrew, I've moved this over to the galaxy-dev list since it involves a local installation. Have you uncommented the following option in universe_wsgi.ini?: filter-with = proxy-prefix --nate > > > _______________________________________________ > galaxy-user mailing list > galaxy-user(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-user

1 0

XML/XSLT output in Galaxy
by Jens Lichtenberg 13 Aug '10

13 Aug '10

Hello, I am integrating a tool that produces XML output. Is there a way for me to visualize the results through HTML in galaxy by using XLSTs or something similar? Jens

2 1

minor bug: stderr is not "cloned" between histories.
by Assaf Gordon 12 Aug '10

12 Aug '10

Hi, When sharing+cloning histories between users, the "stderr" information is not cloned (in the database). Steps to reproduce: 1. as one user, run a tool that will generate error (e.g. "Compute" with an invalid python expression) The dataset is red, it's STDERR information contains the python error (see first attached image, "user_1"). 2. Share the history with another user. 3. switch to the other user, clone the shared history. 4. The failed (red) dataset will not have the STDERR information (see second attached image, "user_2"). Note that the "peek" information is properly cloned, so the beginning of the error messag appears in the peek window, but not the entire error message. Not a critical bug, but slightly annoying when users share history with me to show me specific errors. regards, -gordon

1 0

Server error from history view
by simon andrews (BI) 11 Aug '10

11 Aug '10

I'm making pretty good progress setting up our galaxy server, but have hit a snag. Since moving from an SQLite to a MySQL backend I'm getting an internal server error from the history panel on our galaxy instance. Running the galaxy server from a console gives me the following python traceback: ---------------------------------------- Exception happened during processing of request from ('127.0.0.1', 60795) Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 1068, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib64/python2.6/SocketServer.py", line 320, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib64/python2.6/SocketServer.py", line 615, in __init__ self.handle() File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 442, in handle BaseHTTPRequestHandler.handle(self) File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle self.handle_one_request() File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 437, in handle_one_request self.wsgi_execute() File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 287, in wsgi_execute self.wsgi_start_response) File "/usr/lib/python2.6/site-packages/paste/urlmap.py", line 203, in __call__ return app(environ, start_response) File "/data/private/galaxy/galaxy_dist/lib/galaxy/web/framework/middleware/xforwardedhost.py", line 21, in __call__ return self.app( environ, start_response ) File "/usr/lib/python2.6/site-packages/paste/deploy/config.py", line 164, in __call__ app_iter = self.application(environ, start_response) File "/usr/lib/python2.6/site-packages/paste/translogger.py", line 68, in __call__ return self.application(environ, replacement_start_response) File "/usr/lib/python2.6/site-packages/paste/exceptions/errormiddleware.py", line 154, in __call__ exc_info) File "/usr/lib/python2.6/site-packages/paste/translogger.py", line 67, in replacement_start_response return start_response(status, headers) File "/usr/lib/python2.6/site-packages/paste/httpserver.py", line 166, in wsgi_start_response assert 0, "Attempt to set headers a second time w/o an exc_info" AssertionError: Attempt to set headers a second time w/o an exc_info ---------------------------------------- Any ideas? Thanks Simon.

2 6