November 2008 - galaxy-user - lists.galaxyproject.org

Tool XML Question - using option/from_file name in 'label'
by Assaf Gordon 19 Dec '08

19 Dec '08

Hello, I have a tool which has an <option> with 'from_file' attribute: ... <param name="database" type="select" label="Mapping Database"> <options from_file="mapping_filter_databases.txt"> <column name="name" index="1"/> <column name="value" index="0"/> </options> </param> ... I want to use the database name in the label of the output dataset: <outputs> <data format="fasta" name="output_mapped" label="Mappers of $input.name to $database" /> </outputs> The '$database' variables is replaced with the database's filename (the 'value' column). I'd like it to be the 'name' column, but using "$database.name" throws an runtime exception: NotFound: cannot find 'name' while searching for 'database.name' (full backtrace is listed below). Is there a way to display the name instead of the value in an <option> clause ? Thanks! Gordon. URL: http://tango/tool_runner/index File 'build/bdist.solaris-2.11-i86pc/egg/weberror/evalexception/middleware.py', line 364 in respond File 'build/bdist.solaris-2.11-i86pc/egg/paste/debug/prints.py', line 98 in __call__ File 'build/bdist.solaris-2.11-i86pc/egg/paste/wsgilib.py', line 539 in intercept_output File 'build/bdist.solaris-2.11-i86pc/egg/beaker/session.py', line 103 in __call__ File 'build/bdist.solaris-2.11-i86pc/egg/paste/recursive.py', line 80 in __call__ File 'build/bdist.solaris-2.11-i86pc/egg/paste/httpexceptions.py', line 632 in __call__ File '/media/sdb1/galaxy/galaxy_devel/lib/galaxy/web/framework/base.py', line 126 in __call__ body = method( trans, **kwargs ) File '/media/sdb1/galaxy/galaxy_devel/lib/galaxy/web/controllers/tool_runner.py', line 45 in index template, vars = tool.handle_input( trans, params.__dict__ ) File '/media/sdb1/galaxy/galaxy_devel/lib/galaxy/tools/__init__.py', line 708 in handle_input out_data = self.execute( trans, incoming=params ) File '/media/sdb1/galaxy/galaxy_devel/lib/galaxy/tools/__init__.py', line 915 in execute return self.tool_action.execute( self, trans, incoming=incoming, set_output_hid=set_output_hid ) File '/media/sdb1/galaxy/galaxy_devel/lib/galaxy/tools/actions/__init__.py', line 159 in execute data.name = fill_template( output.label, context=params ) File '/media/sdb1/galaxy/galaxy_devel/lib/galaxy/util/template.py', line 9 in fill_template return str( Template( source=template_text, searchList=[context] ) ) File '<string>', line 28 in respond NotFound: cannot find 'name' while searching for 'database.name'

2 1

Workflow improvement requests (long)
by Assaf Gordon 02 Dec '08

02 Dec '08

Dear all, Recently, users (of our local galaxy server) started using workflows, and are very pleased. However, as workflows get more complicated, it gets harder to track the input and output of the workflows. I'd like to share an example, to illustrate the problems that we encounter. The workflow (pictured in the attached 'workflow.jpg') takes 4 input datasets, and produces 4 output datasets. The first problem is that there's no way to differentiate between the input datasets (They appear simply as "Step 1: Input dataset", "Step 2: Input Dataset", etc). Since each dataset has a specific role, I've had to print the workflow and give the users instructions as to which dataset (in their history) goes into what dataset. (see attached 'crosstab_workflow_input_datasets.jpg'). The second problem is that whenever I change something in the workflow and save it - the order of the dataset change! So what was once dataset 1, can now be dataset 2,3 or 4. Users have no way of knowing this... (keen users might notice the the description of the first tool changed from "Output dataset 'output' from step 2" to "Output dataset' output' from step 4" - but this is very obscure...). The third problem is that once the workflow completes, the resulting dataset have cryptic names such as "Join two queries on Data 10 and Data 2". Since "Data 10" is "Awk on Data 8" and data-8 is "Generic Annotations on Data 7 and Data 1" and data-7 is "Intersect data 1 and data 6" - it gets a bit hard to know what's going on. (see attached 'crosstab_history.png'). For the meantime, I've simply gave written instructions on what each dataset means (see attached 'crosstab_workflow_dataset_explnanations.jpg). If I may suggest a feature - it would be great if I could name a dataset inside the workflow. Instead of naming it "Input dataset" I could give it a descriptive name, so even if the order of the input datasets changes, users will know which dataset goes into which input. Regarding the output dataset names, the 'label' option in the tools' XML is a good start, but still creates very long, hard-to-understand names. Another great feature would be the possibility to add an 'output label' for each step in the workflow. Regardless of the above, I'd like to say (once again) that Galaxy is a great tool, and workflows are really cool - we have several long workflows which do wonderful things. Thanks for reading so far, Gordon.

4 6

Moving and processing large files: will a local installation help?
by Johannes Waage 01 Dec '08

01 Dec '08

Hi all, I'm moving quite a lot of large files (>2 GB) between Galaxy and our local network for processing (mainly sequence retrieval and interval processing). This is a tad on the slow side; will a local installation of Galaxy help improve the speed? Or will the gain in transfer speed be lost in increased processing time (I do not plan on installing on a cluster)? Thankyou in advance, Johannes Waage, Uni. of Copenhagen, Denmark

2 1

prevent tool escaping input
by James Casbon 26 Nov '08

26 Nov '08

Hi, how can I prevent a tool escaping the input to a tool's text field? For example, if I put a [ in the text field, I see it in the console as __ob__ on the command line? thanks, James

2 2

Re: [galaxy-user] Workflow improvement requests
by Eric Schauberger 25 Nov '08

25 Nov '08

I second the request on sometype of labeling system for the workflow--at least a numbering system. I made a workflow with many inputs, then when I tried it out I realized that the first input that I was joining with the second input was intermixed and unidentifable. Then I realized that the inputs are ordered in their creation order and not anytype of order how they are placed. Since I was making many, many, inputs I simply made a bunch of them at once and didn't keep track of their order or where I put them. Thank again for the sweet tool. Eric On Tue, Nov 25, 2008 at 12:00 PM, <galaxy-user-request(a)bx.psu.edu> wrote: > Send galaxy-user mailing list submissions to > galaxy-user(a)bx.psu.edu > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user > or, via email, send a message with subject or body 'help' to > galaxy-user-request(a)bx.psu.edu > > You can reach the person managing the list at > galaxy-user-owner(a)bx.psu.edu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of galaxy-user digest..." > > > Today's Topics: > > 1. Re: Workflow improvement requests (long) (James Casbon) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 25 Nov 2008 11:14:14 +0000 > From: "James Casbon" <casbon(a)gmail.com> > Subject: Re: [galaxy-user] Workflow improvement requests (long) > To: "Assaf Gordon" <gordon(a)cshl.edu> > Cc: galaxy-user(a)bx.psu.edu > Message-ID: > <15d850bd0811250314h4075f1edld1ea2dc6b1a76fef(a)mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Hi Everyone, > > Slightly off-topic, but I see you have awk in your workflows. Awk > could work on text, tabular, and other formats but I'd rather not > define a new tool for each input type. > > Is there a way to define a tool which accepts more any type of input? > It should ideally preserve the format in the output as well. > > thanks, > James > > 2008/11/14 Assaf Gordon <gordon(a)cshl.edu>: > > Dear all, > > > > Recently, users (of our local galaxy server) started using workflows, and > > are very pleased. However, as workflows get more complicated, it gets > harder > > to track the input and output of the workflows. > > > > I'd like to share an example, to illustrate the problems that we > encounter. > > > > The workflow (pictured in the attached 'workflow.jpg') takes 4 input > > datasets, and produces 4 output datasets. > > > > The first problem is that there's no way to differentiate between the > input > > datasets (They appear simply as "Step 1: Input dataset", "Step 2: Input > > Dataset", etc). Since each dataset has a specific role, I've had to print > > the workflow and give the users instructions as to which dataset (in > their > > history) goes into what dataset. (see attached > > 'crosstab_workflow_input_datasets.jpg'). > > > > The second problem is that whenever I change something in the workflow > and > > save it - the order of the dataset change! > > So what was once dataset 1, can now be dataset 2,3 or 4. > > Users have no way of knowing this... (keen users might notice the the > > description of the first tool changed from "Output dataset 'output' from > > step 2" to "Output dataset' output' from step 4" - but this is very > > obscure...). > > > > The third problem is that once the workflow completes, the resulting > dataset > > have cryptic names such as "Join two queries on Data 10 and Data 2". > Since > > "Data 10" is "Awk on Data 8" and data-8 is "Generic Annotations on Data 7 > > and Data 1" and data-7 is "Intersect data 1 and data 6" - it gets a bit > hard > > to know what's going on. (see attached 'crosstab_history.png'). > > > > For the meantime, I've simply gave written instructions on what each > dataset > > means (see attached 'crosstab_workflow_dataset_explnanations.jpg). > > > > > > If I may suggest a feature - it would be great if I could name a dataset > > inside the workflow. Instead of naming it "Input dataset" I could give it > a > > descriptive name, so even if the order of the input datasets changes, > users > > will know which dataset goes into which input. > > > > Regarding the output dataset names, the 'label' option in the tools' XML > is > > a good start, but still creates very long, hard-to-understand names. > > Another great feature would be the possibility to add an 'output label' > > for each step in the workflow. > > > > > > Regardless of the above, I'd like to say (once again) that Galaxy is a > great > > tool, and workflows are really cool - we have several long workflows > which > > do wonderful things. > > > > > > Thanks for reading so far, > > Gordon. > > > > > > _______________________________________________ > > galaxy-user mailing list > > galaxy-user(a)bx.psu.edu > > http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user > > > > > > > ------------------------------ > > _______________________________________________ > galaxy-user mailing list > galaxy-user(a)bx.psu.edu > http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user > > > End of galaxy-user Digest, Vol 29, Issue 16 > ******************************************* > -- ________________________________________________________ Eric M Schauberger Physician Scientist Training Program (DO/PhD) Genetics Program Ewart Lab MSU College of Osteopathic Medicine (MSUCOM) Email: Schaube2(a)msu.edu Skype: Emschaub See my availability: http://www.timebridge.com/mytime/eschauberger __________________________________________________________

1 0

Re: [galaxy-user] galaxy-user Digest, Vol 29, Issue 13
by Ross 20 Nov '08

20 Nov '08

AFAIK, both $userId (the user numeric id) and $userEmail are added to the parameters in jobs/__init__.py, so should work... On Thu, Nov 20, 2008 at 12:00 PM, <galaxy-user-request(a)bx.psu.edu> wrote: > Date: Wed, 19 Nov 2008 13:45:24 -0500 > From: Assaf Gordon <gordon(a)cshl.edu> > Subject: [galaxy-user] Using user name in a tool's XML > To: galaxy-user(a)bx.psu.edu > Message-ID: <49245EC4.7020402(a)cshl.edu> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hello, > > Is it possible to access the user name inside a tool's XML configuration > file ? > > I want to pass the user name (the user who executed the job) as an > argument to the tool. Is there such a variable? > > Something like: > > <tool id="mytool" name="foobar"> > <command>mytool.sh --file $input --title $input.name --user > $galaxy_user</command> > ... > ... -- Ross Lazarus, Associate Professor, DACP, Harvard Medical School. Director of Bioinformatics, Channing Laboratory, BWH 181 Longwood Ave., Boston MA 02115, USA.

1 0

Using user name in a tool's XML
by Assaf Gordon 19 Nov '08

19 Nov '08

Hello, Is it possible to access the user name inside a tool's XML configuration file ? I want to pass the user name (the user who executed the job) as an argument to the tool. Is there such a variable? Something like: <tool id="mytool" name="foobar"> <command>mytool.sh --file $input --title $input.name --user $galaxy_user</command> ... ... Thanks, Gordon

1 0

file conversion for viewing
by James D Cavalcoli 17 Nov '08

17 Nov '08

This is probably a 'newbie' question, but after many years, I'm getting caught up in sequence assembly analysis again. I've been looking around for a way to view *.ace file in a genome browser such as the UCSC or IGV? Is there any straightforward way to view assemblies from Illumina or 454 reads which are in *.ace format in the .bed or .wig formats? Thanks in advance! Jim Cavalcoli Center for Computational Medicine and Biology, University of Michigan

2 1

Running development and live servers side-by-side
by Chris Cole 17 Nov '08

17 Nov '08

What is the best way to work with development and live versions of Galaxy concurrently and being able to have access to the same data (e.g. user histories)? Currently, I'm trying to run two instances of galaxy at the same time using the same MySQL database back-end and pointing to the same user files (via softlinks). Does this way madness lie? Although, it does seem to be working... I'm asking this because if we do go down the route of deploying Galaxy here, we'd like to be able to be able to test new tools without constantly having to restart the server and annoy the users. Cheers, Chris

2 1

Megablast wrapper
by Henrikki Almusa 12 Nov '08

12 Nov '08

Hi, Just wanted to let you know that I had some problems with using the Megablast wrapper in the 'tools/metag' directory. I put the tool to run a sge cluster, but ran into problem. Blast didn't create any output at all but reported successful run in galaxy website. The output and error files were empty (outside of output file having the commandline to be executed). The problem was that megablast kept on segfaulting in the cluster. After testing the blast command manually (from which I removed the '-a 8' option without thinking about that) it run without problems. Finally I realised this and removed the option '-a 8' from the wrapper and got a correct result into galaxy as well. The cluster that I use here has 2 cores on each node. Regards, -- Henrikki Almusa

1 0