June 2010 - galaxy-dev - lists.galaxyproject.org

Setting Output Type with different input types
by Dennis Gascoigne 10 Jun '10

10 Jun '10

I have a tool that takes two input files, Input1 - a file of varying format(interval/tabular/fasta etc) and Input2 - a text format file. If I set the output format="input" it always sets to the text format, regardless of the Input1s format- which is not good for workflows where the attribute setting cannot be automated. I want the output to be of format Input1 Is there any way to specify the output format to be Input1.format or equivalent? Cheers Dennis

1 0

Clean up datsets script
by SHAUN WEBB 09 Jun '10

09 Jun '10

Hi, I recently ran the clean up datasets script to free up some storage space and this resulted in some of the active library datasets being purged from disk. This library was loaded from an external file path. When I wanted to add more files to galaxy from the same path it was easier to load the whole directory again and delete the duplicated files. I'm assuming that the cleanup script looks at these deleted datasets and purges the file they are associated with even though another current dataset also links to this file. Is there a way to check if a file is referenced by another dataset before purging or to prohibit the script from deleting files out with the default galaxy file directory? If I leave out the purge_libraries script will this stop library datasets from being removed or only deleted libraries as a whole? Thanks for your help Shaun -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 3

Fwd: Histogram Data
by Assaf Gordon 09 Jun '10

09 Jun '10

Hi all, I guess the reason Amanda needs to set the axis is to be able to generate multiple plots and then combine them together (in Illustrator, or similar). We had the same request from our users (regarding the XY-plots). Here is a modified version of the XY-plot tool that allows the users to manually set the axis. With it, they can generate multiple plots with the same scale. Similar modifications can be used for the histogram tool. http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-March/002275.html And there's also a tool which does what Amanda asked for (returning the histogram as textual values) here: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-February/002021.html You can experiment with those tool at our demo-galaxy web site (look in 'statistics' and 'visualization'): http://cancan.cshl.edu/publicgalaxy/ Hope it helps, -gordon -------- Original Message -------- Subject: [galaxy-bugs] Histogram Data Date: Tue, 8 Jun 2010 17:10:23 -0500 From: Amanda Supak <amandasupak(a)tamu.edu> To: galaxy-bugs(a)bx.psu.edu Hello Galaxy Staff! I had a quick suggestion for you! Would it be possible to make a tab-delimited output file you could download when making histograms? I noticed that you can only get the picture of a graph, which is obviously generated by your servers somehow. The only problem is when I make different histograms with different data the axises are not the same for each, and there is no way for me to change that fact. If you made an output file then I could create the graphs myself and fix it to my liking. Amanda Supak Hu Lab Texas A&M University _______________________________________________ galaxy-bugs mailing list galaxy-bugs(a)lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-bugs

1 0

Naming workflow steps for use in history
by Dennis Gascoigne 09 Jun '10

09 Jun '10

We have long complicated workflows where we have named each step. Is it possible to have these names carry over to the history when the workflow is run? The default names given by the tools aren't informative when there are 10 steps called "Plot on Data <DSID>", we would prefer if the names from the workflow were adopted - maybe apended with the data set input ID. If not, is it easy enough to hijack - which module should i check? Cheers Dennis

1 0

Incorporating Velvet and Interproscan into Galaxy
by Paszkiewicz, Konrad 08 Jun '10

08 Jun '10

I'd like to incorporate some short-read genome assembly programs into Galaxy. The Velvet assembler is a particular favourite of our facility and ideally I would like to give our users the option of running Velvet assemblies on FASTQ datasets. The ideal scenario would be for the contigs generated by velvet to then be piped into an interface to interproscan. However, in the first instance, I'd like to focus on the velvet interface. My main question centres around whether the format velvet uses to store information is representable as a composite datatype in Galaxy. Velvet comprises of two elements - a prep step (velveth) followed by the actual assembly (velvetg). Velveth generates a directory with files (the files always have the same names). Velvetg is then run with the directory containing the relevant files as a parameter. I have read through the composite datatypes wiki page but I'm not sure if this form of data storage can be represented in galaxy at the moment. Any advice would be much appreciated. All the very best, Konrad. Dr Konrad Paszkiewicz Exeter Sequencing Service, Biosciences, Stocker Road, University of Exeter, Exeter EX4 4QD, UK. http://biosciences.exeter.ac.uk/facilities/sequencing/

4 5

Make Galaxy use multiple processors
by Dennis Gascoigne 08 Jun '10

08 Jun '10

I know it must be possible to make Galaxy use more of the resources available to it but I haven't been able to figure out how. If galaxy is calling python scripts, it only seems to use a single processor to do so. We have an 8 core machine and if other binaries are spawned then they get other processors no problems, but any of the python operations though seem to operate on the same processor no matter how many of these 'python based' jobs are running i.e. If i run 5 tools simultaneously, I would expect allocation to make use of all available processing resources. Is there some config to make this happen? I am sure I am missing something basic. Cheers

5 7

Re: [galaxy-dev] Make Galaxy use multiple processors
by Dennis Gascoigne 08 Jun '10

08 Jun '10

I am seeing two python processes. One is my runner - maxing at appr. 100%, the other is my web process just ticking along at about 5%. We have a user who has kick off 5 version of SAM to Interval. I would expect to see 5 processes, preferably taking multiple processors - I am seeing 1 On Tue, Jun 8, 2010 at 7:38 AM, Dennis Gascoigne <dennis.gascoigne(a)gmail.com > wrote: > I am seeing two python processes. One is my runner - maxing at appr. 100%, > the other is my web process just ticking along at about 5%. There are 5 > instances of SAM to Interval. I would expect to see 5 processes, preferably > taking multiple processors. > > > On Mon, Jun 7, 2010 at 11:35 PM, Nate Coraor <nate(a)bx.psu.edu> wrote: > >> James Taylor wrote: >> >>> Nate, shouldn't python tools run as different processes scale out to >>> multiple processors without any special configuration though? E.g. running 5 >>> gops jobs in parallel should use multiple processes. Is there something that >>> prevents this? >>> >> >> Oh, yes, I suppose I failed to read this email properly. It should >> definitely be starting separate processes, which would remove the GIL's one >> core limitation. Dennis, are you seeing multiple python processes when you >> run these tools? >> >> --nate >> >> >> >>> On Jun 7, 2010, at 9:29 AM, Nate Coraor wrote: >>> >>> Dennis Gascoigne wrote: >>>> >>>>> I know it must be possible to make Galaxy use more of the resources >>>>> available to it but I haven't been able to figure out how. If galaxy is >>>>> calling python scripts, it only seems to use a single processor to do so. We >>>>> have an 8 core machine and if other binaries are spawned then they get other >>>>> processors no problems, but any of the python operations though seem to >>>>> operate on the same processor no matter how many of these 'python based' >>>>> jobs are running i.e. If i run 5 tools simultaneously, I would expect >>>>> allocation to make use of all available processing resources. >>>>> Is there some config to make this happen? I am sure I am missing >>>>> something basic. >>>>> >>>> >>>> Hi Dennis, >>>> >>>> This is something I've referred to on the ProductionServer wiki page, >>>> but have not yet documented. I also covered it in my Developer Conference >>>> talk (slides: http://usegalaxy.org/dev2010 ). >>>> >>>> However, probably the best place to find it right now is the >>>> conversation on the list from last month between Davide Cittaro and I: >>>> >>>> http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002634.html >>>> >>>> I'd suggest copying universe_wsgi.ini in both instances rather than >>>> moving in one, as I originally suggested. Otherwise, egg-checking routines >>>> will fail because of the missing config file. >>>> >>>> --nate >>>> >>>> Cheers >>>>> >>>>> ------------------------------------------------------------------------ >>>>> _______________________________________________ >>>>> galaxy-dev mailing list >>>>> galaxy-dev(a)lists.bx.psu.edu >>>>> http://lists.bx.psu.edu/listinfo/galaxy-dev >>>>> >>>> >>>> _______________________________________________ >>>> galaxy-dev mailing list >>>> galaxy-dev(a)lists.bx.psu.edu >>>> http://lists.bx.psu.edu/listinfo/galaxy-dev >>>> >>> >>> >> > > > -- > Dennis Gascoigne > 0407 639 995 > dennis.gascoigne(a)gmail.com > -- Dennis Gascoigne 0407 639 995 dennis.gascoigne(a)gmail.com

2 1

minor problems with the new custom dropdown-listbox control
by Assaf Gordon 07 Jun '10

07 Jun '10

Hi, The new custom listbox control has some minor problems. 1. I'm not sure when a standard listbox is converted into a custom listbox - it seems to be arbitrary (or based on some hidden criteria). With some tools, the list boxes are replaced at run time, with others - not at all. Is there any way to turn this thing off ? force a parameter in my XML file never to be converted to this custom list box ? 2. The custom listboxes have "wrap-around" mis-feature - when the last item is selected and the user presses "down" or "page-down" - the selection jumps to the first item (and vice versa). Also, On firefox 3 (linux), holding the arrow key (up or down) does not move the selection. one needs to press multiple times. On Chrom 5 (linux) holding the arrow key does work as expected. 3. I assume the list boxes are implemented as HTML, because their content can be selected with the mouse (i.e. marking the text inside the drop-down list box for a 'copy&paste' kind of way). This wreaks havoc on the GUI, the drop-down stays on screen and hides the rest of the controls. One can open two drop-downs the same time, etc. 4. When entering a value in the auto-complete editbox, the drop-down list box shows only relevant items. Clicking outside the list closes the drop-down list. clicking on the blue arrow or on the editbox itself again shows the drop-down list, but with all the items (where I would assume it should show only items matching the existing value in the auto-complete edit-box). 5. Pressing the "ENTER" key while a drop-down list is open activates the next button, instead of selecting the item and closing the list. 6. One issue happens on the local server, which uses an old revision so you might have fixed those in the meantime. For one tool, the list of input datasets is shown with this new custom list box. The items are sorted in ascii order, not in human/version/natural order (see attachments 1 & 2). The order is 1,10,11 etc. instead of 1,2,3,4. The listbox also contains an "unsepcified(?)" item at the bottom (see attachment 3). Thanks to Felix (CC'd) for reporting these issues. -gordon

3 3

Re: [galaxy-dev] minor problems with the new custom dropdown-listbox control
by SHAUN WEBB 07 Jun '10

07 Jun '10

I have a few tools that use a select box to choose multiple items, e.g: <param name="col" type="select" multiple="true" accept_default="true" label="Sample ID" > <options from_dataset="indices"> <column name="name" index="0"/> <column name="value" index="1"/> </options> </param> However the new custom dropdown-listbox does not appear to allow multiple selections. Is this code now deprecated or is there some other way to allow multiple selections from a list? Is this a browser specific error, I have only tried in Firefox. Thanks Shaun Webb -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 1

Reorganizing tool menu
by Bossers, Alex 07 Jun '10

07 Jun '10

Hi galaxy developers, Galaxy is a great tool. We have our local instance running fine and more and more tools are developed. That's also where the question is about: We can put tools in sub directories and they will be as a collapsing list under that dir-name in the tools menu on the left. What is best practice to reorganize these tools? For instance I want to have extra subdividing headers (like in the public release for NGS toolbox beta). These may be static. I assume the order of the individual tools and subdirs is jst the order as how they appear in the config xml files. Cheers and keep up the good work! Alex

2 1