- galaxy-dev - lists.galaxyproject.org

Automatically sending an email to users you've shared a history with
by Leandro Hermida 06 May '11

06 May '11

Hi all, Can Galaxy be configured to automatically send an email to users you have shared a history with (with appropriate link to Galaxy history URL included)? I tried to search the mail list history and could not find, which bring another question, is Google the only way to search the galaxy mailing list archive? regards, Leandro

2 1

Question on setting metadata on upload via API
by Duddy, John 06 May '11

06 May '11

I'm looking at extending the metadata fields for one of the supported file types. The files can get VERY large, and since I'm creating those files, I'd like to save as metadata some of the information I have on the contents. Specifically, I'd like to tag the files with information about the sample as well as the byte offsets in the file for every millionth record (to facilitate fast file splitting). I saw the sample script for uploading a file to a library via the API. It looks to me like the most direct approach would be to modify LibrariesController.create() to handle arbitrary key-value pairs to be set on the data file. However, that might not support things like having metadata values be arrays. So, does my understanding of the problem and the areas involved seem reasonable? Is there something already out there that does what I need? I saw the set_metadata.py script, but that looks like it is meant to operate directly on the files, which might not make a running instance of Galaxy too happy. Is there an API version of that (or something in one of the web controllers I can repurpose to my needs)? Thanks! John Duddy Illumina, Inc.

2 3

Problem downloading file from local Galaxy
by Marco Moretto 06 May '11

06 May '11

Hi all, I successfully installed a Galaxy instance locally. I use Galaxy with Apache. Every time I try to download a file from history (like Options -> Export to file) the browser (Firefox 3.6) reply with the error "/tmp/lFeL6erm.asc.part could not be saved, because the source file could not be read.". I can correctly view the file content with the "Display data in browser" button. Thank you for any help. --- Marco

2 2

typo in galaxy-dist/tool_conf.xml?
by Marina Gourtovaia 06 May '11

06 May '11

Line 298 of galaxy-dist/tool_conf.xml in the zipped distribution downloaded yesterday <tool file="indels/indel_table.xml" />" should be <tool file="indels/indel_table.xml" /> ? Marina -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

2 1

Workflow steps stuck in job queue - since cleanup_datasets.py
by Liisa Koski 06 May '11

06 May '11

Hi, Yesterday I ran the cleanup_datasets.py scripts as follows.. Deleting Userless Histories python cleanup_datasets.py universe_wsgi.ini -d 10 -1 Purging Deleted Histories python cleanup_datasets.py universe_wsgi.ini -d 10 -2 -r Purging Deleted Datasets python cleanup_datasets.py universe_wsgi.ini -d 10 -3 -r Purging Library Folders python cleanup_datasets.py universe_wsgi.ini -d 10 -5 -r Purging Libraries python cleanup_datasets.py universe_wsgi.ini -d 10 -4 -r Deleting Datasets / Purging Dataset Instances python cleanup_datasets.py universe_wsgi.ini -d 10 -6 -r This morning I noticed a number of workflows were either stuck at a certain step (ie..job running) or the step was grey (waiting in queue) but our cluster has free nodes. If I start a new workflow...it completes fine...just the 19 histories that were running yesterday are stuck. Did I do something wrong with the cleanup. Is there a way to restart these stuck histories without having to restart the entire workflow? Thanks in advance, Liisa

2 1

change queue with sge
by Sylvain Thomas 06 May '11

06 May '11

Hi I currently use a galaxy server with cluster. This cluster uses SGE. I'd like to specify a queue other than the default. I have tried many combinaisons with drmaa:/// without success. The queue used is always the default one. Does anyone has solved this problem? Thanks, Sylvain -- Sylvain Thomas tel : +33 (0)5 61 28 54 27 INRA - Unité de BIA Génopôle - plateforme Bio-informatique Chemin de Borde-Rouge - AUZEVILLE BP 52627 - 31326 CASTANET-TOLOSAN CEDEX

3 3

options from file
by SHAUN WEBB 06 May '11

06 May '11

Hi, I have created a tool that will fetch sequences for selected IDs from a tabular file containing multiple IDs and additional info. I want the tool config to scan the first column of the tab file for IDs and provide the user with a selection box where they can select a single ID or multiple IDs and get output for all selected. The following method does this: <param name="tabfile" type="data" format="tabular" label="ID File"/> <param name="selection" type="select" multiple="true" accept_default="true" label="ID" > <options from_dataset="tabfile"> <column name="name" index="0"/> <column name="value" index="0"/> </options> </param> The issue is, if the top file in my history is a SAM file containing ~30,000 IDs in the first column the tool initially attempts to load these all in to the selection box and effectively crashes my local instance. I only want to use this on tab files that ill have ~100 IDs at most. I have got around this by creating a new datatype indexfile as a class of Tabular in tabular.py: class IndexFile( Tabular ): file_ext = 'indexfile' def sniff( self, filename ): return False And changing the input file to: <param name="tabfile" type="data" format="tabular" label="ID File"/> This means I must first set the tabular file to type indexfile, then it will be the only dataset shown under tabfile. Selecting options from a file is really useful, I was wondering if there is a better workaround for this or if a similar indexfile datatype could be included in Galaxy. Thanks Shaun -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 1

Uploading Binary Files into Galaxy
by Darren.Cullerne＠csiro.au 06 May '11

06 May '11

Hi Everyone, I have encountered a problem and I don't know how to fix it. We have built a Galaxy instance on a couple of machines here and have built the XML files to use a couple of tools in Galaxy. The input of one of these is a Suffix Array (a binary file). Unfortunately, when I try and upload the Suffix Array, I get the following error message: The uploaded binary file contains inappropriate content My gut feeling is that it is related to Galaxy trying to preview the file and not being able to determine the type. Is there any way around this? I can provide the file and the tool if you need to investigate it further. Thanks, Darren Cullerne Computational Biology CSIRO Plant Industry x5042 +61 2 62465042 darren.cullerne(a)csiro.au<mailto:darren.cullerne@csiro.au>

3 2

Fresh galaxy installation tips/requests
by Assaf Gordon 05 May '11

05 May '11

Hi all, It's been a long while since I had to install a fresh production galaxy server, and I can offer some tips or requests for minor improvements (but to give credit were it's due - installation is much smoother now, and DRMAA works great with SGE out of the box). These are just minor annoyances, but they can help make new installations easier. --- 1. Debugging REMOTE_USER issues. The mix of apache + authentication + non-root URL + load balancing is a killer. googling "mod_rewrite + mod_auth" shows that I'm not the only one having problems with it... Adding those two lines in "./lib/galaxy/web/framework/middleware/remoteuser.py", line 79 is a wonderful debugging tool, which enables you to see what Apache is passing on to Galaxy: for k,v in environ.items(): sys.stderr.write ( "%s:\t%s\n" % ( k, v ) ) I needed to print all environment variables (not just HTTP_REMOTE_USER) so I could debug mod_rewrite's variables setup. If there was an easy way to enable this debug from the INI file it would be a great help. --- 2. The file name "universe_wsgi.ini" is hard-coded in several places. Most of them are python scripts (like cleanup, ampq, etc.), but one is "./galaxy/eggs/init.py". Not a real problem, except that in the "WebApplicationScaling" wiki page it's kind of implied that "universe_wsgi.ini" is not used any more after creating the two separated ini files (universe_wsgi.runner.ini and universe_wsgi.webapp.ini). If you rename "universe_wsgi.ini" instead of copying it, galaxy will simply not start, even if given a different INI file name. So the best recommendation (in the documents) would probably be to maintain three identical INI files, except for the "[server:main]" part of each (and the "enable_job_running = False" in the runner). Most importantly, make sure that the database connection string is identical in all of them. As a side node, the "cp" command in the "WebApplicationScaling" wiki page is incorrect (you can't copy one file into two files). It'll also be good to mention that the cleanup scripts will always use "universe_wsgi.ini" regardless of what the galaxy server is using. --- 3. Saving SGE/PBS/DRMAA shell scripts if "debug=TRUE" in the jobrunner's INI file - this one is a gem, too bad I discovered it too late. If only it was documented :) --- 4. INI keys that accept multiple comma separated values (e.g. "admin_users") are sensitive to white space. Example: admin_users = gordon@cshl.edu,foobar@cshl.edu works fine, but: admin_users = gordon(a)cshl.edu , foobar(a)cshl.edu will not work (spaces around the comma). Same thing with "ucsc_display_sites": ucsc_display_sites = cshl,main will work (and display both "cshl" and "main"), but: ucsc_display_sites = cshl, main will display only "cshl" (because "main" is " main" with a space). It's probably easier to document it than to fix it, but either will do. --- 5. There's no "ensembl_display_sites" in the INI, so disabling the "display at ensembl" link (for BAM files, etc.) requires commenting out the corresponding "<display>" tag in "datatypes_conf.xml", whereas for other servers (gbrowse,ucsc) it's enough to comment out the "ucsc_display_sites" in the INI. --- 6. For galaxy servers running inside an intranet without access to the main/test UCSC Genome browser server, and want to use a local mirror (this might be an obscure setup but that's what I have): disabling both "main" and "test" in "ucsc_display_sites" will disable the authenticated user bypass, because "remouteuser.py" have them hard-coded before setting "self.allow_ucsc_main = True". I needed to patch and add "cshl" as a third option - this is kind of ugly. --- Two development issues not related to production server installation: 1. Debugging complex cheetah templates is very hard. If you have good tricks (beyond #slient and #breakpoint) - examples would be appriciated. If there's a way to see the compiled cheetah code which produced the error - it would be great. One problem is that the error is always reported without a line number (I guess because it's first converted into a string) and without the offending tag. 2. I only half-understand the new "tool_data_table_conf.xml" mechanism - is it explained somewhere ? Looks quite complicated, especially adding in-lined lamba for reading the "__app__" global variable inside the cheetah code of each tool :( --- And user interface quirks: 1. when managing workflows, the "configure your workflow menu" page is a bit uninformative: there's no title, header, instruction or anything else, and the button says "Submit Query" which doesn't mean much ("Update Workflow Menu" would be better). If you click on "configure your workflow menu" and you don't have any workflows, the page is down right mysterious . 2. JQuery Tipsy balloons - please make them go away. Or, make the appearance delay changeable in the INI file (to something like 600 seconds :) ). --- As always, grumbling aside, galaxy is GREAT! so keep up the good work. thanks, -gordon

2 2

Using a Galaxy image on a local system
by Clare Sloggett 05 May '11

05 May '11

Hi all, I would like to set up Galaxy locally. At the moment, I'm just trying to use it on my desktop (a Mac, OSX 10.6.7), but later we will want a local server to play with. Rather than install galaxy and then install all the tools it can use (and deal with OSX issues for some of them), it seems simpler just to use a virtual machine, since there are images which get regularly updated and come with pretty much everything. Is there anything wrong with this approach? I know there are Amazon EC2 images for Galaxy. So far as I know there are not other kinds of images? So for using it on my desktop, I think my options are either to run an EC2-compatible system locally, or to try to convert the AMI to a VMWare or VirtualBox image. I was just wondering if anyone has already tried either of these approaches? Also, is it possible to get hold of the galaxy AMI files themselves? Any advice welcome! Thanks, Clare

3 4