November 2009 - galaxy-user - lists.galaxyproject.org

setting environment for sge scripts
by Andreas Kuntzagk 16 Apr '10

16 Apr '10

Hi, I followed the advice for multiple installed python versions to set ~galaxy/galaxy-python/python and set the environment for the galaxy user to have that in the PATH Problem is, when a job is started on GridEngine, it does not use the "-V" flag to inherit the environment. So it runs with a different python version and more important it does not set LD_LIBRARY_PATH and fails for some tools which need some special lib. (libRblas in this case) Where can I fix that? regards, Andreas

4 6

Suggestion on functionality for published workflows
by Timothy Hughes 13 Jan '10

13 Jan '10

I have heard from jgoecks <http://bitbucket.org/jgoecks/> that the galaxy team are working on a repository for all galaxy objects. I would like to suggest that all workflows that are shared publically (published) have a mandatory field that briefly describes their functionality and that the repository have search functionality which enables a search of published workflows using this field. I am sure their are loads of great workflows out there which are either immediately usable or are easily adaptable. Getting access to such workflows would also be an excellent way to learn Galaxy in a complementary way to the screencasts. While we are waiting for such functionality, perhaps Galaxy developers could set up a simple page where the public histories / workflows can be posted. The simplest solution might be just a check box on the sharing page ("post link to publicly accessible page"), alternatively just a piece of text on the sharing page encouraging users to manually post on specific forum or mailing list. Tim. -- Tim Hughes PhD (http://digitised.info) Medical Genetics Department Oslo University Hospital Ullevål Kirkeveien 166 0407 Oslo Norway

3 2

purging deleted datasets
by Erick Antezana 18 Dec '09

18 Dec '09

Hi, how can I purge my deleted datasets that were imported so that I could clean a bit my disk? I saw they are under database/files/000/* thanks, Erick

3 6

Command line execution
by Jordan Mendler 11 Dec '09

11 Dec '09

Hi all, I was wondering, is there a way to use Galaxy from the Command Line, or is it all web/point-and-click based? I would like to be able to write scripts to run workflows on new experiments that come off the sequencer, from the command line on Cron, so wondering if anything like this is possible with Galaxy. Thanks, Jordan

3 2

galaxy behind proxy
by Gunnar Raetsch 02 Dec '09

02 Dec '09

Hi everybody, I have a problem with some proxy settings for our galaxy instance. Our admins want to have the server protected better and put it behind a firewall/proxy. That all seems to work after switching on the firewall, but not the external data sources (like biomart or UCSC browser). I did not find an configuration option for galaxy, so I just use the environment variable http_proxy. If I try biomart without setting http_proxy, then I get this error message: The remote data source application may be off line, please try again later. Error: [Errno socket error] (111, 'Connection refused') if I set http_proxy to dmzproxy.tuebingen.mpg.de:8080, then I get this error: The remote data source application may be off line, please try again later. Error: [Errno url error] invalid proxy for http: 'dmzproxy.tuebingen.mpg.de:8080' These errors appear after finishing with biomart and it tries to send the data back to the galaxy instance. What is the right way to get these tools to work behind a firewall/ proxy? Thanks a lot for your help in advance, Gunnar +-------------------------------------------------------------------+ Gunnar Rätsch http://www.fml.mpg.de/raetsch Friedrich Miescher Laboratory Gunnar.Raetsch(a)tuebingen.mpg.de Max Planck Society Tel: (+49) 7071 601 820 Spemannstraße 39, 72076 Tübingen, Germany Fax: (+49) 7071 601 801

5 4

Request for showing output plots in galaxy frame
by Matthew Iyer 25 Nov '09

25 Nov '09

Dear Galaxy team, My name is Matthew Iyer and I am a Bioinformatics PhD student at the University of Michigan. I am doing my thesis in Arul Chinnaiyan's cancer research lab and generally work with NGS data. I am interested in incorporating galaxy for displaying customized bar/scatter plots showing gene expression levels for many samples. I have downloaded and am successfully using a galaxy locally and I really love it. However, all of the graph display/plotting tools create images (PDF) for download. Of course, the option to download the images is important, but is it also possible to display these images in the main window? This feature would make a crucial difference for me and would create many users of galaxy in our lab. I am a python programmer myself and would be interested in contributing to this effort rather than developing this in a separate framework. I am sure that it is a simple modification to show an image in the main window. I look forward to working with you and using galaxy! Best, Matthew

3 2

joining
by Susan Ettinger 21 Nov '09

21 Nov '09

1 0

Questions about Galaxy!
by WangJun 17 Nov '09

17 Nov '09

Hi, This is a question about Galaxy interaction with UCSC data. I hava a dataset platform, and I want to add a button like the UCSC's "Send query to Galaxy". I find that it's actually a form: <form method="POST" action="http://main.g2.bx.psu.edu/tool_runner"> <input type="HIDDEN" value="ucsc_table_direct1" name="tool_id"/> <input type="HIDDEN" value="http://genome.ucsc.edu/cgi-bin/hgTables" name="URL"/> <input type="HIDDEN" value="296389734" name="hguid"/> <input type="HIDDEN" value="hg18" name="db"/> <input type="HIDDEN" value="Human" name="org"/> <input type="HIDDEN" value="multiz28way" name="hgta_table"/> <input type="HIDDEN" value="multiz28way" name="hgta_track"/> <input type="HIDDEN" value="range" name="hgta_regionType"/> <input type="HIDDEN" value="primaryTable" name="hgta_outputType"/> <input type="HIDDEN" value="chr3:130734948-130736581" name="position"/> <input type="HIDDEN" value="get output" name="hgta_doTopSubmit"/> <input type="HIDDEN" value="146683120" name="hgsid"/> <input type="SUBMIT" value="Send query to Galaxy" name="hgta_doGalaxyQuery"/> </form> I have a url and three params, and I tried. But I can't see my data in the right-side green box. And what's the tool_id param used for? Thanks in advance! Best, Kathleen 2009-11-16

2 1

Suggestion on functionality for published workflows
by Timothy Hughes 12 Nov '09

12 Nov '09

I have never used myExperiment but it sounds like a good site to use for posting published workflows. The tricky bit is to get people to post their workflows there. Would Galaxy be willing to suggest that users publish workflows there? I think galaxy is great, but what it lacks at the moment is exactly something like this myExperiment.org which can provide the "social" elements: direct interaction between users, posting of workflows, collaboration, etc. I gather Galaxy is starting work in this direction, but perhaps it would be better to see to what extent galaxy could integrate with an existing solution. Tim. Message: 4 Date: Wed, 11 Nov 2009 09:25:23 +0100 From: Jelle Scholtalbers <j.scholtalbers(a)gmail.com> To: galaxy-user(a)bx.psu.edu Subject: Re: [galaxy-user] Suggestion on functionality for published workflows Message-ID: <b66932c30911110025t45b73d6amd777817dffe3d025(a)mail.gmail.com > Content-Type: text/plain; charset="iso-8859-1" Hi, would it be an idea to post galaxy objects on myExperiment.org? It seems like a good place to share these types of workflows and histories. Although now mostly used by Taverna users, any kind of (scientific) digital object can be placed there. Cheers, Jelle 2009/11/10 Timothy Hughes <timothy.hughes(a)medisin.uio.no> > I have heard from jgoecks <http://bitbucket.org/jgoecks/> that the galaxy > team are working on a repository for all galaxy objects. I would like to > suggest that all workflows that are shared publically (published) have a > mandatory field that briefly describes their functionality and that the > repository have search functionality which enables a search of published > workflows using this field. I am sure their are loads of great workflows out > there which are either immediately usable or are easily adaptable. Getting > access to such workflows would also be an excellent way to learn Galaxy in a > complementary way to the screencasts. > > While we are waiting for such functionality, perhaps Galaxy developers > could set up a simple page where the public histories / workflows can be > posted. The simplest solution might be just a check box on the sharing page > ("post link to publicly accessible page"), alternatively just a piece of > text on the sharing page encouraging users to manually post on specific > forum or mailing list. > > Tim. > > -- > Tim Hughes PhD (http://digitised.info) > Medical Genetics Department > Oslo University Hospital Ullev?l > Kirkeveien 166 > 0407 Oslo > Norway > > On Wed, Nov 11, 2009 at 7:15 PM, <galaxy-user-request(a)lists.bx.psu.edu>wrote: > Send galaxy-user mailing list submissions to > galaxy-user(a)lists.bx.psu.edu > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.bx.psu.edu/listinfo/galaxy-user > or, via email, send a message with subject or body 'help' to > galaxy-user-request(a)lists.bx.psu.edu > > You can reach the person managing the list at > galaxy-user-owner(a)lists.bx.psu.edu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of galaxy-user digest..." > > > Today's Topics: > > 1. Re: setting environment for sge scripts (Assaf Gordon) > 2. Re: galaxy passing an incorrect directory to a script... > (Nate Coraor) > 3. Suggestion on functionality for published workflows > (Timothy Hughes) > 4. Re: Suggestion on functionality for published workflows > (Jelle Scholtalbers) > 5. Getting sequences for multiple species from UCSC (Farhat Habib) > 6. Re: Getting sequences for multiple species from UCSC > (Anton Nekrutenko) > 7. Re: Collecting User Information on the Registration page > (Greg Von Kuster) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 20 Oct 2009 12:18:44 -0400 > From: Assaf Gordon <gordon(a)cshl.edu> > To: Nate Coraor <nate(a)bx.psu.edu> > Cc: Andreas Kuntzagk <andreas.kuntzagk(a)mdc-berlin.de>, > galaxy-user(a)bx.psu.edu > Subject: Re: [galaxy-user] setting environment for sge scripts > Message-ID: <4ADDE2E4.3000209(a)cshl.edu> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > SGE supports per-user defaults file called "~/.sge_request" , > which can be used to set job's environment variables (and other qsub > parameters). > > I think it was mentioned in the galaxy-dev list (I don't remember by whom). > > Example: > ============ > $ cat ~/.sge_request > > -v PERL5LIB=/data/hannon/gordon/perl_modules/lib64/perl5 > > -v DUMMY=HELLO > ============ > > Will add two environment variables (PERL5LIB and DUMMY) to each qsub job. > I guess this way you can also change the PATH. > > see "man sge_request" or "man ge_request" (in some places it is mentioned > as "ge_request" instead of "sge_request"). > > "man qsub" tells which options take precedence ( command-line VS script VS > user-sge_request VS gloval-sge_request ). > > Tested with SGE 6.1 and 6.2u3. > > -gordon > > Nate Coraor wrote, On 10/20/2009 11:51 AM: > > Andreas Kuntzagk wrote: > > > >> It is set in ~/.bashrc, which is source by ~/.bash_profile. If I > >> normally login to a cluster node via ssh it is set correctly. But if I > >> qsub a job, it is not read. Except when I set the "-V" switch. So I > >> either need to change gridengines behaviour or make galaxy use that > switch. > >> > >> Is there a easy way to config how to submit cluster jobs? > > > > Andreas, > > > > SGE must not be running the shell as a login or interactive shell, which > > PBS does. You may want to check the 'INVOCATION' section of the bash(1) > > man page to see how you can force this. > > > > If you can't change how your shell is run by SGE, you could add what you > > need to the SGE job template in: > > > > galaxy_dist/lib/galaxy/jobs/runners/sge.py > > > > but this would be an ugly hack. > > > > --nate > > _______________________________________________ > > galaxy-user mailing list > > galaxy-user(a)bx.psu.edu > > http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user > > > > ------------------------------ > > Message: 2 > Date: Tue, 20 Oct 2009 13:05:53 -0400 > From: Nate Coraor <nate(a)bx.psu.edu> > To: Nick Schurch <N.Schurch(a)dundee.ac.uk> > Cc: galaxy-user(a)bx.psu.edu > Subject: Re: [galaxy-user] galaxy passing an incorrect directory to a > script... > Message-ID: <4ADDEDF1.8010608(a)bx.psu.edu> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Nick Schurch wrote: > > > Does anyone have any idea why galaxy would pass a 'working directory' > > to the script rather than the directory where the files are actually > > stored? Is it that galaxy actually creates the files in the working > > directory and then moves them? > > Hi Nick, > > This was originally written for PBS file staging - since the > database/files/ directory was not mounted on cluster nodes, a path > common to the Galaxy application server and the node's tool execution > environment had to be used (the job working directory, which we put in > NFS). > > The current method to use this functionality is such: > > If reading from an input's extra files, use $input.extra_files_path. > The extra_files_path attribute always points to the location in > database/files/. > > If writing to an output's extra files, use $output.files_path. The > files_path attribute points to a directory in the job's working > directory. The framework will automatically relocate these extra files > to the real location in database/files/ as part of the job finish method. > > This functionality was never well documented and is rather confusing, in > the future we plan to simplify it. A ticket exists for this: > > > http://bitbucket.org/galaxy/galaxy-central/issue/206/eliminate-the-use-of-f… > > > I'm also really confused as to why the 'development' installation > > works fine, but the 'live' installation throws this error. Any insight > > would be greatly appreciated. > > This shouldn't really be the case, it might help to see your tool's xml > and the code that generates the html, to understand how you're using the > extra files directory. Note that links to files in the extra files > directory should all be relative. > > --nate > > > > ------------------------------ > > Message: 3 > Date: Tue, 10 Nov 2009 23:36:50 +0100 > From: Timothy Hughes <timothy.hughes(a)medisin.uio.no> > To: galaxy-user(a)bx.psu.edu > Subject: [galaxy-user] Suggestion on functionality for published > workflows > Message-ID: > <3508097b0911101436g39243390if960dae24a6b3df0(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > I have heard from jgoecks <http://bitbucket.org/jgoecks/> that the galaxy > team are working on a repository for all galaxy objects. I would like to > suggest that all workflows that are shared publically (published) have a > mandatory field that briefly describes their functionality and that the > repository have search functionality which enables a search of published > workflows using this field. I am sure their are loads of great workflows > out > there which are either immediately usable or are easily adaptable. Getting > access to such workflows would also be an excellent way to learn Galaxy in > a > complementary way to the screencasts. > > While we are waiting for such functionality, perhaps Galaxy developers > could > set up a simple page where the public histories / workflows can be posted. > The simplest solution might be just a check box on the sharing page ("post > link to publicly accessible page"), alternatively just a piece of text on > the sharing page encouraging users to manually post on specific forum or > mailing list. > > Tim. > > -- > Tim Hughes PhD (http://digitised.info) > Medical Genetics Department > Oslo University Hospital Ullev?l > Kirkeveien 166 > 0407 Oslo > Norway >

2 1

Collecting User Information on the Registration page
by lentaing＠jimmy.harvard.edu 12 Nov '09

12 Nov '09

Hi, We are interested in collecting more information about the user, e.g. which lab they're part of etc, as they register to use our local mirror of galaxy. Is there an easy way to add these fields to the registration page; is it also easy to store these values to the (galaxy) database? Thanks! Len

3 2

Getting sequences for multiple species from UCSC
by Farhat Habib 12 Nov '09

12 Nov '09

Hi all, I am wondering if there is a way to download sequence information from UCSC into Galaxy for multiple species. E.g, looking at a region of the genome for Drosophila melanogaster, you can also see the level of conservation across other insect species. Is it possible to get the sequences from all species that align to that region of the genome in Drosophila. Thanks. -Farhat

2 1

Search for TF binding site patterns in Galaxy
by Brown, Stuart 06 Nov '09

06 Nov '09

I am trying to come up with a nice workflow/tutorial for the use of Galaxy to search for Transcription Factor binding sites on a genome wide scale using pattern search tools. I want to train my students to think genomically and to use clever tools to leverage their abilities. Galaxy is absolutely awesome for grabbing the upstream promoter regions for all genes from any organism with a whole genome in UCSC. It is also possible to use the integrated EMBOSS tools such as fuzznuc and dreg to search for a known TFBS (or any other simple nucleotide pattern). However, I can't get past the simple search into a more clever infomation-based search. In particular I have the following workflow in mind: 1. Collect upstream regions for all mouse (or human) genes 2. Search for a published TF binding site with a single base mismatch using FUZZNUC 3. Make a multiple alignment of the sequences returned by FUZZNUC (not possible in any way that I have been able to find) 4. Make a logo from the alignment to identify informative positions and conserved substitutions (not in Galaxy) 5. Make a PSSM profile, HMM profile, or other smart searching tool from the aligned sequences (not in Galaxy) 6. Search the upstream regions again with this more sensitive pattern search method. (not in Galaxy). 7. Make a list of genes targeted with this TFBS, 8. Compare list of genes to microarray data showing co-regulation of this gene set, or to pathways I am frustrated at step 3. Even if I bring the FUZZNUC results to my desktop, there is no easy way to extract just sequences and make a multiple alignment. Many of the 'allowed' Fuzznuc optional output formats produce an error, or no useable output. Thanks for any suggestions. Stuart M. Brown, Ph.D. Associate Professor Center for Health Informatics and Bioinformatics NYU School of Medicine 550 First Ave, NY, NY 10016 stuart.brown(a)med.nyu.edu (212)263-7689 FAX (212) 263-8139 ------------------------------------------------------------ This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is proprietary, confidential, and exempt from disclosure under applicable law. Any unauthorized review, use, disclosure, or distribution is prohibited. If you have received this email in error please notify the sender by return email and delete the original message. Please note, the recipient should check this email and any attachments for the presence of viruses. The organization accepts no liability for any damage caused by any virus transmitted by this email. =================================

4 3

Using mysql instead of sqlite
by Erick Antezana 05 Nov '09

05 Nov '09

Hello, I would like to use MySQL instead of sqlite to store my data. I coudn't find on the Galaxy web site a HOWTO or some guidelines to do it. I only found some lines that might need to be changed/enabled in the universe_wsgi.ini file: #database_file = database/universe.sqlite database_connection = mysql:///galaxy #database_engine_option_echo = true #database_engine_option_echo_pool = true #database_engine_option_pool_size = 10 #database_engine_option_max_overflow = 20 Could you point out to some doc or briefly describe what I need to do in order to go for mysql? Are there any plans to support other DBMS's (like Oracle for instance)? thanks, Erick

5 15

Default global variables in Galaxy
by gerrit 04 Nov '09

04 Nov '09

Hi there, Can anyone tell where I can find a description of the default global variables in Galaxy. From the tool examples I can gather that GALAXY_DATA_INDEX_DIR is one but can not find documentation of any other. Thanks, Gerrit

2 1

Experience with Loading NGS data on standalone instance of galaxy
by Abhishek Pratap 04 Nov '09

04 Nov '09

Hi All I recently came to know about NGS analysis on galaxy during ISMB. Getting excited I tried couple of things basically to play with it. Few comments : I may have interepretted something described below in a wrong way. My apologies before hand. On a standalone installation of galaxy while I was trying to explore one FASTQ(sequence) file. It takes considerable (> 20 min) for a fastq file to get uploaded (2 GB). I am not sure what is the rationale behind that. Ideally I think there should be no need to upload such heavy files into the workspace. They could actually be used straight away by the path specified. Also is there any way to access the scripts for analysis on the command line. I know this undermines the main aim of working with galaxy but rite now I am concerned about the performance/time. I will be happy to discuss more about this in case you have some comments/questions for me. Best, -Abhi ----------------------------- Abhishek Pratap Bioinformatics Software Engineer Institute for Genome Sciences School of Medicine, Univ of Maryland 801, W. Baltimore Street, Baltimore, MD 21209 Ph: (+1)-410-706-2296 www.igs.umaryland.edu/

8 26