- galaxy-dev - lists.galaxyproject.org

Data Retention Policies
by John Chilton 24 Oct '11

24 Oct '11

Hello All, This is more of a general policy question. The head of our Research Informatics Support Systems group here at the University of Minnesota wants to know how other institutions are handling the large amount of data generated by next generation sequencing facilities and processed by local Galaxy instances. Specifically, I think she is interested in what sort of data retention policies institutions have in place - for instance how long do you store data, what sort of quotas do you put in place, what sort of backups do you have in place, etc.... Any comments would be appreciated. Thanks for your time, -John

3 2

Re: [galaxy-dev] [galaxy-user] tool.xml "&>" console redirection within command-tag
by Peter Cock 24 Oct '11

24 Oct '11

Hi Uwe, On Mon, Oct 24, 2011 at 10:47 AM, Appelt, Uwe <uwe.appelt(a)nct-heidelberg.de> wrote: > Hi again, > > yes, you're right. The problem source is something else. As soon as I specify one of the following > > <command interpreter="bash">$hisapWrapperScript &> $logFile</command> > <command interpreter="sh">$hisapWrapperScript &> $logFile</command> > <command>bash $hisapWrapperScript &> $logFile</command> > <command>sh $hisapWrapperScript &> $logFile</command> > > I get a according debug-line in the galaxy.log, reporting something like: > > galaxy.jobs.runners.local DEBUG 2011-10-24 11:26:55,360 executing: bash /home/extuser/www/galaxy/galaxy_working/database/job_working_directory/2222/tmpZmFMfg &> /home/extuser/www/galaxy/galaxy_working/database/files/005/dataset_5410.dat > > So the "&>" definitely makes it through XML, Cheetah, etc. -parsers and > finally to the cmd-line. However, neither BASH nor SH produce a log-file > as intended. You didn't try the precise combination I meant, <command>$hisapWrapperScript &> $logFile</command> where $hisapWrapperScript is marked as executable and has a Unix hashbang line saying which shell to use, e.g. #!/usr/bin/bash Apologies if I was too vague. > For some strange reason (Peter, you're right again) all log-lines appear in > the green box. To be expected, Galaxy captures any stdout (assuming nothing else captured it first) and puts it in the info field. > And even more, the green box turns of course red as soon as something is > written to stderr. Yes, it is a long standing Galaxy bug that any stderr out is treated as an error condition, rather than looking at the return code: https://bitbucket.org/galaxy/galaxy-central/issue/325/ > Aaaaaah, but the different redirection syntax works. So to conclude: > > <command interpreter="bash">$hisapWrapperScript > $logFile 2>&1</command> > > works like a charme - problem solved! Thanks Peter!!! > > Cheers, > Uwe Glad you got there in the end :) Peter By the way - this whole discussion would have been better suited to the galaxy-dev list (CC'd) since it is about developing tools for Galaxy rather than Galaxy end users.

1 0

Fill field with user email, if logged in
by Steven Platt 23 Oct '11

23 Oct '11

Hi, I a similar vein to http://gmod.827538.n3.nabble.com/Dynamic-default-values-tt3010740.html I am trying to populate a default value based on settings elsewhere, in this case the user email if they are logged in. So far I have: <conditional name="email_on_completion"> <param name="send_email" type="select" label="Send email when job is complete?"> <option value="yes">Yes</option> <option value="no">No</option> </param> <when value="no"/> <when value="yes"> <param name="address" type="text" value="$userEmail" label="Email address"/> </when> </conditional> Which views OK, except that the field contains $userEmail instead of e.g. bob(a)gmail.com Can anyone tell me how to dynamically populate a field with a users email address? (I'd like to allow non-registered users to manually enter their address if necessary, hence the text field rather than capturing the email address directly in the command string.) Thanks Steve ----------------------------------------- ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk **************************************************************************

2 2

update of cufflinks and tophat to use dbkey attribute of input files
by Chorny, Ilya 21 Oct '11

21 Oct '11

Hi Jeremy, I made some more modifications to the wrappers to allow for the tools to pull the genome information from the input files. The only drawback is that there is no way to validate whether the input has the dbkey attribute as a parameter (can you suggest a way to validate?). Attached are the diff's to the current version in galaxy-central. Best, Ilya Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

2 2

Re: [galaxy-dev] HOW TO RETRIEVE DATA FROM HISTORY??!!
by colin molter 20 Oct '11

20 Oct '11

> > > Is there a way to directly move/copy data from your galaxy history to a > given location in the filesystem of the same galaxy server? > Said differently, there is a nice way to import data from the server to > galaxy, is it possible to do the reverse? > > So far, I am obliged to download the file from galaxy to my client machine > and then back to the server!!!! with huge bam files of 3Gb it is not so > convenient!! > OK, I found a better way by (a) go to the admin panel, push the 'add a new dataset in the library' button and selecting the one needed from the current history (b) move the selected dataset from the library to a location mounted to galaxy. that is ok for me. However, if someone has a better solution, any advices are fine for me thnks colin

5 5

Multiple formats for tool options
by Alex R Bigelow 20 Oct '11

20 Oct '11

Hi, I've been tweaking the GATK wrappers to try to get my pipeline working in Galaxy; for drop-down menus, I've been able to add additional eligible formats to some parameters: <param name="input_rod" type="data" format="vcf,gatk_dbsnp,bed" label="ROD file" /> This works just fine, but it doesn't work for others: <param name="input_intervals" type="data" format="bed,gatk_interval,picard_interval_list" optional="True" label="A list of genomic intervals over which to operate" /> When I make the change, and reload the configuration (or even restart Galaxy), the menu still doesn't show any options, even though I have a bed file loaded in my history (the same bed file WILL show up in the first menu). Does anyone have an idea what might be wrong? Thanks, Alex Bigelow

2 4

Upload File Button
by Chorny, Ilya 20 Oct '11

20 Oct '11

I create an upload file button using upload.xml in one of my xml scripts. <param name="file_data" type="file" size="30" label="File" ajax-upload="true" help="TIP: Due to browser limitations, uploading files larger than 2GB is guaranteed to fail. To upload large files, use the URL method (below) or FTP (if enabled by the site administrator)."> </param> But when I add it to a work flow and try to have it set the parameters at runtime, the set at runtime options fails. Any ideas why? Thanks, Ilya Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

2 4

Questions about reusing workflows
by Lukasse, Pieter 19 Oct '11

19 Oct '11

Hi, I have some questions about adding new custom tools to my Galaxy instance: 1- How can I reuse workflows? a. Should I wrap a workflow as a tool? b. Or can I just add a custom workflow as a workflow step in another workflow? 2- I noticed that option 1.b above seems to copy the complete workflow into my new workflow, so if I have an existing workflow A with N steps and want to use this in workflow B which already has M steps, I end up with a workflow that has M+N steps while I would expect M+1 steps... This behaviour gives me the impression that if I change workflow A, this will not automatically imply a change in workflow B. Is my interpretation correct? 3- If option 1.a above is the way to go for now, what are the steps to do this? E.g. what is the command to run a workflow from the command line? Thanks and regards, Pieter Lukasse.

3 3

Blast database permissions
by Jeroen Crappé 19 Oct '11

19 Oct '11

Dear all, I have recently installed a local galaxy on our servers. I could succesfully add a blastable database via the blastdb.loc file. All is working, so no problems regarding that. My question however is, how can I set user permissions on the use of these blast databases, or is this not possible? For the moment it looks as though every user, no matter what its permisions are, can use and see the database, which is not what I want because different groups will be using this galaxy instance. Could somebody help me out on this issue? kind regards, Crappé Jeroen

3 4

Tab sniffer
by SHAUN WEBB 19 Oct '11

19 Oct '11

Hi. I am using the upload tool to enter some test data in the text box and checking the convert spaces to tabs box. However Galaxy is setting the file as txt rather than tabular, this seems to be a new bug. Shaun -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 1