October 2011 - galaxy-dev - lists.galaxyproject.org

Integrating a new display tool with bigwig format
by Jeremy Villalobos 31 Oct '11

31 Oct '11

Hello: I am trying to add a display tool to a local galaxy instance. However, I don't understand the scripts used. In the tutorial at: http://wiki.g2.bx.psu.edu/Admin/Tools/External%20Display%20Applications%20T… The script for "Basic Topics" uses variables that are not traceable. Is there a reference for the "global" variables for galaxy?. I imagine this is like the "global" variables one uses when creating html pages with javascript. Where can I find the log files for galaxy ?. If the scripts don't work, I imagine information about a possible errors must be printed in some log somewhere. Do I need to restart the portal after I perform changes to the display_applications directory and datatypes_conf.xml ? Thanks for any help Jeremy Villalobos

1 0

Re: [galaxy-dev] Importing existing data into Galaxy
by Greg Von Kuster 28 Oct '11

28 Oct '11

The files are not touched - metadata is generated for the files, but the content of the files themselves is not changed. On Oct 28, 2011, at 3:21 PM, Oren Livne wrote: > Dear Greg, > Great, thanks! Just one more question: if I uncheck the "copy data files" option, does Galaxy change the original day files in any way when it loads them into a data library (e.g., prepend them with some metadata header), or does it leave them intact and stores the metadata separately? > Thanks, > Oren > > On 10/28/2011 1:44 PM, Greg Von Kuster wrote: >> Hello Oren, >> >> We use a sort of informal approach to implementing new Galaxy API features (they're implemented when the need arises), so unfortunately it doesn't look like the features you are looking for exist yet. If you'd like to contribute them, the community would certainly appreciate it! >> >> Greg Von Kuster >> >> On Oct 28, 2011, at 12:45 PM, Oren Livne wrote: >> >>> Dear Greg, >>> >>> Thank you so much, this is exactly what I need. I will use the API access (scripts/api/*.py) to create the new data library. I have two more questions: >>> - Is there an API function for deleting a library? >>> - Is there an API function to make a library accessible only to the user whose API key is passed in? >>> >>> Oren >>> >>> On 10/27/2011 7:55 PM, Greg Von Kuster wrote: >>>> Oren, >>>> >>>> The best way to do this would be to use galaxy data libraries, uploading files from file system paths and not copying the files into the Galaxy file location. >>>> >>>> The following provides all of the details about data libraries: >>>> >>>> http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries >>>> >>>> >>>> The following provides specific details about the various options for uploading files to data libraries. >>>> >>>> http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files >>>> >>>> Let us know if you encounter any issues. >>>> >>>> Greg Von Kuster >>>> >>>> >>>> On Oct 27, 2011, at 4:31 PM, Oren Livne wrote: >>>> >>>>> Dear All, >>>>> >>>>> We have a large collection of large data files on our shared file system. We would like to make a subset of them available to a galaxy user session (based on user privileges; different subsets for different users). We want to leave files in their original locations and point galaxy to their paths without copying them. What is the best option for implementing this bridge? Remote data source/data library uploaded on the fly/other? >>>>> >>>>> Thanks, >>>>> Oren >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> >>>>> http://lists.bx.psu.edu/ >>>> Greg Von Kuster >>>> Galaxy Development Team >>>> greg(a)bx.psu.edu >>>> >>>> >>>> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Re: [galaxy-dev] Importing existing data into Galaxy
by Greg Von Kuster 28 Oct '11

28 Oct '11

Hello Oren, We use a sort of informal approach to implementing new Galaxy API features (they're implemented when the need arises), so unfortunately it doesn't look like the features you are looking for exist yet. If you'd like to contribute them, the community would certainly appreciate it! Greg Von Kuster On Oct 28, 2011, at 12:45 PM, Oren Livne wrote: > Dear Greg, > > Thank you so much, this is exactly what I need. I will use the API access (scripts/api/*.py) to create the new data library. I have two more questions: > - Is there an API function for deleting a library? > - Is there an API function to make a library accessible only to the user whose API key is passed in? > > Oren > > On 10/27/2011 7:55 PM, Greg Von Kuster wrote: >> Oren, >> >> The best way to do this would be to use galaxy data libraries, uploading files from file system paths and not copying the files into the Galaxy file location. >> >> The following provides all of the details about data libraries: >> >> http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries >> >> >> The following provides specific details about the various options for uploading files to data libraries. >> >> http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files >> >> Let us know if you encounter any issues. >> >> Greg Von Kuster >> >> >> On Oct 27, 2011, at 4:31 PM, Oren Livne wrote: >> >>> Dear All, >>> >>> We have a large collection of large data files on our shared file system. We would like to make a subset of them available to a galaxy user session (based on user privileges; different subsets for different users). We want to leave files in their original locations and point galaxy to their paths without copying them. What is the best option for implementing this bridge? Remote data source/data library uploaded on the fly/other? >>> >>> Thanks, >>> Oren >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Some questions about cloudman
by Cittaro Davide 28 Oct '11

28 Oct '11

Hi there, I'm in the middle of a decision: should I go into the cloud or not? I'm reading the docs on galaxy wiki, and I see that besides EC2, EBS I need S3 storage. What is that for (meaning: why galaxy needs S3)? @people already using it: how to you send NGS data? I need some feedback! :-) Thanks d /* Davide Cittaro, PhD Head of Bioinformatics Core Center for Translational Genomics and Bioinformatics San Raffaele Scientific Institute Via Olgettina 58 20132 Milano Italy Office: +39 02 26439140 Mail: cittaro.davide(a)hsr.it<mailto:cittaro.davide@hsr.it> Skype: daweonline */

3 3

Re: [galaxy-dev] Importing Historys in other Galaxy instances
by Stefanie Ververs 28 Oct '11

28 Oct '11

Am 24.10.2011 21:05, schrieb Jeremy Goecks: >> So - I'm logged in and the link is accessible. >> (I tried with different accounts on the public server - that works; but not importing to our own instance.) > The history must be "Accessible via link": Options --> Share/Publish --> Make History Accessible via Link > > Can you share the history with me? > > Thanks, > J. I tried it, following your instructions. And different follow-ups: 1) Pasting the "sharing"-Link into the "Import a History from an Archive" field -> doesn't work 2) Opening the link being logged in to the account on OUR instance, that means, the target account, but NOT on the source account. -> doesn't work - he tries to import the history on the public server (I assume because the message was about "this will delete your current history") When I imported histories from one account to another on the public server, I never used "Make History Accessible via Link". I just took the export-Link (the one you can download the history as an archive file.) I even tried uploading the file somewhere else. This is my test-History: http://main.g2.bx.psu.edu/u/stef85/h/neue-history-zum-test Steffi

3 7

Importing existing data into Galaxy
by Oren Livne 28 Oct '11

28 Oct '11

Dear All, We have a large collection of large data files on our shared file system. We would like to make a subset of them available to a galaxy user session (based on user privileges; different subsets for different users). We want to leave files in their original locations and point galaxy to their paths without copying them. What is the best option for implementing this bridge? Remote data source/data library uploaded on the fly/other? Thanks, Oren

2 1

Re: [galaxy-dev] Modifying the GATK wrappers to pick genome infomration from bam/sam dbkey attribute
by Chorny, Ilya 27 Oct '11

27 Oct '11

Hi Dan, I want to modify the wrappers to get genome information from the bam/sam attribute. I did this for the tophat/cufflinks wrappers. I add another option to select genome called attribute and then I pass $bam.metadata.dkey in the command line (see example below). Is this something you might be interested in? Can you suggest an alternative way to do this? My only concern is my inability to validate that the $input.metadata.dbkey exists. Thanks, Ilya See example below. #if $reference_annotation.use_ref == "Use reference annotation": #if $reference_annotation.annotationSource.reference_annotation_file == "indexed": -G "${ filter( lambda x: str( x[0] ) == str( $reference_annotation.annotationSource.indices ), $__app__.tool_data_tables[ 'gtf_index' ].get_fields() )[0][-1] }" #else: #if $reference_annotation.annotationSource.reference_annotation_file == "attribute": -G "${ filter( lambda x: str( x[0] ) == str( $input.metadata.dbkey ), $__app__.tool_data_tables[ 'gtf_index' ].get_fields() )[0][-1] }" #else: -G "${reference_annotation.annotationSource.ownFile}" #end if #end if #end if <conditional name="annotationSource"> <param name="reference_annotation_file" type="select" label="Please select a reference Aonnotation"> <option value="indexed">Use a built-in index</option> <option value="history">Use one from the history</option> <option value="attribute">Use input bam metadata.dbkey attribute</option> </param> <when value="indexed"> <param name="indices" type="select" label="Select genome for gtf annotation"> <options from_data_table="gtf_index"> <filter type="sort_by" column="3" /> <validator type="no_options" message="No indexes are available for the selected input dataset" /> </options> </param> </when> <when value="history"> <param name="ownFile" type="data" format="gff3, gtf" label="Select a reference annotation file" /> </when> </conditional> Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

2 2

Re: [galaxy-dev] [galaxy-user] Problem with bam and/or bai files
by Peter Cock 27 Oct '11

27 Oct '11

Sending to galaxy-dev ... On Thu, Oct 27, 2011 at 5:51 AM, Jim Robinson <jrobinso(a)broadinstitute.org> wrote: > > Hi Mike, > > Someone from the Galaxy team can perhaps give some insight on > what went wrong, I can comment on the error message from IGV. > That error is thrown from Picard, in every case I've investigated so > far it was traced to a problem with the index. Useful background re: "Error reading bam file. This usually indicates a problem with the index (bai) file. ArrayIndexOutofBoundsException: 4682 (4682)." > The most common causes are (1) a problem with the sequence > dictionary in the BAM header itself, specifically incorrect sequence > lengths, Any idea what tools produce that kind of thing? > and (2) indexing an un-sorted BAM. Apparently samtools will > make invalid indexes from such files without any complaints in > both cases. You can even use samtools tview on such files, > it happily will show you some random region when you query. That is news to me - I recall "samtools index" being recommended as a way to determine if a BAM files was sorted or not (error on unsorted, you get an index if it was sorted) and again from memory this is what Galaxy uses internally as part of preparing BAM files on upload. Might this be tied to a specific version of samtools? e.g. a possible regression? > I don't see a "Sort" step in your workflow, maybe that's the problem? > > Please CC me on any reply, I might miss it in the list. > > Jim Thanks, Peter

3 2

Server Error when trying to upload repository.
by David Murphy 27 Oct '11

27 Oct '11

Hi I've been trying to create a repository for a small tool called CodonLogo at http://toolshed.g2.bx.psu.edu/. The problem is that the first zip I uploaded contained a few backup copies of some text files ending with a "~". It now won't allow me to delete them by checking them and clicking "delete selected files" it then simply gives the error "The selected files were deleted from the repository. Change set revision '7ff079013d98' includes no tools or exported workflows for which metadata can be set." but the selected files are not deleted. When I try to upload a new zip without the older files it gives an error "Server Error An error occurred. See the error logs for more information. (Turn debug on to display exception reports here) " Am I doing something foolish or is this a bug? Cheers David

3 3

Local Galaxy Instance MarkupSafe error
by Jerico Nico De Leon Revote 27 Oct '11

27 Oct '11

Hi, I'm just doing a simple get->data from UCSC on our local Galaxy instance and got the following error: WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched The job box then is displayed as red on the history panel. The job runner states that the job finished normally on the cluster. Galaxy is checkout from galaxy-central (changeset: 6176:34fffbf01183). Thanks, Jerico

2 2