September 2011 - galaxy-dev - lists.galaxyproject.org

OpenID Issues
by Greenway, Matthew [BSD] - MED 28 Sep '11

28 Sep '11

Hello Galaxy Developers: I noticed a few issues with the OpendID system. The first is that attempting to log in with an OpenID not associated with a galaxy account gives a "Server Error" page. Also using Flickr as an OP gives: "Failed to begin OpenID authentication: No usable OpenID services found for http://flickr.com" I haven't looked into whether Flickr has a different OP URL or none at all. I was wondering if someone was working on these or if they had been fixed in a different version. If not then I will work on fixing them and try to make my code available for others. Thanks, Matt Greenway This email is intended only for the use of the individual or entity to which it is addressed and may contain information that is privileged and confidential. If the reader of this email message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is prohibited. If you have received this email in error, please notify the sender and destroy/delete all copies of the transmittal. Thank you.

2 1

database_engine_option_pool_size
by Simon Taylor 28 Sep '11

28 Sep '11

Hi, What exactly are the two options database_engine_option_pool_size and database_engine_option_max_overflow in universe_wsgi.ini? Occasionally I have a problem with galaxy timing out with an exception such as: TimeoutError: QueuePool limit of size 30 overflow 40 reached, connection timed out, timeout 30 Checking the process list I don't see more than a couple of galaxy jobs running so wouldn't expect 30+ connections to the db. The timeout doesn't seem to resolve itself either and the only way to get the site back is to kill and restart the process. Are there plans to handle the exception in any way and maybe reset the db connection, or at least display a user friendly error message? Thanks, Simon | | Systems utvikler | Centre for Integrative Genetics (CIGENE) | UNIVERSITETET FOR MILJØ- OG BIOVITENSKAP

2 1

October 2011 Events
by Dave Clements 28 Sep '11

28 Sep '11

Hello all, There are (at least) three Galaxy related events<http://wiki.g2.bx.psu.edu/Events>going on next month that you may want to consider attending. See the Events <http://wiki.g2.bx.psu.edu/Events> page for a full listing of upcoming events. If you have a Galaxy related event please send it to outreach(a)galaxyproject.org and we will help you get the word out. Date Topic/Event Venue/Location Comments Contact October 9-12 *Exploit your Data: Galaxy and the GMOD Tool Suite<http://www.amata.org.au/index.php/2011-conference/workshops> * AMATA 2011 <http://www.amata.org.au/index.php/2011-conference>, Canberra, Australia This workshop will cover both Galaxy and provide an overview of the GMOD<http://gmod.org/wiki/>tool suite. Many thanks to CSIRO and AMATA for sponsoring this workshop Dave Clements <http://wiki.g2.bx.psu.edu/DaveClements> October 12-14 *October 2011 GMOD Meeting and Galaxy Workshop<http://gmod.org/wiki/October_2011_GMOD_Meeting> * Ontario Institute for Cancer Research (OICR), Toronto, Canada Dan will give a Galaxy update at the meeting, and then lead a satellite workshop on Galaxy the day following the meeting. (Or, if you already know all about Galaxy you could attend workshops on BioMart<http://gmod.org/wiki/BioMart>and InterMine <http://gmod.org/wiki/InterMine>, two data sources for Galaxy.) Register now <http://gmod.org/wiki/October_2011_GMOD_Meeting> as the meeting and associated workshop have limited space. Dan Blankenberg October 23-24 *The Galaxy Track Browser: Transforming the Genome Browser from Visualization Tool to Analysis Tool<http://www.biovis.net/papers_abstracts/papers/111.html> * BioVis 2011 <http://www.biovis.net/>, Providence, Rhode Island, United States Learn about Trackster and other biological visualization. This is part of VisWeek 2011 <http://visweek.org/>. Jeremy Goecks Thanks, Dave Clements <http://wiki.g2.bx.psu.edu/DaveClements> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

1 0

Re: [galaxy-dev] Proxy login issues
by Alex R Bigelow 28 Sep '11

28 Sep '11

Fixed: just for fun I tried using Safari instead of Chrome, and it worked fine - this is probably a Chrome bug, not a Galaxy one. -Alex

1 0

Re: [galaxy-dev] uploading binary files checksum changes, Galaxy doing something to file?
by Paniagua, Eric 28 Sep '11

28 Sep '11

Hi everyone, I am experiencing a similar problem to that of Leandro. I have defined a new datatype deriving Data, and having extension "foobar". The simplest and first thing I'd like to do with it is create a dataset of this new type "Foobar" by uploading from my local machine to my Galaxy development server. When I do the upload, I specify the file format as "foobar" rather than "Auto-detect". The file itself is a zip archive containing a folder containing other files, but the extension is ".foobar" and not ".zip". I encounter at least 2 (I believe separate) problems while trying to upload. 1. I can see that on upload the first non-directory entry in the ".foobar" file is being extracted and replacing the original. It seems this is how *unknown* zip archives are "supposed" to be handled by Galaxy, but as to why that is, I haven't a clue. 2. (This is what's similar to Leandro's case.) The dataset_XXX.dat file produced by the upload is not actually the same as the file it is "copied" from in the uploaded archive. The checksums are different, and the sizes are different (the dataset is 1 byte longer). I did a diff of the hexdumps of the dataset file and the corresponding file from the uploaded archive, and discovered the following: 1. Every occurrence of '^M' (aka '\r' aka 0x0D) has been replaced with '\n' (aka 0x0A). 2. A newline ('\n' aka 0x0A) is added to the end of the file. This file from the archive is not a text file, it is binary, so any code in Galaxy that tries to fix line endings shouldn't be doing this. (Where) Are there such places? Leandro, have you solved your problem? If not, what do you see when you do this kind of comparison? I am unable to reproduce those changes by stepping through the code in upload.py which handles zip files (and replaces them by their first file member) using the same python installation used for running Galaxy. This suggests the problem is elsewhere. Does anyone know why this '\r' -> '\n' mapping is affecting this file? Does anyone know why the default behaviour for uploading zip archives is to keep one file arbitrarily and throw out the rest? Even with an argument in favor of this behaviour, why is there not a "unzippable_file_formats" list for exceptions to be made like there is for sniffing? Any enlightenment on these matters would be greatly appreciated. Best, Eric ________________________________________ From: Leandro Hermida [softdev(a)leandrohermida.com] Sent: Friday, September 16, 2011 10:03 AM To: Paniagua, Eric Subject: Re: [galaxy-dev] uploading binary files checksum changes, Galaxy doing something to file? Hi Eric, On Fri, Sep 16, 2011 at 3:58 PM, Paniagua, Eric <epaniagu(a)cshl.edu> wrote: > Hi Leandro, > > Is there an entry in your history for the upload? What file format does it show? Is there any chance your original file was zipped? If Galaxy detected it as a zip file on upload, it may have unzipped it and taken the first file in it as the dataset. Yes there is an history entry for the upload. The format it shows is the new datatype I created (in datatypes_conf.xml, subclassing Binary) which I selected in the drop-down menu before uploading the file in the Get Data form. It is not a zip file. > That's at least the version of your problem that I've run into before. Specifying the file format manually (rather than choosing Auto-detect) may help if it's a similar problem. I suspect the correct solution is to write a sniffer for your datatype to help ensure it is identified correctly by Galaxy, but I haven't tried this yet. > Essentially the basic question is, how do you tell Galaxy not to do or touch absolutely *anything* with an uploaded binary file??? The checksums should always match. > Best of luck, > Eric > ________________________________________ > From: galaxy-dev-bounces(a)lists.bx.psu.edu [galaxy-dev-bounces(a)lists.bx.psu.edu] on behalf of Leandro Hermida [softdev(a)leandrohermida.com] > Sent: Friday, September 16, 2011 9:42 AM > To: Galaxy Dev > Subject: [galaxy-dev] uploading binary files checksum changes, Galaxy doing something to file? > > Hi all, > > We tried to find something in the docs and mailing list no luck. We > created a new datatype the is a straight subclass of Binary and then > when we upload such a file in the Galaxy UI and check the checksums > between the original file and the file located in the Galaxy > database/files/... directory their checksums are different! > > What are we doing wrong? We simply want Galaxy to upload and no touch > the file at all. > > regards, > Leandro > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ >

2 1

Galaxy multiple input files in a specific directory structure
by Henk van den Toorn 28 Sep '11

28 Sep '11

Dear all, I'm new here, although I was at a galaxy hack-a-thon before (Belgium this year). I've tried searching the history of the mailing list, but couldn't find something useful quickly, so I hope you can help! I have a rather specific problem with an application I am trying to incorporate into galaxy. This application does statistical analysis on any number of input files. The structure of the files in a (nested) directory structure determines which ones are conditions, and which ones are replicas etcetera. I'd like to be able to generate the input files in Galaxy, tell the application which file is what in terms of condition/replica, run the application and retrieve the output files (multiple again!). My question is, does anybody have experience in solving these kind of problems? If you have an answer, I'd be much grateful. If not, we'll try to find a solution ourselves. Of course we'll give it to the community if we do. Regards, Henk -- ------------------------------------------------------------------------------------------------------- *Henk van den Toorn, bioinformatician* Biomolecular Mass Spectrometry and Proteomics Group Bijvoet Center for Biomolecular Research and Utrecht Institute for Pharmaceutical Sciences Utrecht University Padualaan, 8 3584 CH Utrecht The Netherlands E-mail: h.w.p.vandentoorn(a)uu.nl Tel: +31 30 253 6758 Fax: +31 30 253 6919 --------------------------------------------------------------------------------------------------

1 0

Data transfer between two galaxy instances
by lentaing＠jimmy.harvard.edu 28 Sep '11

28 Sep '11

Hi, We are running two instances of galaxy on our site--one internal, the other external. Some of our internal users have data on the external instance. What might be the easiest way to transfer the data of our internal users from the external to the internal instance? OR Is there a way to transfer data between galaxy instances? I thought that I'd give this mailing list a shot before telling our internal users that they would have to re-load the data on the internal instance. Thanks, Len Taing

3 3

Fetch failed in run.sh - Ubuntu 11.04 zlib not available
by Joe Harrison 27 Sep '11

27 Sep '11

Hello - I am a microbiologist and I am attempting to set up a a local instance of Galaxy. I'm very green with the Linux environment! So you will have to pardon a very naive question from a biologist... I have installed VirtualBox on my PC and I am currently running Ubuntu Linux 11.04 (64 bit). I have downloaded the galaxy distribution from the mercurial repository and have executed the run.sh script. However, Galaxy fails to launch, and I get the following error message: zipimport.ZipImportError: can't decompress data; zlib not available Fetch failed. Would anyone have a fix for this problem? I have tried installing zlibc using the following command: sudo apt-get install zlibc But this does not seem to resolve the issue... Is this something to do with Ubuntu (should I try a different Linux environment)? Any recommendations would be deeply appreciated. Joe --- Joe J. Harrison Senior Fellow Department of Microbiology University of Washington 1705 NE Pacific Street, HSB J181 Seattle WA USA (206) 543-3147

2 1

HOW TO RETRIEVE DATA FROM HISTORY??!!
by colin molter 27 Sep '11

27 Sep '11

Hi all, i am still stuck with the same problem. Is there a way to directly move/copy data from your galaxy history to a given location in the filesystem of the same galaxy server? Said differently, there is a nice way to import data from the server to galaxy, is it possible to do the reverse? So far, I am obliged to download the file from galaxy to my client machine and then back to the server!!!! with huge bam files of 3Gb it is not so convenient!! thank you all colin

5 6

conversion of fastq files
by mallika V 27 Sep '11

27 Sep '11

hi I am very new to the galaxy software. I have to convert one fastq file of nearly 5 gb size and unable to do that. i tried to install galaxy in locally but it tells some eggs are out of date. Can anyone plz help me in this regard -- "The best and most beautiful things in the world cannot be seen or even touched. They must be felt within the heart" (Helen Keller)

2 1