- galaxy-dev - lists.galaxyproject.org

Downloading datasets without username/password when login is required
by Sytchev, Ilya 26 Feb '13

26 Feb '13

Hi, Is it possible to download a dataset when require_login=True in universe_wsgi.ini using the API key? Thanks, Ilya

2 3

error installing BWA from toolshed
by Quang Trinh 26 Feb '13

26 Feb '13

Hi dev, I am trying to install bwa from toolshed on my Galaxy instance on Amazon just now and get this error ( please see below). Installing bwa worked for me before. URL: http://xxx.xxx.xxx.xxx/admin_toolshed/prepare_for_install?tool_shed_url=htt… Module paste.exceptions.errormiddleware:143 in __call__ >> app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ >> environ, self.app) Module paste.wsgilib:539 in intercept_output >> app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ >> return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ >> body = method( trans, **kwargs ) Module galaxy.web.framework:206 in decorator >> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:1199 in prepare_for_install >> description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, tool_dependencies = repo_info_tuple ValueError: too many values to unpack Thanks, Q

3 3

Re: [galaxy-dev] [galaxy-user] How to create a tool from a script that loads additional source code files?
by Peter Cock 26 Feb '13

26 Feb '13

On Mon, Feb 25, 2013 at 7:11 PM, Samuel Lampa <samuel.lampa(a)gmail.com> wrote: > Hi, > > I have aGalaxy toolwith some R code which is split up into a few files, > which are loaded by a "master R script", which orcestrates and executes the > whole thing. > > I realize that when the master script ("run.R" in my case) is run, it will > not find the other files, since it is itself moved away to a temporary > working directory (something like > ".../galaxy/database/job_working_directory/000/165"). > > Is there any "standard way" of dealing with this, any special toolconfig > syntax etc? ... or do I have to just hard code the absolute paths to the > extra files in my script? :| Presumably all the child R scripts will be in the same folder as your master R script (or you know the relative path if not). So all you need to do is workout the path of the master R script, and from that compute the path to the child R scripts. I guess you can do this in R using commandArgs(). For future reference, queries about writing tools and extending Galaxy should go to the galaxy-dev list (CC'd), not galaxy-users. Peter

2 2

2013 Galaxy Community Conference (GCC2013) Registration and Abstract Submission are Now Open
by Dave Clements 24 Feb '13

24 Feb '13

Dear Galaxy Community, We are pleased to announce that early registration<http://wiki.galaxyproject.org/Events/GCC2013/Register> and paper and poster abstract submission<http://wiki.galaxyproject.org/Events/GCC2013/Abstracts> are now open for the 2013 Galaxy Community Conference (GCC2013)<http://wiki.galaxyproject.org/Events/GCC2013> . GCC2013 will be held 30 June through July 2 in Oslo Norway, at the University of Oslo <http://uio.no/>. GCC2013 <http://wiki.galaxyproject.org/Events/GCC2013> is an opportunity to participate in two full days of presentations, discussions, poster sessions, keynotes, lightning talks and breakouts, all about high-throughput biology and the tools that support it. The conference also includes a Training Day<http://wiki.galaxyproject.org/Events/GCC2013/TrainingDay> for the second year in a row, this year with more in-depth topic coverage, more concurrent sessions, and more topics. If you are a biologist or bioinformatician performing or enabling high-throughput biological research, then please consider attending. GCC2013 is aimed at: - Bioinformatics tool developers and data providers - Workflow developers and power bioinformatics users - Sequencing and Bioinformatics core staff - Data archival and analysis reproducibility specialists *Early registration <http://wiki.galaxyproject.org/Events/GCC2013/Register>* *saves up to 75% off regular registration costs,* and is very affordable, with combined registration (Training Day<http://wiki.galaxyproject.org/Events/GCC2013/TrainingDay> + main meeting) starting at ~ €95 for post-docs and students. Registering early also assures you a spot in the Training Day workshops you want to attend. Once a Training Day session becomes full, it will be closed to new registrations. Early registration closes 24 May. *Abstract submission<http://wiki.galaxyproject.org/Events/GCC2013/Abstracts> * for oral presentations closes 12 April, and for posters on 3 May. Please consider presenting your work. If you are working with big biological data, then the people at this meeting want to hear about your work. Thanks, and hope to see you in Oslo! The GCC2013 Organizing Committee<http://wiki.galaxyproject.org/Events/GCC2013/Organizers> PS: And please help get the word out<http://wiki.galaxyproject.org/Events/GCC2013/Promotion> ! -- http://galaxyproject.org/GCC2013 <http://galaxyproject.org/wiki/GCC2012>http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

1 0

Amazon EC2 Cloudman: Running parallel Tophats and Master node has 260% load, worker idle and at 46%
by Brian Lin 24 Feb '13

24 Feb '13

Hi guys, I'm fairly new to cloud computing and about 2 days in to using cloudman for galaxy... I have setup a m2.4xlarge master node and flexible load for up to 4 workers of the m2.xlarge type, minimum one. I was able to upload 6 samples of paired end RNAseq->12 files, gz file sizes around 5-8 gb. Grooming files took about a day, but my previous experience was on an in-house Galaxy install which was pretty small so I didn't think of anything at the time. I started 3 Tophat jobs and I noticed the UI being a bit sluggish to respond, added a 4th one hoping it might push it over the edge for the worker nodes. Unfortunately, 12 hours later, the Tophats are still running, the master node is way over 100%, and the worker is reported idle. While the cluster log has a few errors in it, it ends saying that the instance for the worker node is ready, despite it encountering errors adding it to the SGE host, code1. Has anyone run into this before or has any insight to fixing this? I'm pasting the lower portion of the status log below. Thanks very much for any help! (Also, several other emails about cloud installs were directed to /dev. If that's not the right place for this question, I apologize and can change to the /user list.) -- Brian Lin contact(a)brian-lin.com brian.lin(a)tufts.edu 13:15:38 - Instance 'i-e2bb0491' reported alive 13:15:38 - Sent master public key to worker instance 'i-e2bb0491'. 13:15:54 - Adding instance i-e2bb0491 as SGE administrative host. 13:16:06 - Adding instance 'i-e2bb0491' to SGE execution host list. 13:16:11 - Process encountered problems adding instance 'i-e2bb0491' as an SGE execution host. Process returned code 1 13:16:26 - Waiting on worker instance 'i-e2bb0491' to configure itself... 13:17:05 - Instance 'i-e2bb0491' (IP: 23.23.24.81) ready 13:42:23 - Rebooting instance i-e2bb0491 (reboot #3). 13:44:27 - Instance 'i-e2bb0491' reported alive 13:44:27 - Sent master public key to worker instance 'i-e2bb0491'. 13:44:56 - Adding instance i-e2bb0491 as SGE administrative host. 13:44:56 - Adding instance 'i-e2bb0491' to SGE execution host list. 13:44:56 - Process encountered problems adding instance 'i-e2bb0491' as an SGE execution host. Process returned code 1 13:44:56 - Waiting on worker instance 'i-e2bb0491' to configure itself... 13:46:09 - Instance 'i-e2bb0491' (IP: 23.23.24.81) ready 15:19:37 - Rebooting instance i-e2bb0491 (reboot #4). 15:21:16 - Instance 'i-e2bb0491' reported alive 15:21:16 - Sent master public key to worker instance 'i-e2bb0491'. 15:21:38 - Adding instance i-e2bb0491 as SGE administrative host. 15:21:44 - Adding instance 'i-e2bb0491' to SGE execution host list. 15:21:48 - Process encountered problems adding instance 'i-e2bb0491' as an SGE execution host. Process returned code 1 15:21:51 - Waiting on worker instance 'i-e2bb0491' to configure itself... 15:22:13 - Instance 'i-e2bb0491' (IP: 23.23.24.81) ready 15:54:32 - Instance i-e2bb0491 not responding after 4 reboots. Terminating instance. 15:54:32 - Terminating instance i-e2bb0491 15:54:35 - Instance 'i-e2bb0491' removed from the internal instance list. 15:56:10 - Adding 1 on-demand instance(s) 15:56:14 - Cannot get cloud instance object without an instance ID? 15:58:26 - Instance 'i-98932deb' reported alive 15:58:26 - Sent master public key to worker instance 'i-98932deb'. 15:59:02 - Adding instance i-98932deb as SGE administrative host. 15:59:09 - Adding instance 'i-98932deb' to SGE execution host list. 15:59:17 - Successfully added instance 'i-98932deb' to SGE 15:59:17 - Waiting on worker instance 'i-98932deb' to configure itself... 15:59:46 - Instance 'i-98932deb' (IP: 54.234.100.22) ready 16:12:47 - Rebooting instance i-98932deb (reboot #1). 16:12:47 - Rebooting instance i-98932deb (reboot #1). 16:12:47 - Rebooting instance i-98932deb (reboot #1). 16:14:06 - Instance 'i-98932deb' reported alive 16:14:06 - Sent master public key to worker instance 'i-98932deb'. 16:14:29 - Adding instance i-98932deb as SGE administrative host. 16:14:29 - Adding instance 'i-98932deb' to SGE execution host list. 16:14:29 - Process encountered problems adding instance 'i-98932deb' as an SGE execution host. Process returned code 1 16:14:29 - Waiting on worker instance 'i-98932deb' to configure itself... 16:14:39 - Instance 'i-98932deb' (IP: 54.234.100.22) ready

1 0

BWA mapping taking >48 hours
by stefano cardinale 22 Feb '13

22 Feb '13

Hello, I am not sure whether this is a bug or a problem with the server, or if it is just normal but I started a BWA mapping on some paired-end reads of a bacterium genome two days ago late evening. The job appears in yellow as 'running' but is it normal that is taking all these hours? I must say this is not the first time it happens. any advice? thanks stefano

2 1

Re: [galaxy-dev] Installing Galaxy on ARM Architecture
by Nate Coraor 22 Feb '13

22 Feb '13

On Feb 21, 2013, at 11:47 AM, Shun Liang wrote: > Sorry just forgot to CC the mailing list. > ________________________________________ > > Hi James, > > I followed the manual installation instruction on "https://bitbucket.org/james_taylor/bx-python/wiki/HowToInstall", and surprisingly, bx-python seems to be built on the ARM server without any problem: > > Installed /nfs/users/bx-python/lib/python/bx_python-0.7.1-py2.7-linux-armv7l.egg > Processing dependencies for bx-python==0.7.1 > Finished processing dependencies for bx-python==0.7.1 Hi Shun. I'm moving this over to galaxy-dev since it's about a local installation. Galaxy's eggs lib isn't finding the egg you built because Galaxy adds the Mercurial revision to the egg version, and then expects the egg to be physically located in galaxy-dist/eggs/. The library will be looking for: bx_python-0.7.1_7b95ff194725-py2.7-linux-armv71.egg If you are able to build it through scramble that would be ideal since it handles that naming for you, and there will be other C-Extension eggs to build under similar constraints. There should have been an error earlier in the scramble output with more details on why building bx-python failed. If you rerun scramble on just that egg, you should be able to recapture the output: % python ./scripts/scramble.py -e bx_python --nate > > > > Regards, > Shun > ________________________________________ > From: james(a)taylorlab.org [james(a)taylorlab.org] on behalf of James Taylor [james(a)jamestaylor.org] > Sent: 21 February 2013 16:07 > To: Shun Liang > Cc: galaxy-user(a)lists.bx.psu.edu > Subject: Re: [galaxy-user] Installing Galaxy on ARM Architecture > > You are definitely in uncharted territory with an arm server. Can you > try building just bx-python itself and let us know what the errors > are? > > -- > James Taylor, Assistant Professor, Biology/CS, Emory University > > > On Thu, Feb 21, 2013 at 11:03 AM, Shun Liang <Shun.Liang(a)cruk.cam.ac.uk> wrote: >> Hi, >> >> I am trying to install Galaxy on an ARMV7 architecture Linux server. I ran >> "run.sh" and failed because some of the python eggs could not be fetched. I >> then have a look at "http://eggs.galaxyproject.org/" and I have realized >> there aren't any armv7 builds for those eggs. >> >> I then decided to build (or scramble) the eggs on my own. I ran >> "scripts/scramble.py" and this also failed with the following message (after >> building a lot of things): >> >> scramble(): Copied egg to: >> /nfs/users/galaxy_dist/eggs/twill-0.9-py2.7.egg >> Traceback (most recent call last): >> File "scripts/scramble.py", line 26, in <module> >> eggs = c.scramble() >> File "/nfs/users/galaxy_dist/lib/galaxy/eggs/scramble.py", line 242, in >> scramble >> raise last_exc # only 1 failure out of the crate, be more informative >> galaxy.eggs.scramble.ScrambleFailure: run_scramble_script(): Egg build >> failed for bx_python 0.7.1 >> >> >> May I ask what may cause this problem? Have I done anything wrong? Or, is it >> even possible to install Galaxy on ARM architecture Linux? >> >> >> >> Many thanks, >> Shun >> >> >> ___________________________________________________________ >> The Galaxy User list should be used for the discussion of >> Galaxy analysis and other features on the public server >> at usegalaxy.org. Please keep all replies on the list by >> using "reply all" in your mail client. For discussion of >> local Galaxy instances and the Galaxy source code, please >> use the Galaxy Development list: >> >> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> To manage your subscriptions to this and other Galaxy lists, >> please use the interface at: >> >> http://lists.bx.psu.edu/ > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

3 4

Tabular file metadata - columns names
by Peter 22 Feb '13

22 Feb '13

Hi all, I'd like to know more about Galaxy's column metadata for tabular files. In the workflow editor under "Edit Step Actions" you can pick "Assign Columns", and then give column numbers for five predefined cases: Chrom, Start, End, Strand, Name. Do these "named columns" get shown anywhere in the Galaxy UI? For example, in a column select parameter widget? Is it possible to assign these columns in a tool's wrapper XML file? >From http://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax I'm aware of the metadata_source attribute to *copy* the meta data from the input file, but that isn't always relevant. Can I somehow specify that my tool has tabular output where column 1 is "Name"? Is it possible to introduce additional column types? e.g. "evalue" or "Description". Thanks, Peter

3 7

Filter tool: how to override automatic guessing of column types ?
by Marc Logghe 22 Feb '13

22 Feb '13

Hi, I have an issue with filtering using the Filter tool in the sense that the first line is considered invalid and I think it has to do with the automatic guessing of column types. The condition is like: c4 !='XXX' The value of c4 for the first line is 'NA' When the paster.log is checked one can see that the last argument for filtering.py are the column types: "str,str,int,int,int ". As you can see for whatever reason, the system thinks c4 is an int, it should be 'str'. Indeed, most values for that field look like numbers but they should be treated as string types. In an attempt to override the guessing, a custom tabular type was defined in lib/galaxy/datatypes/tabular.py as such: class CloneAdminTable( Tabular ): """ Tabular format Clone Administration """ column_names = ['Name', 'SeqInAlignment', 'Family', 'Animal/library', 'NR Cluster'] column_types = ['str','str','int','str','int'] def display_peek( self, dataset ): """Returns formated html of peek""" return Tabular.make_html_table( self, dataset, column_names=self.column_names ) But that seems not to help very much; the system still guesses the column types. Any idea how to solve this ? Thanks and regards, Marc ________________________________________ THIS E-MAIL MESSAGE IS INTENDED ONLY FOR THE USE OF THE INDIVIDUAL OR ENTITY TO WHICH IT IS ADDRESSED AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, CONFIDENTIAL AND EXEMPT FROM DISCLOSURE. If the reader of this E-mail message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately at ablynx(a)ablynx.com. Thank you for your co-operation. "NANOBODY" and "NANOCLONE" are registered trademarks of Ablynx N.V. ________________________________________

2 4

Datatype display_in_upload not respected in upload1
by Nicola Soranzo 22 Feb '13

22 Feb '13

Hi all, I noticed that when I go in "Get Data" -> "Upload File from your computer" (i.e. upload1 tool) the "File Format" list contains also datatypes which have display_in_upload="false" in datatypes_conf.xml (e.g. "afg", "fli"), while the list does not contain datatypes which have no display_in_upload attribute at all (e.g. "bedstrict", "bed6"). This sounds like a bug to me, but I have not found any reference around. Thanks, Nicola -- Nicola Soranzo, Ph.D. CRS4 Bioinformatics Program Loc. Piscina Manna 09010 Pula (CA), Italy http://www.bioinformatica.crs4.it/

3 3