August 2011 - galaxy-dev - lists.galaxyproject.org

TEMP variable in cluster install
by Shantanu Pavgi 06 Sep '11

06 Sep '11

We have configured galaxy to work with our SGE cluster using drmaa job runner interface. We are using 'unified method' for this install and both TEMP environment variable and new_file_path in universe_wsgi.ini file have been configured correctly. However, we are seeing some errors where local /tmp space on compute nodes is being referenced by the galaxy tools. Specifically we saw it mentioned in error messages from following tools: * bwa_wrapper and upload tools: 'No space left on device: /tmp...' * sam_to_bam tool: 'Error sorting alignments from /tmp/filename..' Shouldn't it be referencing TEMP environment variable or new_file_path configuration value? Is it getting overridden by TMP or TMPDIR variables in python wrapper code? Has anyone else experienced similar issue? -- Thanks, Shantanu.

5 6

mycoplasma genomes - suggestion
by Lukasz Kielpinski 06 Sep '11

06 Sep '11

Hello everybody, I would like to suggest including mycoplasma genomes (preferably all genomes in one file) as reference genomes. It would enable fast checking if cells used for deep-seq experiments were not contaminated. Best, Lukasz

2 1

contribute tools to galaxy?
by Dongjun Chung 01 Sep '11

01 Sep '11

Hi All, I'm a newbie to galaxy and enjoying it a lot these days. Thanks for the great work. I have a question regarding contribution of software to galaxy. We developed a ChIP-seq peak calling algorithm and software (R package) and hope to contribute it to galaxy. I have read the wiki and prior mailing list about the contribution system but it is still somewhat confusing to me. 1. It seems that I can contribute our software to "tool shed" if I prepare appropriate code & definition files. Then, users can download and use it with their locally installed galaxy. However, these files committed to tool shed will not appear in galaxy main or test servers. Am I correct? 2. What is clear relationship between main/test servers & tool shed? Can we contribute our software to main or test servers as well? Or only galaxy core developers can add new tools to galaxy main or test servers? If so, which software is considered to be added to servers? Are they chosen from tools contributed to tool shed? 3. If our software is a R package, then users need to download and install it first in their R system before they use our software within their galaxy system even in the case they have appropriate definition files. Am I right? Or is there any better solution for this? Thanks! Best, Dongjun

2 2

Re: [galaxy-dev] Display BAM with IGV
by Iry Witham 01 Sep '11

01 Sep '11

Hi Everyone, I have been tasked with getting the IGV working within our Galaxy instance. I have modified our Nginx configuration based on the more advanced configuration under "Sending files using nginx" on the nginxProxy page in the wiki. The IGV viewer component in Galaxy does not display the results from a BAM file in either the web or local instance of IGV. I have tested with IGV1.5 and with version 2.03. Neither produce a visual representation of the sequence. When attempting to use the web option we get the same error provided in the original post as noted below. Has anyone gotten this to work with Nginx-0.8.53 or any of the newer versions? Thanks, Iry Witham Applications Administrator Scientific Computing Group Computational Sciences Dept. The Jackson Laboratory 600 Main Street Bar Harbor, ME 04609 Phone: 207-288-6744 email: iry.witham(a)jax.org On Apr 27, 2011; 10:06am “Hello all, I just updated our local copy of Galaxy and just wanted to try the new IGV display application. When I try to display a BAM file with the local IGV or the web IGV I encounter following error complaining about the Null content-length. Has any of you encountered the same problem? Do I miss something? Thanks in advance Alex INFO [2011-04-27 16:00:14,440] [TrackManager.java:225] [Thread-6] Loading1 resources. ERROR [2011-04-27 16:00:14,589] [SeekableHTTPStream.java:59] [Thread-6] Null content-length for: http://localhost:8080/display_application/a1f8a55a946ee078/igv_bam/local/45… ERROR [2011-04-27 16:00:14,590] [TrackLoader.java:821] [Thread-6] Error loading sam track java.lang.RuntimeException: Error loading file: http://localhost:8080/display_application/a1f8a55a946ee078/igv_bam/local/45… NULL content-length at org.broad.igv.util.SeekableHTTPStream.<init>(SeekableHTTPStream.java:60) at org.broad.igv.util.SeekableStreamFactory.getStreamFor(SeekableStreamFactory.java:42) at org.broad.igv.sam.reader.BAMHttpQueryReader.loadHeader(BAMHttpQueryReader.java:129) at org.broad.igv.sam.reader.BAMHttpQueryReader.getHeader(BAMHttpQueryReader.java:70) at org.broad.igv.sam.reader.BAMHttpQueryReader.getSequenceNames(BAMHttpQueryReader.java:80) at org.broad.igv.sam.reader.CachingQueryReader.getSequenceNames(CachingQueryReader.java:97) at org.broad.igv.sam.AlignmentDataManager.initChrMap(AlignmentDataManager.java:75) at org.broad.igv.sam.AlignmentDataManager.<init>(AlignmentDataManager.java:69) at org.broad.igv.sam.AlignmentDataManager.getDataManager(AlignmentDataManager.java:54) at org.broad.igv.track.TrackLoader.loadAlignmentsTrack(TrackLoader.java:761) at org.broad.igv.track.TrackLoader.load(TrackLoader.java:143) at org.broad.igv.track.TrackManager.load(TrackManager.java:271) at org.broad.igv.track.TrackManager.loadResources(TrackManager.java:241) at org.broad.igv.main.CommandExecutor.loadFiles(CommandExecutor.java:282) at org.broad.igv.main.CommandExecutor.hget(CommandExecutor.java:148) at org.broad.igv.main.CommandExecutor.execute(CommandExecutor.java:88) at org.broad.igv.main.CommandListener.processGet(CommandListener.java:193) at org.broad.igv.main.CommandListener.run(CommandListener.java:93) at java.lang.Thread.run(Thread.java:636) ERROR [2011-04-27 16:00:14,592] [TrackLoader.java:211] [Thread-6] java.lang.RuntimeException: java.lang.RuntimeException: Error loading file: http://localhost:8080/display_application/a1f8a55a946ee078/igv_bam/local/45… NULL content-length” On Apr 27, 2011; 12:33pm “Hi Alex, Displaying BAM files at external applications usually requires an http server that supports partial gets. Unfortunately, the current lightweight http server that is built into Galaxy does not support these advanced http options. However, this limitation can be overcome by using a proxy server that is able to handle http1.1 requests. Please see "Using a Proxy server" at https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ProductionServer for help on setting this up. Thanks for using Galaxy, Dan” On Apr 27, 2011; 9:07pm “I’m getting this same error and I’m running nginx as a proxy server. Is there a specfic parameter we should be aware of to allow partial gets? I don’t see anything in the wiki page for setting up the ProductionServer. Thanks, Dave” On May 04, 2011; 7:47am “Could someone who has successfully gotten the IGV tool to work using NGINX as your proxy, tell me if there was anything specific they needed to do with their NGINX or galaxy config to get it working? Thanks, Dave” On May 09, 2011; 2:52pm “It's necessary to set up the more advanced configuration under "Sending files using nginx" on the nginxProxy page in the wiki. Could you let us know whether you're using that configuration or just a basic pass-through proxy? Thanks, --nate”

3 8

database migration error 79->80 (NameError: name 'BigInteger' is not defined)
by Edward Kirton 01 Sep '11

01 Sep '11

hi, we are getting the following error migrating from 79 to 80. curiously, my own galaxy didn't have this problem, but another developer here was getting this error. i moved him from sqlite to postgres but that didn't help. thanks for any assistance, ed Migrating 79 -> 80... galaxy.model.migrate.check INFO 2011-08-31 13:15:31,446 Traceback (most recent call last): File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-jeff/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-jeff/lib/galaxy/app.py", line 32, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options ) File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-jeff/lib/galaxy/model/migrate/check.py", line 67, in create_or_verify_database migrate_to_current_version( engine, db_schema ) File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-jeff/lib/galaxy/model/migrate/check.py", line 125, in migrate_to_current_version schema.runchange( ver, change, changeset.step ) File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-depot/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/schema.py", line 184, in runchange File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-depot/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/script/py.py", line 100, in run File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-depot/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/script/py.py", line 112, in _func File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-depot/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/script/py.py", line 108, in module File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-depot/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/script/py.py", line 65, in verify_module File "/house/groupdirs/genetic_analysis/jfroula/Projects/Galaxy/galaxy-depot/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/util/importpath.py", line 12, in import_path File "lib/galaxy/model/migrate/versions/0080_quota_tables.py", line 29, in <module> Column( "bytes", BigInteger ), NameError: name 'BigInteger' is not defined

2 2

Question about data size
by Jennifer Jackson 31 Aug '11

31 Aug '11

Hi Doug, I am going to forward your question to the galaxy-dev mailing list so that the development community can offer feedback based on their experience. Best, Jen Galaxy team On 8/15/11 9:47 AM, Douglas Hinerfeld wrote: > HI There, > I am trying to get my head around how much data is generated when > analyzing Illumina HiSeq data in Galaxy. I understand that it totally > depends on what type of analytical pipelines are used. From our own > usage, we are seeing the generation of about 30 bytes of data generated > per base of sequence. I don’t know if you have any data on this that can > be shared, but I am just trying to get a sense if this is similar to > other’s experience. > Thanks, > Doug -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/Support

1 0

[API] managing histories
by Louise-Amélie Schmitt 31 Aug '11

31 Aug '11

Hello everyone, I'm currently trying to automate data loading and pre-processing through Galaxy's API, and I will need to delete and share histories at some point. I know the API is still very new but is there a way to do that by any chance? Thanks, L-A

2 2

Re: [galaxy-dev] [galaxy-user] install tools on cluster
by Nate Coraor 31 Aug '11

31 Aug '11

Wei,Xintao wrote: > Hi, > > I am learning how to install and configure Galaxy on our computer cluster. Many third party tools, for example, bowtie, samtools, R and so on, have to be installed and configured in order to install galaxy. Do I have to convert the third party tools to MPI version in order to run the tools on a computer cluster? Or do I just install the "regular" version of the tools on the "shared" folder of our cluster? Hi Xintao, I've moved this question over to the dev list since it concerns a local installation. You don't need to convert them to MPI, they run as serial jobs on a single node. Note that you can specify the number of cores to use in the tool's config file, which you'll want to set for the nodes you're running them on. --nate > > Thank you! > > Xintao > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

1 0

Backup strategy
by Oleksandr Moskalenko 31 Aug '11

31 Aug '11

Hello, Considering the fact that a galaxy instance can grow to be quite large I wonder what Galaxy backup strategies and implementations other people actually have in use and are happy with. Thanks, Alex

1 0

bug or feature :)
by Bossers, Alex 31 Aug '11

31 Aug '11

In our local instance (central version) up to changeset (June 28th) I noticed in the admin panel to manage jobs some odd behaviour; When I set the update Jobs cutoff on 10 seconds and press refresh.... the item below (administrative Job Lock) is also activated and new jobs are locked. Is that a planned feature, a bug or is something wrong in our local instance. Another refresh toggles it back to normal. Alex

3 2