I also second this request to get it addressed (Where can we vote on bug fixes ?! :) ...It is very weird that samtools is run on the local machine and it even does the indexing sequentially...

Thon

On Jan 23, 2013, at 03:28 PM, Kyle Ellrott <kellrott@soe.ucsc.edu> wrote:

I'm currently in the process of loading (path paste) a large library of BAM files (>10000) into the shared Data Libraries of our local galaxy installation, but I'm finding this process to be very slow.

I'm doing a path paste, and not actually copying the files. I have disabled local running of 'upload1', so that it will run on the cluster, and set 'set_metadata_externally' to true. 

It looks like the job handlers are calling 'samtools index' directly. Looking through the code, that seems to happen in galaxy/datatypes/binary in Bam.dataset_content_needs_grooming, where it calls 'samtools index' and then waits. 

What would be the most efficient way to start changing the code so that this process can be done by an external script, at a deferred time out on the cluster?

Kyle

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/