On Dec 20, 2011, at 5:04 AM, Cittaro Davide wrote:
Hi developers, I have a question that may be an OT, but since galaxy can work in a clustered environment withh queueing system, I'll try to ask here. Is there anibody here who copies data in a local temporary directory before performing any analysis step and copy it back into the "final results"?
Hi Davide, We did this for a while when we had a poorly performing fileserver. It can reduce load in that environment, but in cases where you are only going to read small portions of input files, you'll probably have longer execution time. Likewise if you'll simply be writing the output(s) in one big stream, since you then have to write it once locally and then back over the network. That said, if you have a lot interim steps that produce large data that then get merged via some process back to final outputs, it absolutely makes sense to use local disk for those steps (assuming local disk is large enough - another problem that we sometimes encounter). --nate
Thanks
d
Sent from my iPad ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: