On Mar 12, 2012, at 9:44 AM, Louise-Amélie Schmitt wrote:
Hi Nate, and thanks for the reply
- We saw the outputs_to_working_directory option in the .ini file, but it only concerns output files, is there a way to make a local copy of all the input files in the job working directory? (including indices) Hi L-A,
Unfortunately no. This option was only created so that you could mount the file_path directory read-only on your cluster nodes for security purposes.
In the end it shouldn't be necessary. The guy in charge of the cluster said so.
- Can we set the job working directory as an absolute path so it's local to the nodes, like /tmp ? No, things like the external metadata inputs and outputs are written to the working directory and are expected to be available on both the cluster and the application server at the same path.
Ok, I see. How do you manage that then? Is it actually not an issue I/O-wise?
Most of the things written to the working directory aren't large enough to cause performance problems. The worst culprit, before we split up the load across a pool of servers, was tophat. Ideally, Galaxy needs a mechanism to define whether local or network temp space should be used (or perhaps more abstractly, a way to estimate the size needed based on the inputs). --nate
- If it's possible, will the job working directories created for each job be cleaned properly at the end? Yes, depending on the value of "cleanup_job" in the config file.
Enhancements that remove this "shared directory" limitation are on our roadmap for this year. Ok, thanks a lot for all this information!
Best, L-A
--nate
Thanks, L-A ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: