Peter Cock wrote:
On Wed, Jul 27, 2011 at 9:06 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Tue, Jul 26, 2011 at 8:16 PM, Luobin Yang <yangluob@isu.edu> wrote:
I benchmarked MrBayes 3.1.2 program on my cluster for two cases: 1. use local /tmp for temporary files 2. use the network shared /home/galaxy/galaxy-dist/database/tmp MrBayes is about 10 times slower for case 2 than for case 1. What I did was to set the network shared folder as the default but in the MrBayes wrapper, I changed the environment variable TEMP to be a local folder. Luobin
Does that mean Galaxy will configure the TEMP environment variable for tools to point at the universe_wsgi.ini new_file_path setting? In your case, /home/galaxy/galaxy-dist/database/tmp
This is the kind of thing I think should be documented somewhere for tool authors ...
... and mentioned in the universe_wsgi.ini text for the new_file_path setting.
I've just seen Shantanu Pavgi's thread which also covers this issue, and the related TMP and TMPDIR environment variables set up or overridden by SGE. I think we agree that Galaxy needs some documentation and guidance in this area for tool authors.
Galaxy doesn't modify $TEMP, but in the past, setting $TEMP would cause Galaxy to use that directory for all files created by Python's tempfile module. Since there was confusion over the difference between $TEMP (which was documented as the way to control the creation of temp files) and new_file_path, Galaxy was changed to put all temp files in the location of new_file_path, ignoring $TEMP. However, this only applies to the framework. Tools run in a seperate process and are not forced to use new_file_path. If you want your tool to use it, you could pass the value of new_file_path to your tool. Otherwise, whatever rules are used by whatever you're using to create temporary files will be what determines where temp files go; in Python, it's this: http://docs.python.org/library/tempfile.html#tempfile.tempdir Our cluster nodes have the majority of their local disk partitioned as /space, so we set TEMP=/space in the environment used on the nodes. --nate
Peter
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: