RFC: GALAXY_MEMORY similar to GALAXY_SLOTS
Hi, if we want to include JAVA programs into Galaxy we are usually forced to hardcode -Xmx4g -Xms3g or similar into the wrapper. Would it make sense to have something like GALAXY_MEMORY, similar to GALAXY_SLOTS, to obtain the maximum of allowed memory from the scheduler. With some cheetah magic we would be able to create dynamic "-Xmx4g -Xms3g" entries. Ciao, Bjoern
Hey Bjoern, This seems a reasonable request - I need to do some research of different DRMs and see how different ones specify this and see how easy this is to implement. I have taken over a Trello card for this - https://trello.com/c/HGEYPQf6. If some kind soul the community were to send me "these are the environment variables and formulae for PBS, GE, and SLURM to compute this unit of memory (whatever it may be KB, MB, or GB)" the process would go faster :). The one modification I would make is the cheetah magic comment - GALAXY_SLOTS is set at job runtime on the worker node - cheetah is evaluated at job creation time on the Galaxy web server node (or wherever the handler is running). So you need to use more challenging sh magic or use a wrapper script. I know Nicola updated the wiki in a way that made it seem like GALAXY_SLOTS was a cheetah variable - this is entirely my fault - I have largely failed at explaining the distinction between cheetah and the job script and what gets requested by Galaxy versus what gets assigned by the DRM. I will find a competent way to talk about this stuff at some point. In the meantime I have updated the wiki to clarify the distinction. -John On Fri, Nov 29, 2013 at 8:08 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Hi,
if we want to include JAVA programs into Galaxy we are usually forced to hardcode -Xmx4g -Xms3g or similar into the wrapper. Would it make sense to have something like GALAXY_MEMORY, similar to GALAXY_SLOTS, to obtain the maximum of allowed memory from the scheduler. With some cheetah magic we would be able to create dynamic "-Xmx4g -Xms3g" entries.
Ciao, Bjoern
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (2)
-
Bjoern Gruening
-
John Chilton