how to add settings of TMPDIR=/scratch for sge/drmaa in universe_wsgi.ini
Hi Tin, I am going to move this over to the galaxy-dev@bx.psu.edu mailing list (good for local install questions) to give it better viability. I am by no means the best person to help with this, so hopefully others will jump in, but just from looking at this I am wondering if you shouldn't be instead looking at the " job_conf.xml.sample_advanced" file. There is the default version as well, but it is probably not what you are looking for if you want to add custom logging scripts. For universe_wsgi.ini, it seems like maybe the problem with the syntax is most likely the unknown "default_cluster_job_runner" variable and the double "=" in the line. Slashes should be fine unescaped here. But as I said, I think the other file is the place for the changes anyway, or maybe both, but using known variables and a single "=" per path. But let's see what the developers think! Jen Galaxy team On 8/12/13 3:49 PM, tin h wrote:
Hello galaxy gurus,Â
I am trying to configure a local galaxy server to submit to SGE using DRMAA, it works, but I would like to specify the equivalent of "-v TMPDIR=/scratch" for qsub.
I tried updating universe_wsgi.ini with something like: Â default_cluster_job_runner = Â drmaa://-q default.q -V -v TMPDIR=/scratch/ but that doesn't work because "/" is a delimiter, and I have not been able to find way to escape it. Â Is there any tricks for this?
Much thanks in advance for your help in this matter. -Tin
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson http://galaxyproject.org
Thank you Jennifer. I am not sure if my previous email got gabled up... Just to be sure, I specified the following in universe_wsgi.ini default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/ when i start galaxy, looking at the console log, this is what I see: galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Loaded job runner 'galaxy.jobs.runners.drmaa:DRMAAJobRunner' as 'drmaa' galaxy.jobs.runners.drmaa DEBUG 2013-08-14 14:08:14,885 Converted URL 'drmaa://-q default.q -V -v TMPDIR=/scratch/' to destination runner=drmaa, params={'nativeSpecification': '-q default.q -V -v TMPDIR='} galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Legacy destination with id 'drmaa://-q default.q -V -v TMPDIR=/scratch/', url 'drmaa://-q default.q -V -v TMPDIR=/scratch/' converted, got params: galaxy.jobs DEBUG 2013-08-14 14:08:14,885 nativeSpecification: -q default.q -V -v TMPDIR= The parser has stopped at the / before scratch and ignored the rest :( I did refer to http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster and tried to define job_conf.xml as <?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <destinations default="sge_default"> <destination id="sge_default" runner="drmaa"> <param id="nativeSpecification">-q default.q -V -v TMPDIR=/scratch</param> </destination> </destinations> </job_conf> But then galaxy won't start, giving me the error of: Traceback (most recent call last): File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 37, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/app.py", line 95, in __init__ self.job_config = jobs.JobConfiguration(self) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 107, in __init__ self.__parse_job_conf_xml(tree) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 157, in __parse_job_conf_xml self.default_handler_id = self.__get_default(handlers, self.handlers.keys()) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 286, in __get_default rval = parent.get('default') AttributeError: 'NoneType' object has no attribute 'get' I am not sure what the "default" refers to, doesn't seems to be the destinations default specified as I tried changing that name but the error message remains the same. Any help so that I can pass TMPDIR=/scratch into SGE qsub would be greatly appreciated. -Tin -- I am riding to fundraise for the Cystic Fibrosis Foundation. Please consider a tax deductible donation at http://www.cff.org/LWC/dsp_DonationPage.cfm?idEvent=23815&idUser=615989 ~~~ __o Engineering is the art of making compromises. ~~~ _ <_ Science is the reverse engineering of the compromises made by nature. ~~~ (_)/(_) Medicine is the hacking of the scientific knowledge base. - A comp sci student :-) On Tue, Aug 13, 2013 at 7:28 PM, Jennifer Jackson <jen@bx.psu.edu> wrote:
Hi Tin,
I am going to move this over to the galaxy-dev@bx.psu.edu mailing list (good for local install questions) to give it better viability.
I am by no means the best person to help with this, so hopefully others will jump in, but just from looking at this I am wondering if you shouldn't be instead looking at the " job_conf.xml.sample_advanced" file. There is the default version as well, but it is probably not what you are looking for if you want to add custom logging scripts.
For universe_wsgi.ini, it seems like maybe the problem with the syntax is most likely the unknown "default_cluster_job_runner" variable and the double "=" in the line. Slashes should be fine unescaped here. But as I said, I think the other file is the place for the changes anyway, or maybe both, but using known variables and a single "=" per path.
But let's see what the developers think!
Jen Galaxy team
On 8/12/13 3:49 PM, tin h wrote:
Hello galaxy gurus,Â
I am trying to configure a local galaxy server to submit to SGE using DRMAA, it works, but I would like to specify the equivalent of "-v TMPDIR=/scratch" for qsub.
I tried updating universe_wsgi.ini with something like: Â default_cluster_job_runner = Â drmaa://-q default.q -V -v TMPDIR=/scratch/ but that doesn't work because "/" is a delimiter, and I have not been able to find way to escape it. Â Is there any tricks for this?
Much thanks in advance for your help in this matter. -Tin
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
-- Jennifer Hillman-Jacksonhttp://galaxyproject.org
-------- Original Message -------- Subject: Re: how to add settings of TMPDIR=/scratch for sge/drmaa in universe_wsgi.ini Date: Tue, 13 Aug 2013 23:21:16 -0700 From: tin h <tin6150@gmail.com> To: Jennifer Jackson <jen@bx.psu.edu> CC: Galaxy Dev <galaxy-dev@bx.psu.edu> Thank you Jennifer. I am not sure if my previous email got gabled up... Just to be sure, I specified the following in universe_wsgi.ini default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/ when i start galaxy, looking at the console log, this is what I see: galaxy.jobs <http://galaxy.jobs> DEBUG 2013-08-14 14:08:14,885 Loaded job runner 'galaxy.jobs.runners.drmaa:DRMAAJobRunner' as 'drmaa' galaxy.jobs.runners.drmaa DEBUG 2013-08-14 14:08:14,885 Converted URL 'drmaa://-q default.q -V -v TMPDIR=/scratch/' to destination runner=drmaa, params={'nativeSpecification': '-q default.q -V -v TMPDIR='} galaxy.jobs <http://galaxy.jobs> DEBUG 2013-08-14 14:08:14,885 Legacy destination with id 'drmaa://-q default.q -V -v TMPDIR=/scratch/', url 'drmaa://-q default.q -V -v TMPDIR=/scratch/' converted, got params: galaxy.jobs <http://galaxy.jobs> DEBUG 2013-08-14 14:08:14,885 nativeSpecification: -q default.q -V -v TMPDIR= The parser has stopped at the / before scratch and ignored the rest :( I did refer to http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster and tried to define job_conf.xml as <?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <destinations default="sge_default"> <destination id="sge_default" runner="drmaa"> <param id="nativeSpecification">-q default.q -V -v TMPDIR=/scratch</param> </destination> </destinations> </job_conf> But then galaxy won't start, giving me the error of: Traceback (most recent call last): File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 37, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/app.py", line 95, in __init__ self.job_config = jobs.JobConfiguration(self) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 107, in __init__ self.__parse_job_conf_xml(tree) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 157, in __parse_job_conf_xml self.default_handler_id = self.__get_default(handlers, self.handlers.keys()) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 286, in __get_default rval = parent.get('default') AttributeError: 'NoneType' object has no attribute 'get' I am not sure what the "default" refers to, doesn't seems to be the destinations default specified as I tried changing that name but the error message remains the same. Any help so that I can pass TMPDIR=/scratch into SGE qsub would be greatly appreciated. -Tin -- I am riding to fundraise for the Cystic Fibrosis Foundation. Please consider a tax deductible donation at http://www.cff.org/LWC/dsp_DonationPage.cfm?idEvent=23815&idUser=615989 ~~~ __o Engineering is the art of making compromises. ~~~ _ <_ Science is the reverse engineering of the compromises made by nature. ~~~ (_)/(_) Medicine is the hacking of the scientific knowledge base. - A comp sci student :-) On 8/13/13 7:28 PM, Jennifer Jackson wrote:
Hi Tin,
I am going to move this over to the galaxy-dev@bx.psu.edu mailing list (good for local install questions) to give it better viability.
I am by no means the best person to help with this, so hopefully others will jump in, but just from looking at this I am wondering if you shouldn't be instead looking at the " job_conf.xml.sample_advanced" file. There is the default version as well, but it is probably not what you are looking for if you want to add custom logging scripts.
For universe_wsgi.ini, it seems like maybe the problem with the syntax is most likely the unknown "default_cluster_job_runner" variable and the double "=" in the line. Slashes should be fine unescaped here. But as I said, I think the other file is the place for the changes anyway, or maybe both, but using known variables and a single "=" per path.
But let's see what the developers think!
Jen Galaxy team
On 8/12/13 3:49 PM, tin h wrote:
Hello galaxy gurus,Â
I am trying to configure a local galaxy server to submit to SGE using DRMAA, it works, but I would like to specify the equivalent of "-v TMPDIR=/scratch" for qsub.
I tried updating universe_wsgi.ini with something like: Â default_cluster_job_runner = Â drmaa://-q default.q -V -v TMPDIR=/scratch/ but that doesn't work because "/" is a delimiter, and I have not been able to find way to escape it. Â Is there any tricks for this?
Much thanks in advance for your help in this matter. -Tin
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson http://galaxyproject.org
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Jennifer Hillman-Jackson http://galaxyproject.org
Hello gurus, I made some slight progress... If I specify a handlers section into the job_conf.xml file with: <handlers> <handler id="main"/> </handlers> Then galaxy would at least start. However, it is not very functional, I can't upload files or run job, it says: An error occurred with this dataset:*Unable to run job due to a misconfiguration of the Galaxy job running system. Please contact a site administrator.* My universe_wsgi.conf is fairly stock, the [server:main] section is certainly there. The most relevant changes are: start_job_runners = drmaa default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/ If someone can point me to a job_conf.xml so that I can set the TMPDIR param correctly for qsub (sge), that would be greatly appreciated! Thanks -Tin On Wed, Aug 14, 2013 at 8:04 AM, Jennifer Jackson <jen@bx.psu.edu> wrote:
-------- Original Message -------- Subject: Re: how to add settings of TMPDIR=/scratch for sge/drmaa in universe_wsgi.ini Date: Tue, 13 Aug 2013 23:21:16 -0700 From: tin h <tin6150@gmail.com> <tin6150@gmail.com> To: Jennifer Jackson <jen@bx.psu.edu> <jen@bx.psu.edu> CC: Galaxy Dev <galaxy-dev@bx.psu.edu> <galaxy-dev@bx.psu.edu>
Thank you Jennifer.
I am not sure if my previous email got gabled up... Just to be sure, I specified the following in universe_wsgi.ini default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/
when i start galaxy, looking at the console log, this is what I see:
galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Loaded job runner 'galaxy.jobs.runners.drmaa:DRMAAJobRunner' as 'drmaa' galaxy.jobs.runners.drmaa DEBUG 2013-08-14 14:08:14,885 Converted URL 'drmaa://-q default.q -V -v TMPDIR=/scratch/' to destination runner=drmaa, params={'nativeSpecification': '-q default.q -V -v TMPDIR='} galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Legacy destination with id 'drmaa://-q default.q -V -v TMPDIR=/scratch/', url 'drmaa://-q default.q -V -v TMPDIR=/scratch/' converted, got params: galaxy.jobs DEBUG 2013-08-14 14:08:14,885 nativeSpecification: -q default.q -V -v TMPDIR=
The parser has stopped at the / before scratch and ignored the rest :(
I did refer to http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster and tried to define job_conf.xml as
<?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <destinations default="sge_default"> <destination id="sge_default" runner="drmaa"> <param id="nativeSpecification">-q default.q -V -v TMPDIR=/scratch</param> </destination> </destinations> </job_conf>
But then galaxy won't start, giving me the error of:
Traceback (most recent call last): File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 37, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/app.py", line 95, in __init__ self.job_config = jobs.JobConfiguration(self) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 107, in __init__ self.__parse_job_conf_xml(tree) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 157, in __parse_job_conf_xml self.default_handler_id = self.__get_default(handlers, self.handlers.keys()) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 286, in __get_default rval = parent.get('default') AttributeError: 'NoneType' object has no attribute 'get'
I am not sure what the "default" refers to, doesn't seems to be the destinations default specified as I tried changing that name but the error message remains the same.
Any help so that I can pass TMPDIR=/scratch into SGE qsub would be greatly appreciated.
-Tin
--
I am riding to fundraise for the Cystic Fibrosis Foundation. Please consider a tax deductible donation at http://www.cff.org/LWC/dsp_DonationPage.cfm?idEvent=23815&idUser=615989
~~~ __o Engineering is the art of making compromises. ~~~ _ <_ Science is the reverse engineering of the compromises made by nature. ~~~ (_)/(_) Medicine is the hacking of the scientific knowledge base. - A comp sci student :-)
On 8/13/13 7:28 PM, Jennifer Jackson wrote:
Hi Tin,
I am going to move this over to the galaxy-dev@bx.psu.edu mailing list (good for local install questions) to give it better viability.
I am by no means the best person to help with this, so hopefully others will jump in, but just from looking at this I am wondering if you shouldn't be instead looking at the " job_conf.xml.sample_advanced" file. There is the default version as well, but it is probably not what you are looking for if you want to add custom logging scripts.
For universe_wsgi.ini, it seems like maybe the problem with the syntax is most likely the unknown "default_cluster_job_runner" variable and the double "=" in the line. Slashes should be fine unescaped here. But as I said, I think the other file is the place for the changes anyway, or maybe both, but using known variables and a single "=" per path.
But let's see what the developers think!
Jen Galaxy team
On 8/12/13 3:49 PM, tin h wrote:
Hello galaxy gurus,Â
I am trying to configure a local galaxy server to submit to SGE using DRMAA, it works, but I would like to specify the equivalent of "-v TMPDIR=/scratch" for qsub.
I tried updating universe_wsgi.ini with something like: Â default_cluster_job_runner = Â drmaa://-q default.q -V -v TMPDIR=/scratch/ but that doesn't work because "/" is a delimiter, and I have not been able to find way to escape it. Â Is there any tricks for this?
Much thanks in advance for your help in this matter. -Tin
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
-- Jennifer Hillman-Jacksonhttp://galaxyproject.org
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Jennifer Hillman-Jacksonhttp://galaxyproject.org
On Aug 14, 2013, at 9:46 PM, tin h wrote:
Hello gurus,
I made some slight progress...
If I specify a handlers section into the job_conf.xml file with: <handlers> <handler id="main"/> </handlers>
Then galaxy would at least start. However, it is not very functional, I can't upload files or run job, it says:
An error occurred with this dataset:Unable to run job due to a misconfiguration of the Galaxy job running system. Please contact a site administrator.
My universe_wsgi.conf is fairly stock, the [server:main] section is certainly there. The most relevant changes are: start_job_runners = drmaa default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/
If someone can point me to a job_conf.xml so that I can set the TMPDIR param correctly for qsub (sge), that would be greatly appreciated!
Thanks -Tin
Hi Tin, The start_job_runners and default_cluster_job_runner parameters are ignored once you switch to the new XML configuration. The problem is most likely that the 'runner' attribute on your destination ("drmaa") does not match the 'id' attribute of the DRMAAJobRunner plugin ("sge"), so try changing one or the other so that they match. More details regarding the error should be shown in the log. --nate
On Wed, Aug 14, 2013 at 8:04 AM, Jennifer Jackson <jen@bx.psu.edu> wrote: -------- Original Message -------- Subject: Re: how to add settings of TMPDIR=/scratch for sge/drmaa in universe_wsgi.ini Date: Tue, 13 Aug 2013 23:21:16 -0700 From: tin h <tin6150@gmail.com> To: Jennifer Jackson <jen@bx.psu.edu> CC: Galaxy Dev <galaxy-dev@bx.psu.edu>
Thank you Jennifer.
I am not sure if my previous email got gabled up... Just to be sure, I specified the following in universe_wsgi.ini default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/
when i start galaxy, looking at the console log, this is what I see:
galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Loaded job runner 'galaxy.jobs.runners.drmaa:DRMAAJobRunner' as 'drmaa' galaxy.jobs.runners.drmaa DEBUG 2013-08-14 14:08:14,885 Converted URL 'drmaa://-q default.q -V -v TMPDIR=/scratch/' to destination runner=drmaa, params={'nativeSpecification': '-q default.q -V -v TMPDIR='} galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Legacy destination with id 'drmaa://-q default.q -V -v TMPDIR=/scratch/', url 'drmaa://-q default.q -V -v TMPDIR=/scratch/' converted, got params: galaxy.jobs DEBUG 2013-08-14 14:08:14,885 nativeSpecification: -q default.q -V -v TMPDIR=
The parser has stopped at the / before scratch and ignored the rest :(
I did refer to http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster and tried to define job_conf.xml as
<?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <destinations default="sge_default"> <destination id="sge_default" runner="drmaa"> <param id="nativeSpecification">-q default.q -V -v TMPDIR=/scratch</param> </destination> </destinations> </job_conf>
But then galaxy won't start, giving me the error of:
Traceback (most recent call last): File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 37, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/app.py", line 95, in __init__ self.job_config = jobs.JobConfiguration(self) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 107, in __init__ self.__parse_job_conf_xml(tree) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 157, in __parse_job_conf_xml self.default_handler_id = self.__get_default(handlers, self.handlers.keys()) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 286, in __get_default rval = parent.get('default') AttributeError: 'NoneType' object has no attribute 'get'
I am not sure what the "default" refers to, doesn't seems to be the destinations default specified as I tried changing that name but the error message remains the same.
Any help so that I can pass TMPDIR=/scratch into SGE qsub would be greatly appreciated.
-Tin
--
I am riding to fundraise for the Cystic Fibrosis Foundation. Please consider a tax deductible donation at http://www.cff.org/LWC/dsp_DonationPage.cfm?idEvent=23815&idUser=615989
~~~ __o Engineering is the art of making compromises. ~~~ _ <_ Science is the reverse engineering of the compromises made by nature. ~~~ (_)/(_) Medicine is the hacking of the scientific knowledge base. - A comp sci student :-)
On 8/13/13 7:28 PM, Jennifer Jackson wrote:
Hi Tin,
I am going to move this over to the galaxy-dev@bx.psu.edu mailing list (good for local install questions) to give it better viability.
I am by no means the best person to help with this, so hopefully others will jump in, but just from looking at this I am wondering if you shouldn't be instead looking at the " job_conf.xml.sample_advanced" file. There is the default version as well, but it is probably not what you are looking for if you want to add custom logging scripts.
For universe_wsgi.ini, it seems like maybe the problem with the syntax is most likely the unknown "default_cluster_job_runner" variable and the double "=" in the line. Slashes should be fine unescaped here. But as I said, I think the other file is the place for the changes anyway, or maybe both, but using known variables and a single "=" per path.
But let's see what the developers think!
Jen Galaxy team
On 8/12/13 3:49 PM, tin h wrote:
Hello galaxy gurus,Â
I am trying to configure a local galaxy server to submit to SGE using DRMAA, it works, but I would like to specify the equivalent of "-v TMPDIR=/scratch" for qsub.
I tried updating universe_wsgi.ini with something like: Â default_cluster_job_runner = Â drmaa://-q default.q -V -v TMPDIR=/scratch/ but that doesn't work because "/" is a delimiter, and I have not been able to find way to escape it. Â Is there any tricks for this?
Much thanks in advance for your help in this matter. -Tin
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org . Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Thanks Nate! I did think that job_conf.xml would overwrite settings in universe_wsgi.ini. I rewrote it and see that the galaxy job gets to run on the cluster. However, the TMPDIR clause defined in nativeSpecification didn't seems to be effective :( the console output "log" does not show what the final nativeSpecification was parsed as. Given I didn't see any errors, I can assume that it was taken as defined and it is SGE somehow not heeding the parameter? Any other ideas of how I may fix this? Or should I look at cluster configuration to change it without relaying on galaxy? relevant console output: galaxy.jobs DEBUG 2013-08-16 02:50:51,622 Loading job configuration from ./job_conf.xml galaxy.jobs DEBUG 2013-08-16 02:50:51,622 Read definition for handler 'main' galaxy.jobs INFO 2013-08-16 02:50:51,623 Setting <handlers> default to child with id 'main' galaxy.jobs DEBUG 2013-08-16 02:50:51,623 <destinations> default set to child with id or tag 'sge' galaxy.jobs DEBUG 2013-08-16 02:50:51,623 Done loading job configuration ... galaxy.jobs.manager DEBUG 2013-08-16 02:51:01,190 Starting job handler galaxy.jobs.runners DEBUG 2013-08-16 02:51:01,192 Starting 4 LocalRunner workers galaxy.jobs DEBUG 2013-08-16 02:51:01,194 Loaded job runner 'galaxy.jobs.runners.local:LocalJobRunner' as 'local' galaxy.jobs.runners DEBUG 2013-08-16 02:51:01,242 Starting 4 DRMAARunner workers galaxy.jobs DEBUG 2013-08-16 02:51:01,243 Loaded job runner 'galaxy.jobs.runners.drmaa:DRMAAJobRunner' as 'drmaa' galaxy.jobs.handler DEBUG 2013-08-16 02:51:01,243 Loaded job runners plugins: drmaa:local galaxy.jobs.handler INFO 2013-08-16 02:51:01,244 job handler stop queue started galaxy.jobs.handler INFO 2013-08-16 02:51:01,253 job handler queue started ... galaxy.jobs DEBUG 2013-08-16 02:51:22,316 (218) Working directory for job is: /usr/prog/galaxy/galaxy-dist/database/job_working_directory/000/218 galaxy.jobs.handler DEBUG 2013-08-16 02:51:22,320 (218) Dispatching to drmaa runner galaxy.jobs DEBUG 2013-08-16 02:51:22,399 (218) Persisting job destination (destination id: sge) galaxy.jobs.handler INFO 2013-08-16 02:51:22,415 (218) Job dispatched galaxy.tools DEBUG 2013-08-16 02:51:22,648 Building dependency shell command for dependency 'bwa' galaxy.jobs.runners.drmaa DEBUG 2013-08-16 02:51:22,764 (218) submitting file /usr/prog/galaxy/galaxy-dist/database/job_working_directory/000/218/galaxy_218.sh galaxy.jobs.runners.drmaa DEBUG 2013-08-16 02:51:22,765 (218) command is: PACKAGE_BASE=/usr/prog/galaxy/tool_dependency/bwa/0.5.9/devteam/bwa_wrappers/150b3fe44caa; export PACKAGE_BASE; . /usr/prog/galaxy/tool_dependency/bwa/0.5.9/devteam/bwa_wrappers/150b3fe44caa/env.sh; python /usr/prog/galaxy/shed_tools/ toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/150b3fe44caa/bwa_wrappers/bwa_wrapper.py --threads="4" --fileSource="indexed" --ref="/db/ngs-bioinfo/genomes/human/hg19/Homo_sapiens_assembly19.fasta" --do_not_build_index --input1="/usr/prog/galaxy/galaxy-dist/database/files/000/dataset_268.dat" --output="/usr/prog/galaxy/galaxy-dist/database/files/000/dataset_287.dat" --genAlignType="single" --params="full" --maxEditDist="0" --fracMissingAligns="0.04" --maxGapOpens="0" --maxGapExtens="-1" --disallowLongDel="16" --disallowIndel="5" --seed="-1" --maxEditDistSeed="2" --mismatchPenalty="3" --gapOpenPenalty="11" --gapExtensPenalty="4" --suboptAlign="" --noIterSearch="false" --outputTopN="3" --outputTopNDisc="10" --maxInsertSize="500" --maxOccurPairing="100000" --suppressHeader="true" galaxy.jobs.runners.drmaa INFO 2013-08-16 02:51:22,773 (218) queued as 37234 galaxy.jobs DEBUG 2013-08-16 02:51:22,805 (218) Persisting job destination (destination id: sge) galaxy.jobs.runners.drmaa DEBUG 2013-08-16 02:51:23,273 (218/37234) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-08-16 02:51:24,301 (218/37234) state change: job is running the updated job_conf.xml :: <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <handlers> <handler id="main"/> </handlers> <destinations default="sge"> <destination id="sge" runner="drmaa"> <!-- runner need to refer to a plugin id --> <param id="nativeSpecification">-q default.q -V -v TMPDIR=/scratch</param> </destination> <destination id="local" runner="local"/> </destinations> </job_conf> Much thanks again, -Tin -- I am riding to fundraise for the Cystic Fibrosis Foundation. Please consider a tax deductible donation at http://www.cff.org/LWC/dsp_DonationPage.cfm?idEvent=23815&idUser=615989 ~~~ __o Engineering is the art of making compromises. ~~~ _ <_ Science is the reverse engineering of the compromises made by nature. ~~~ (_)/(_) Medicine is the hacking of the scientific knowledge base. - A comp sci student :-) On Thu, Aug 15, 2013 at 7:19 AM, Nate Coraor <nate@bx.psu.edu> wrote:
On Aug 14, 2013, at 9:46 PM, tin h wrote:
Hello gurus,
I made some slight progress...
If I specify a handlers section into the job_conf.xml file with: <handlers> <handler id="main"/> </handlers>
Then galaxy would at least start. However, it is not very functional, I
can't upload files or run job, it says:
An error occurred with this dataset:Unable to run job due to a
misconfiguration of the Galaxy job running system. Please contact a site administrator.
My universe_wsgi.conf is fairly stock, the [server:main] section is
certainly there. The most relevant changes are:
start_job_runners = drmaa default_cluster_job_runner = drmaa://-q default.q -V -v
TMPDIR=/scratch/
If someone can point me to a job_conf.xml so that I can set the TMPDIR
param correctly for qsub (sge), that would be greatly appreciated!
Thanks -Tin
Hi Tin,
The start_job_runners and default_cluster_job_runner parameters are ignored once you switch to the new XML configuration. The problem is most likely that the 'runner' attribute on your destination ("drmaa") does not match the 'id' attribute of the DRMAAJobRunner plugin ("sge"), so try changing one or the other so that they match. More details regarding the error should be shown in the log.
--nate
On Wed, Aug 14, 2013 at 8:04 AM, Jennifer Jackson <jen@bx.psu.edu>
-------- Original Message -------- Subject: Re: how to add settings of TMPDIR=/scratch for sge/drmaa in universe_wsgi.ini Date: Tue, 13 Aug 2013 23:21:16 -0700 From: tin h <tin6150@gmail.com> To: Jennifer Jackson <jen@bx.psu.edu> CC: Galaxy Dev <galaxy-dev@bx.psu.edu>
Thank you Jennifer.
I am not sure if my previous email got gabled up... Just to be sure, I specified the following in universe_wsgi.ini default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/
when i start galaxy, looking at the console log, this is what I see:
galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Loaded job runner 'galaxy.jobs.runners.drmaa:DRMAAJobRunner' as 'drmaa' galaxy.jobs.runners.drmaa DEBUG 2013-08-14 14:08:14,885 Converted URL 'drmaa://-q default.q -V -v TMPDIR=/scratch/' to destination runner=drmaa,
galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Legacy destination with id 'drmaa://-q default.q -V -v TMPDIR=/scratch/', url 'drmaa://-q default.q -V -v TMPDIR=/scratch/' converted, got params: galaxy.jobs DEBUG 2013-08-14 14:08:14,885 nativeSpecification: -q default.q -V -v TMPDIR=
The parser has stopped at the / before scratch and ignored the rest :(
I did refer to http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster and tried to define job_conf.xml as
<?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <destinations default="sge_default"> <destination id="sge_default" runner="drmaa"> <param id="nativeSpecification">-q default.q -V -v TMPDIR=/scratch</param> </destination> </destinations> </job_conf>
But then galaxy won't start, giving me the error of:
Traceback (most recent call last): File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 37, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/app.py", line 95, in __init__ self.job_config = jobs.JobConfiguration(self) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 107, in __init__ self.__parse_job_conf_xml(tree) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 157, in __parse_job_conf_xml self.default_handler_id = self.__get_default(handlers, self.handlers.keys()) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 286, in __get_default rval = parent.get('default') AttributeError: 'NoneType' object has no attribute 'get'
I am not sure what the "default" refers to, doesn't seems to be the destinations default specified as I tried changing that name but the error message remains the same.
Any help so that I can pass TMPDIR=/scratch into SGE qsub would be greatly appreciated.
-Tin
--
I am riding to fundraise for the Cystic Fibrosis Foundation. Please consider a tax deductible donation at http://www.cff.org/LWC/dsp_DonationPage.cfm?idEvent=23815&idUser=615989
~~~ __o Engineering is the art of making compromises. ~~~ _ <_ Science is the reverse engineering of the compromises made by nature. ~~~ (_)/(_) Medicine is the hacking of the scientific knowledge base. - A comp sci student :-)
On 8/13/13 7:28 PM, Jennifer Jackson wrote:
Hi Tin,
I am going to move this over to the galaxy-dev@bx.psu.edu mailing list (good for local install questions) to give it better viability.
I am by no means the best person to help with this, so hopefully others will jump in, but just from looking at this I am wondering if you shouldn't be instead looking at the " job_conf.xml.sample_advanced" file. There is
wrote: params={'nativeSpecification': '-q default.q -V -v TMPDIR='} the default version as well, but it is probably not what you are looking for if you want to add custom logging scripts.
For universe_wsgi.ini, it seems like maybe the problem with the syntax
is most likely the unknown "default_cluster_job_runner" variable and the double "=" in the line. Slashes should be fine unescaped here. But as I said, I think the other file is the place for the changes anyway, or maybe both, but using known variables and a single "=" per path.
But let's see what the developers think!
Jen Galaxy team
On 8/12/13 3:49 PM, tin h wrote:
Hello galaxy gurus,Â
I am trying to configure a local galaxy server to submit to SGE using
DRMAA,Â
it works, but I would like to specify the equivalent of "-v TMPDIR=/scratch" for qsub.
I tried updating universe_wsgi.ini with something like: Â default_cluster_job_runner = Â drmaa://-q default.q -V -v TMPDIR=/scratch/ but that doesn't work because "/" is a delimiter, and I have not been able to find way to escape it. Â Is there any tricks for this?
Much thanks in advance for your help in this matter. -Tin
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org . Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Aug 18, 2013, at 1:58 AM, tin h wrote:
Thanks Nate!
I did think that job_conf.xml would overwrite settings in universe_wsgi.ini. I rewrote it and see that the galaxy job gets to run on the cluster. However, the TMPDIR clause defined in nativeSpecification didn't seems to be effective :(
the console output "log" does not show what the final nativeSpecification was parsed as. Given I didn't see any errors, I can assume that it was taken as defined and it is SGE somehow not heeding the parameter?
Any other ideas of how I may fix this? Or should I look at cluster configuration to change it without relaying on galaxy?
You might want to have a dummy tool output `set` on the cluster. Chances are that SGE is overriding $TEMP or $TMPDIR at job runtime. You may need to set this in ~/.sge_request or use Galaxy's environment_setup_file option. --nate
relevant console output:
galaxy.jobs DEBUG 2013-08-16 02:50:51,622 Loading job configuration from ./job_conf.xml galaxy.jobs DEBUG 2013-08-16 02:50:51,622 Read definition for handler 'main' galaxy.jobs INFO 2013-08-16 02:50:51,623 Setting <handlers> default to child with id 'main' galaxy.jobs DEBUG 2013-08-16 02:50:51,623 <destinations> default set to child with id or tag 'sge' galaxy.jobs DEBUG 2013-08-16 02:50:51,623 Done loading job configuration ... galaxy.jobs.manager DEBUG 2013-08-16 02:51:01,190 Starting job handler galaxy.jobs.runners DEBUG 2013-08-16 02:51:01,192 Starting 4 LocalRunner workers galaxy.jobs DEBUG 2013-08-16 02:51:01,194 Loaded job runner 'galaxy.jobs.runners.local:LocalJobRunner' as 'local' galaxy.jobs.runners DEBUG 2013-08-16 02:51:01,242 Starting 4 DRMAARunner workers galaxy.jobs DEBUG 2013-08-16 02:51:01,243 Loaded job runner 'galaxy.jobs.runners.drmaa:DRMAAJobRunner' as 'drmaa' galaxy.jobs.handler DEBUG 2013-08-16 02:51:01,243 Loaded job runners plugins: drmaa:local galaxy.jobs.handler INFO 2013-08-16 02:51:01,244 job handler stop queue started galaxy.jobs.handler INFO 2013-08-16 02:51:01,253 job handler queue started ... galaxy.jobs DEBUG 2013-08-16 02:51:22,316 (218) Working directory for job is: /usr/prog/galaxy/galaxy-dist/database/job_working_directory/000/218 galaxy.jobs.handler DEBUG 2013-08-16 02:51:22,320 (218) Dispatching to drmaa runner galaxy.jobs DEBUG 2013-08-16 02:51:22,399 (218) Persisting job destination (destination id: sge) galaxy.jobs.handler INFO 2013-08-16 02:51:22,415 (218) Job dispatched galaxy.tools DEBUG 2013-08-16 02:51:22,648 Building dependency shell command for dependency 'bwa' galaxy.jobs.runners.drmaa DEBUG 2013-08-16 02:51:22,764 (218) submitting file /usr/prog/galaxy/galaxy-dist/database/job_working_directory/000/218/galaxy_218.sh galaxy.jobs.runners.drmaa DEBUG 2013-08-16 02:51:22,765 (218) command is: PACKAGE_BASE=/usr/prog/galaxy/tool_dependency/bwa/0.5.9/devteam/bwa_wrappers/150b3fe44caa; export PACKAGE_BASE; . /usr/prog/galaxy/tool_dependency/bwa/0.5.9/devteam/bwa_wrappers/150b3fe44caa/env.sh; python /usr/prog/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/150b3fe44caa/bwa_wrappers/bwa_wrapper.py --threads="4" --fileSource="indexed" --ref="/db/ngs-bioinfo/genomes/human/hg19/Homo_sapiens_assembly19.fasta" --do_not_build_index --input1="/usr/prog/galaxy/galaxy-dist/database/files/000/dataset_268.dat" --output="/usr/prog/galaxy/galaxy-dist/database/files/000/dataset_287.dat" --genAlignType="single" --params="full" --maxEditDist="0" --fracMissingAligns="0.04" --maxGapOpens="0" --maxGapExtens="-1" --disallowLongDel="16" --disallowIndel="5" --seed="-1" --maxEditDistSeed="2" --mismatchPenalty="3" --gapOpenPenalty="11" --gapExtensPenalty="4" --suboptAlign="" --noIterSearch="false" --outputTopN="3" --outputTopNDisc="10" --maxInsertSize="500" --maxOccurPairing="100000" --suppressHeader="true" galaxy.jobs.runners.drmaa INFO 2013-08-16 02:51:22,773 (218) queued as 37234 galaxy.jobs DEBUG 2013-08-16 02:51:22,805 (218) Persisting job destination (destination id: sge) galaxy.jobs.runners.drmaa DEBUG 2013-08-16 02:51:23,273 (218/37234) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-08-16 02:51:24,301 (218/37234) state change: job is running
the updated job_conf.xml ::
<job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <handlers> <handler id="main"/> </handlers> <destinations default="sge"> <destination id="sge" runner="drmaa"> <!-- runner need to refer to a plugin id --> <param id="nativeSpecification">-q default.q -V -v TMPDIR=/scratch</param> </destination> <destination id="local" runner="local"/> </destinations> </job_conf>
Much thanks again, -Tin
--
I am riding to fundraise for the Cystic Fibrosis Foundation. Please consider a tax deductible donation at http://www.cff.org/LWC/dsp_DonationPage.cfm?idEvent=23815&idUser=615989
~~~ __o Engineering is the art of making compromises. ~~~ _ <_ Science is the reverse engineering of the compromises made by nature. ~~~ (_)/(_) Medicine is the hacking of the scientific knowledge base. - A comp sci student :-)
On Thu, Aug 15, 2013 at 7:19 AM, Nate Coraor <nate@bx.psu.edu> wrote: On Aug 14, 2013, at 9:46 PM, tin h wrote:
Hello gurus,
I made some slight progress...
If I specify a handlers section into the job_conf.xml file with: <handlers> <handler id="main"/> </handlers>
Then galaxy would at least start. However, it is not very functional, I can't upload files or run job, it says:
An error occurred with this dataset:Unable to run job due to a misconfiguration of the Galaxy job running system. Please contact a site administrator.
My universe_wsgi.conf is fairly stock, the [server:main] section is certainly there. The most relevant changes are: start_job_runners = drmaa default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/
If someone can point me to a job_conf.xml so that I can set the TMPDIR param correctly for qsub (sge), that would be greatly appreciated!
Thanks -Tin
Hi Tin,
The start_job_runners and default_cluster_job_runner parameters are ignored once you switch to the new XML configuration. The problem is most likely that the 'runner' attribute on your destination ("drmaa") does not match the 'id' attribute of the DRMAAJobRunner plugin ("sge"), so try changing one or the other so that they match. More details regarding the error should be shown in the log.
--nate
On Wed, Aug 14, 2013 at 8:04 AM, Jennifer Jackson <jen@bx.psu.edu> wrote: -------- Original Message -------- Subject: Re: how to add settings of TMPDIR=/scratch for sge/drmaa in universe_wsgi.ini Date: Tue, 13 Aug 2013 23:21:16 -0700 From: tin h <tin6150@gmail.com> To: Jennifer Jackson <jen@bx.psu.edu> CC: Galaxy Dev <galaxy-dev@bx.psu.edu>
Thank you Jennifer.
I am not sure if my previous email got gabled up... Just to be sure, I specified the following in universe_wsgi.ini default_cluster_job_runner = drmaa://-q default.q -V -v TMPDIR=/scratch/
when i start galaxy, looking at the console log, this is what I see:
galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Loaded job runner 'galaxy.jobs.runners.drmaa:DRMAAJobRunner' as 'drmaa' galaxy.jobs.runners.drmaa DEBUG 2013-08-14 14:08:14,885 Converted URL 'drmaa://-q default.q -V -v TMPDIR=/scratch/' to destination runner=drmaa, params={'nativeSpecification': '-q default.q -V -v TMPDIR='} galaxy.jobs DEBUG 2013-08-14 14:08:14,885 Legacy destination with id 'drmaa://-q default.q -V -v TMPDIR=/scratch/', url 'drmaa://-q default.q -V -v TMPDIR=/scratch/' converted, got params: galaxy.jobs DEBUG 2013-08-14 14:08:14,885 nativeSpecification: -q default.q -V -v TMPDIR=
The parser has stopped at the / before scratch and ignored the rest :(
I did refer to http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster and tried to define job_conf.xml as
<?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <destinations default="sge_default"> <destination id="sge_default" runner="drmaa"> <param id="nativeSpecification">-q default.q -V -v TMPDIR=/scratch</param> </destination> </destinations> </job_conf>
But then galaxy won't start, giving me the error of:
Traceback (most recent call last): File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 37, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/app.py", line 95, in __init__ self.job_config = jobs.JobConfiguration(self) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 107, in __init__ self.__parse_job_conf_xml(tree) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 157, in __parse_job_conf_xml self.default_handler_id = self.__get_default(handlers, self.handlers.keys()) File "/usr/prog/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 286, in __get_default rval = parent.get('default') AttributeError: 'NoneType' object has no attribute 'get'
I am not sure what the "default" refers to, doesn't seems to be the destinations default specified as I tried changing that name but the error message remains the same.
Any help so that I can pass TMPDIR=/scratch into SGE qsub would be greatly appreciated.
-Tin
--
I am riding to fundraise for the Cystic Fibrosis Foundation. Please consider a tax deductible donation at http://www.cff.org/LWC/dsp_DonationPage.cfm?idEvent=23815&idUser=615989
~~~ __o Engineering is the art of making compromises. ~~~ _ <_ Science is the reverse engineering of the compromises made by nature. ~~~ (_)/(_) Medicine is the hacking of the scientific knowledge base. - A comp sci student :-)
On 8/13/13 7:28 PM, Jennifer Jackson wrote:
Hi Tin,
I am going to move this over to the galaxy-dev@bx.psu.edu mailing list (good for local install questions) to give it better viability.
I am by no means the best person to help with this, so hopefully others will jump in, but just from looking at this I am wondering if you shouldn't be instead looking at the " job_conf.xml.sample_advanced" file. There is the default version as well, but it is probably not what you are looking for if you want to add custom logging scripts.
For universe_wsgi.ini, it seems like maybe the problem with the syntax is most likely the unknown "default_cluster_job_runner" variable and the double "=" in the line. Slashes should be fine unescaped here. But as I said, I think the other file is the place for the changes anyway, or maybe both, but using known variables and a single "=" per path.
But let's see what the developers think!
Jen Galaxy team
On 8/12/13 3:49 PM, tin h wrote:
Hello galaxy gurus,Â
I am trying to configure a local galaxy server to submit to SGE using DRMAA, it works, but I would like to specify the equivalent of "-v TMPDIR=/scratch" for qsub.
I tried updating universe_wsgi.ini with something like: Â default_cluster_job_runner = Â drmaa://-q default.q -V -v TMPDIR=/scratch/ but that doesn't work because "/" is a delimiter, and I have not been able to find way to escape it. Â Is there any tricks for this?
Much thanks in advance for your help in this matter. -Tin
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org . Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Thanks again Nate. You are right that SGE is overwriting TMPDIR. When the job runs, TMPDIR is actually set to a job specific dir like /tmp/3579.1.default.q It seems that I will have to change the SGE queue settings so that TMPDIR only set the "prefix" to /scratch that I want... but this would affect all users of the queue... We will probably have to live with that. Thanks for all your help! -Tin You might want to have a dummy tool output `set` on the cluster. Chances
are that SGE is overriding $TEMP or $TMPDIR at job runtime. You may need to set this in ~/.sge_request or use Galaxy's environment_setup_file option.
--nate
participants (3)
-
Jennifer Jackson -
Nate Coraor -
tin h