On Mon, Jan 25, 2016 at 11:33 AM, Peter Cock p.j.a.cock@googlemail.com wrote:
Hello all,
We're currently looking at changing our Galaxy setup to link user accounts with Linux user accounts for better cluster integration (running jobs as the actual user on SGE). As part of this, we've tried setting up a fresh installation on a new VM which has thrown up some issues.
[snip]
We eventually got Galaxy to talk to SGE and submit jobs successfully as the individual user's Linux account, with external_chown_script.py being called to handle ownership of the files.
One small stumbling block was missing the cluster_files_directory directory. https://github.com/galaxyproject/galaxy/blob/dev/config/galaxy.ini.sample says:
# If using a cluster, Galaxy will write job scripts and stdout/stderr to this # directory. #cluster_files_directory = database/pbs
According the the current code, this is still the default value being used:
https://github.com/galaxyproject/galaxy/blob/274af50792aaaf367520c22d7c13e22...
However, Galaxy itself did not seem to create the cluster_files_directory directory - and there doesn't seem to be a ready-made pbs folder in the git repository. Should that happen automatically?
At one stage, run.sh would create database/files and database/tmp - I'm note sure what that happens now:
https://github.com/galaxyproject/galaxy/commit/4d720d63c631862778e54bd68c4e3... https://github.com/galaxyproject/galaxy/commit/7560751dc87492471880c82b00857...
Peter
Peter -
Nate and I are in agreement that the goal is to eliminate that directory, so I don't want to put effort into automating its creation.
https://github.com/galaxyproject/galaxy/issues/1576
If you wish to open a PR that does this though, I would be happy to merge this. The best place to ensure this directory exists is in the drmaa runner where it is being used.
-John
On Mon, Jan 25, 2016 at 4:20 PM, Peter Cock p.j.a.cock@googlemail.com wrote:
On Mon, Jan 25, 2016 at 11:33 AM, Peter Cock p.j.a.cock@googlemail.com wrote:
Hello all,
We're currently looking at changing our Galaxy setup to link user accounts with Linux user accounts for better cluster integration (running jobs as the actual user on SGE). As part of this, we've tried setting up a fresh installation on a new VM which has thrown up some issues.
[snip]
We eventually got Galaxy to talk to SGE and submit jobs successfully as the individual user's Linux account, with external_chown_script.py being called to handle ownership of the files.
One small stumbling block was missing the cluster_files_directory directory. https://github.com/galaxyproject/galaxy/blob/dev/config/galaxy.ini.sample says:
# If using a cluster, Galaxy will write job scripts and stdout/stderr to this # directory. #cluster_files_directory = database/pbs
According the the current code, this is still the default value being used:
https://github.com/galaxyproject/galaxy/blob/274af50792aaaf367520c22d7c13e22...
However, Galaxy itself did not seem to create the cluster_files_directory directory - and there doesn't seem to be a ready-made pbs folder in the git repository. Should that happen automatically?
At one stage, run.sh would create database/files and database/tmp - I'm note sure what that happens now:
https://github.com/galaxyproject/galaxy/commit/4d720d63c631862778e54bd68c4e3... https://github.com/galaxyproject/galaxy/commit/7560751dc87492471880c82b00857...
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Thanks John,
The long term vision you outline makes a lot of sense; I've just added a note to the wiki to try to help anyone else running into this:
https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster?action=diff&...
Peter
On Mon, Jan 25, 2016 at 5:27 PM, John Chilton jmchilton@gmail.com wrote:
Peter -
Nate and I are in agreement that the goal is to eliminate that directory, so I don't want to put effort into automating its creation.
https://github.com/galaxyproject/galaxy/issues/1576
If you wish to open a PR that does this though, I would be happy to merge this. The best place to ensure this directory exists is in the drmaa runner where it is being used.
-John
On Mon, Jan 25, 2016 at 4:20 PM, Peter Cock p.j.a.cock@googlemail.com wrote:
On Mon, Jan 25, 2016 at 11:33 AM, Peter Cock p.j.a.cock@googlemail.com wrote:
Hello all,
We're currently looking at changing our Galaxy setup to link user accounts with Linux user accounts for better cluster integration (running jobs as the actual user on SGE). As part of this, we've tried setting up a fresh installation on a new VM which has thrown up some issues.
[snip]
We eventually got Galaxy to talk to SGE and submit jobs successfully as the individual user's Linux account, with external_chown_script.py being called to handle ownership of the files.
One small stumbling block was missing the cluster_files_directory directory. https://github.com/galaxyproject/galaxy/blob/dev/config/galaxy.ini.sample says:
# If using a cluster, Galaxy will write job scripts and stdout/stderr to this # directory. #cluster_files_directory = database/pbs
According the the current code, this is still the default value being used:
https://github.com/galaxyproject/galaxy/blob/274af50792aaaf367520c22d7c13e22...
However, Galaxy itself did not seem to create the cluster_files_directory directory - and there doesn't seem to be a ready-made pbs folder in the git repository. Should that happen automatically?
At one stage, run.sh would create database/files and database/tmp - I'm note sure what that happens now:
https://github.com/galaxyproject/galaxy/commit/4d720d63c631862778e54bd68c4e3... https://github.com/galaxyproject/galaxy/commit/7560751dc87492471880c82b00857...
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
galaxy-dev@lists.galaxyproject.org
-
John Chilton -
Peter Cock