On Mon, Feb 22, 2016 at 9:02 PM, Nate Coraor nate@bx.psu.edu wrote:
On Mon, Feb 22, 2016 at 5:54 AM, Peter Cock p.j.a.cock@googlemail.com wrote:
Hi all,
Last week at the Galaxy Admins video hangout Nate gave an overview of the way Python dependency handling is changing from using egg files to using wheel files instead - slide links at:
...
Recently we've been trying to setup a replacement Galaxy instance where jobs are submitted to our SGE cluster as the linux user requesting the job. This has been a bumpy road, ...
...
Clearly with Galaxy v16.01 the eggs and $PYTHON_EGG_CACHE will go away. What happens with wheels, Python virtual environments, and running jobs as the real user?
Hi Peter,
Unlike eggs, which can be installed in their package format (as a single .egg file) and used in place, wheels are unpacked and their contents reside in the virtualenv's site-packages just as if you had `python setup.py install`ed them, there's no cache directory. The "real user" should be able to use Galaxy's virtualenv as long as the appropriate read/execute permissions are set on a common group or other.
Galaxy's virtualenv is passed down to the jobs when possible:
https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/jobs/runners/uti...
If you need to create a separate virtualenv for jobs to use, you can do this with an <env> tag on the destination:
https://docs.galaxyproject.org/en/release_16.01/admin/framework_dependencies...
There are instructions in that document on how to create your own Galaxy virtualenv:
https://docs.galaxyproject.org/en/release_16.01/admin/framework_dependencies...
I have not used the "real user" stuff in a very long time so I haven't tested this, but I can't think of any major roadblocks that should prevent it from working.
--nate
Thanks Nate, this sounds promising - fingers crossed :)
Maybe we should test the pre-release branch for v16.10 out?
Peter