On Mon, Feb 22, 2016 at 9:02 PM, Nate Coraor firstname.lastname@example.org wrote:
On Mon, Feb 22, 2016 at 5:54 AM, Peter Cock email@example.com wrote:
Last week at the Galaxy Admins video hangout Nate gave an overview of the way Python dependency handling is changing from using egg files to using wheel files instead - slide links at:
Recently we've been trying to setup a replacement Galaxy instance where jobs are submitted to our SGE cluster as the linux user requesting the job. This has been a bumpy road, ...
Clearly with Galaxy v16.01 the eggs and $PYTHON_EGG_CACHE will go away. What happens with wheels, Python virtual environments, and running jobs as the real user?
Unlike eggs, which can be installed in their package format (as a single .egg file) and used in place, wheels are unpacked and their contents reside in the virtualenv's site-packages just as if you had `python setup.py install`ed them, there's no cache directory. The "real user" should be able to use Galaxy's virtualenv as long as the appropriate read/execute permissions are set on a common group or other.
Galaxy's virtualenv is passed down to the jobs when possible:
If you need to create a separate virtualenv for jobs to use, you can do this with an <env> tag on the destination:
There are instructions in that document on how to create your own Galaxy virtualenv:
I have not used the "real user" stuff in a very long time so I haven't tested this, but I can't think of any major roadblocks that should prevent it from working.
Thanks Nate, this sounds promising - fingers crossed :)
Maybe we should test the pre-release branch for v16.10 out?