On Tue, Feb 23, 2016 at 4:04 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:

On Mon, Feb 22, 2016 at 9:02 PM, Nate Coraor <nate@bx.psu.edu> wrote:
> On Mon, Feb 22, 2016 at 5:54 AM, Peter Cock <p.j.a.cock@googlemail.com>
> wrote:
>>
>> Hi all,
>>
>> Last week at the Galaxy Admins video hangout Nate gave an
>> overview of the way Python dependency handling is changing
>> from using egg files to using wheel files instead - slide links at:
>>
>> ...
>>
>> Recently we've been trying to setup a replacement Galaxy instance
>> where jobs are submitted to our SGE cluster as the linux user
>> requesting the job. This has been a bumpy road, ...
>>
>> ...
>>
>> Clearly with Galaxy v16.01 the eggs and $PYTHON_EGG_CACHE
>> will go away. What happens with wheels, Python virtual environments,
>> and running jobs as the real user?
>
>
> Hi Peter,
>
> Unlike eggs, which can be installed in their package format (as a single
> .egg file) and used in place, wheels are unpacked and their contents reside
> in the virtualenv's site-packages just as if you had `python setup.py
> install`ed them, there's no cache directory. The "real user" should be able
> to use Galaxy's virtualenv as long as the appropriate read/execute
> permissions are set on a common group or other.
>
> Galaxy's virtualenv is passed down to the jobs when possible:
>
> https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/jobs/runners/util/job_script/DEFAULT_JOB_FILE_TEMPLATE.sh#L17
>
> If you need to create a separate virtualenv for jobs to use, you can do this
> with an <env> tag on the destination:
>
>
> https://docs.galaxyproject.org/en/release_16.01/admin/framework_dependencies.html#galaxy-job-handlers
>
> There are instructions in that document on how to create your own Galaxy
> virtualenv:
>
>
> https://docs.galaxyproject.org/en/release_16.01/admin/framework_dependencies.html#managing-dependencies-manually
>
> I have not used the "real user" stuff in a very long time so I haven't
> tested this, but I can't think of any major roadblocks that should prevent
> it from working.
>
> --nate

Thanks Nate, this sounds promising - fingers crossed :)

Maybe we should test the pre-release branch for v16.10 out?

Yes, I don't expect much to change before the release, which, fingers crossed, will be tomorrow or Wednesday.

Peter