Hi Brian,
$PYTHONPATH was previously only modified when a tool's command line started with `python`. This somewhat arbitrary change was modified in ee1a2b8 to allow the external set_metadata process to run without requiring an absolute path to a script on disk.
However, the job script template should only prepend to $PYTHONPATH if it already exists, it should not fully overwrite it. That said, it looks like the problem may be with a conflict between Galaxy's packaged pkg_resources and the one in your virtualenv.
This may be fixed by appending Galaxy's lib directory rather than prepending it. We're working on removing the need for passing Galaxy's lib directory to tools entirely.
--nate
On Tue, May 12, 2015 at 11:57 PM, Brian Claywell bclaywel@fredhutch.org wrote:
After upgrading from the October 2014 release to 15.03 we started experiencing job failures for one of our custom tools, which I eventually tracked down to PYTHONPATH now being set in the job environment to $GALAXY_HOME/lib. The dependencies for this tool are installed in a virtualenv that is activated by the tool dependency package's env.sh. It looks like one of the scripts in this virtualenv attempts to use setuptools to find its dependencies, but because of the PYTHONPATH setting ends up in Galaxy's $GALAXY_HOME/lib/pkg_resources.py:
Traceback (most recent call last): File "/mnt/scdata/scdata_03/galaxy/containers/galaxy-qa/export/tool_deps/yapp_env/new/venv/bin/sequencing_quality_report.py", line 8, in <module> from pkg_resources import load_entry_point File "/mnt/scdata/scdata_03/galaxy/containers/galaxy-qa/stable/lib/pkg_resources.py", line 2603, in <module> working_set.require(__requires__) File "/mnt/scdata/scdata_03/galaxy/containers/galaxy-qa/stable/lib/pkg_resources.py", line 666, in require needed = self.resolve(parse_requirements(requirements)) File "/mnt/scdata/scdata_03/galaxy/containers/galaxy-qa/stable/lib/pkg_resources.py", line 565, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: pytz>=2011kI worked around this issue by prepending the virtualenv's lib/python2.7/site-packages directory to PYTHONPATH in the tool dependency package's env.sh, but I'm a bit confused why this is now necessary. Can anyone shed some light on what changed?
Cheers,
Brian
-- Brian Claywell | programmer/analyst Matsen Group | http://matsen.fredhutch.org http://matsen.fhcrc.org/ Fred Hutchinson Cancer Research Center
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/