PATH set in tool env.sh leaking through to set_metadata.py
Hi, I am not sure whether I am diagnosing this problem approximately right or not, but it seems that with the latest release of Galaxy (stable branch, latest_2015.01.13) enviroment variables set in a tool-dependency's env.sh file and sourced before a job run are now leaking through to ./scripts/set_metadata.py called after the tool run finishes. In my specific case, the sourced env.sh adds a python 3 virtualenv directory to PATH, which then causes set_metadata.py to fail with: File "./scripts/set_metadata.py", line 130 except Exception, e: ^ SyntaxError: invalid syntax because that (amazingly enough :)) seems to be the first line not compatible with Python 3. I have not seen this error before (I believe not with latest_2014.10.06) so I guess my question is: has the way that set_metadata.py is called been changed recently and would you consider the current behavior a bug or is this expected and ok ? Best, Wolfgang
On 01/20/2015 06:20 PM, Wolfgang Maier wrote:
I have not seen this error before (I believe not with latest_2014.10.06)
update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again. The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately. Wolfgang
Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)? -John On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> wrote:
On 01/20/2015 06:20 PM, Wolfgang Maier wrote:
I have not seen this error before (I believe not with latest_2014.10.06)
update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again.
The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately.
Wolfgang
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Yes, this is the local job runner on a single machine. On 01/20/2015 06:52 PM, John Chilton wrote:
Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)?
-John
On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> wrote:
On 01/20/2015 06:20 PM, Wolfgang Maier wrote:
I have not seen this error before (I believe not with latest_2014.10.06)
update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again.
The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately.
Wolfgang
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
So the tool dependency stuff didn't change - but the local job runner now behaves like the other job runners (DRMAA, PBS, etc...) so you have uncovered a bug for all of them I think. Is this a tool shed tool or a locally created dependency? I need to think about how to solve this more generally but if you want a quick work around - I think it would work to just replace "python" in $GALAXY_ROOT/set_metadata.sh with a hard-coded path to Galaxy's python or perhaps just "python2". Sorry for the inconvenience and thanks for bringing this to our attention. -John On Tue, Jan 20, 2015 at 12:52 PM, John Chilton <jmchilton@gmail.com> wrote:
Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)?
-John
On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> wrote:
On 01/20/2015 06:20 PM, Wolfgang Maier wrote:
I have not seen this error before (I believe not with latest_2014.10.06)
update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again.
The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately.
Wolfgang
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On 21.01.2015 16:11, John Chilton wrote:
So the tool dependency stuff didn't change - but the local job runner now behaves like the other job runners (DRMAA, PBS, etc...) so you have uncovered a bug for all of them I think. Is this a tool shed tool or a locally created dependency?
This is a tool shed tool (package_mimodd_0_1_5), but one I uploaded myself (I discovered the bug when I did a test install after updating our local Galaxy).
I need to think about how to solve this more generally but if you want a quick work around - I think it would work to just replace "python" in $GALAXY_ROOT/set_metadata.sh with a hard-coded path to Galaxy's python or perhaps just "python2".
Since I can change the tool shed tool a more "global" solution could be to simply remove the "python" symlink from the python3 virtualenv after its created and have the tool use "python3" explicitly at runtime. I'll try whether that works, but I think it should.
Sorry for the inconvenience and thanks for bringing this to our attention.
No problem. I'm glad if I could help with something instead of just asking questions all the time. Wolfgang
On Tue, Jan 20, 2015 at 12:52 PM, John Chilton <jmchilton@gmail.com> wrote:
Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)?
-John
On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> wrote:
On 01/20/2015 06:20 PM, Wolfgang Maier wrote:
I have not seen this error before (I believe not with latest_2014.10.06)
update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again.
The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately.
Wolfgang
Hi, Could this be related to the fact that galaxy is not seeing the tools that are installed in /usr/local/bin?? I'm on latest_2015.01.13 and since the two last upgrades I saw that interproscan and bsmap were not found by galaxy. Both are installed in /usr/local and when invoked on the command line, as the galaxy user, they work perfectly well. Before the last two upgrades both worked from within galaxy. Cristian
Date: Wed, 21 Jan 2015 10:11:43 -0500 From: jmchilton@gmail.com To: wolfgang.maier@biologie.uni-freiburg.de CC: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py
So the tool dependency stuff didn't change - but the local job runner now behaves like the other job runners (DRMAA, PBS, etc...) so you have uncovered a bug for all of them I think. Is this a tool shed tool or a locally created dependency?
I need to think about how to solve this more generally but if you want a quick work around - I think it would work to just replace "python" in $GALAXY_ROOT/set_metadata.sh with a hard-coded path to Galaxy's python or perhaps just "python2".
Sorry for the inconvenience and thanks for bringing this to our attention.
-John
On Tue, Jan 20, 2015 at 12:52 PM, John Chilton <jmchilton@gmail.com> wrote:
Are you using the local job runner (this will be the case if you haven't explicitly configured something like pbs or DRMAA in your job_conf.xml file)?
-John
On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> wrote:
On 01/20/2015 06:20 PM, Wolfgang Maier wrote:
I have not seen this error before (I believe not with latest_2014.10.06)
update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again.
The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately.
Wolfgang
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
C. Ch. -
John Chilton -
Wolfgang Maier