Hi Peter/ John requirementx.txt already includes the latest version of nose

nose==1.3.7

It is nose that is complaining it does not like the sys,args passed in by FunctionalTest.py

Christian

________________________________________ From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 11:30 AM To: Christian Brenninkmeijer; John Chilton Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

On Fri, Nov 20, 2015 at 9:59 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

Hi All,

I am unable to run Planemo against the latest dev branch.

1. import twill error

I was able to fix this by adding to requirements.txt twill==1.8.0 lxml==3.5.0 cssselect==0.9.1

But then I get

functional_tests.py: error: no such option: --with-nosehtml functional_tests.py ERROR 2015-11-20 09:57:23,066 Failure running tests Traceback (most recent call last): ... SystemExit: 2

That's not a very helpful error, is it? It reminds me of a missing dependency / installation problem I had long ago when first setting up TravisCI to run Galaxy tool tests:

https://github.com/peterjc/pico_galaxy/commit/c8265f4a42caf62eb3bc8b36b475f1...

@John: My guess is we need to add "nose" to planemo's requirements.txt to ensure this gets installed automatically?

Peter

I was referring to Galaxy's requirements .txt

This appears to be an issue with changes made to galaxy last weekend.

The very same Planemo work with

commit 1aecca7889681ad9518e26e2a62b470809d2cdd1 Author: Björn Grüning bjoern@gruenings.eu Date: Sat Nov 14 23:56:19 2015 +0100

But fails at or BEFORE

commit b19e71ec465c7145840acf684f8f09eeebb99b5a Author: Nicola Soranzo nicola.soranzo@tgac.ac.uk Date: Mon Nov 16 14:57:53 2015 +0000

Christian ________________________________________ From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 12:41 PM To: Christian Brenninkmeijer Cc: John Chilton; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

Which file are you looking at? There is no nose entry here:

https://github.com/galaxyproject/planemo/blob/master/requirements.txt

Perhaps a more recent version of nose is needed?

Peter

On Fri, Nov 20, 2015 at 11:51 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

Hi Peter/ John requirementx.txt already includes the latest version of nose

nose==1.3.7

It is nose that is complaining it does not like the sys,args passed in by FunctionalTest.py

Christian

---

From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 11:30 AM To: Christian Brenninkmeijer; John Chilton Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

On Fri, Nov 20, 2015 at 9:59 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

...

Hi All,

I am unable to run Planemo against the latest dev branch.

1. import twill error

I was able to fix this by adding to requirements.txt twill==1.8.0 lxml==3.5.0 cssselect==0.9.1

But then I get

functional_tests.py: error: no such option: --with-nosehtml functional_tests.py ERROR 2015-11-20 09:57:23,066 Failure running tests Traceback (most recent call last): ... SystemExit: 2

That's not a very helpful error, is it? It reminds me of a missing dependency / installation problem I had long ago when first setting up TravisCI to run Galaxy tool tests:

https://github.com/peterjc/pico_galaxy/commit/c8265f4a42caf62eb3bc8b36b475f1...

@John: My guess is we need to add "nose" to planemo's requirements.txt to ensure this gets installed automatically?

Peter

Do you have virtualenv installed and available to planemo? If not, might be worth reviewing this thread - http://dev.list.galaxyproject.org/Planemo-0-20-0-and-xunit-td4668334.html and installing it.

In dev, virtualenv is a hard dependency for running Galaxy and this is new, in the past Galaxy would run without it. We (Nate) may find a way around this before the next release (16.01) specifically to allow Galaxy to run in anaconda based environments.

If you have virtualenv installed, I really don't think Galaxy's dependency list should require modification especially for Python 2.7, is this for Linux or Mac OS X?

-John

On Fri, Nov 20, 2015 at 2:27 PM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

I have gotten past this stage and an now getting errors which may be my own.

The fix was to edit https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/dependencies/req... and add twill==1.8.0 lxml==3.5.0 cssselect==0.9.1 NoseHTML==0.4.2

There may well be other missing as well so that needs a look. For example in the old version I found an egg NoseTestDiff-0.1-py2.7.egg

Christian ________________________________________ From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 12:41 PM To: Christian Brenninkmeijer Cc: John Chilton; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

Which file are you looking at? There is no nose entry here:

https://github.com/galaxyproject/planemo/blob/master/requirements.txt

Perhaps a more recent version of nose is needed?

Peter

On Fri, Nov 20, 2015 at 11:51 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

...

Hi Peter/ John requirementx.txt already includes the latest version of nose

nose==1.3.7

It is nose that is complaining it does not like the sys,args passed in by FunctionalTest.py

Christian

---

From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 11:30 AM To: Christian Brenninkmeijer; John Chilton Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

On Fri, Nov 20, 2015 at 9:59 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

...

Hi All,

I am unable to run Planemo against the latest dev branch.

1. import twill error

I was able to fix this by adding to requirements.txt twill==1.8.0 lxml==3.5.0 cssselect==0.9.1

But then I get

functional_tests.py: error: no such option: --with-nosehtml functional_tests.py ERROR 2015-11-20 09:57:23,066 Failure running tests Traceback (most recent call last): ... SystemExit: 2

That's not a very helpful error, is it? It reminds me of a missing dependency / installation problem I had long ago when first setting up TravisCI to run Galaxy tool tests:

https://github.com/peterjc/pico_galaxy/commit/c8265f4a42caf62eb3bc8b36b475f1...

@John: My guess is we need to add "nose" to planemo's requirements.txt to ensure this gets installed automatically?

Peter

Confirmed with a fresh galaxy checkout commit 05ed245353331489ee443ac483b1569dab38f500 Author: Dannon Baker dannon.baker@gmail.com Date: Thu Nov 19 20:48:16 2015 -0500

Then ran sh.run.sh to get the updates

without the additions to requirements.txt Planemo fails.

Added the four lines.

restarted the server with sh run,sh

Noticed galaxy installing twill ect

Then Planemo test worked.

===

I don't know enough about the inner workings of Galaxy to say if the adding these lines to requirements.txt is the correct fix or a hack that covers a different bug elsewhere.

Christian

PS. I noticed that if you try to use planemo against the fresh checkout without starting the server with run.sh Galaxy will not fetch the required updates.. I hope that is because Planemo intentional does not check if galaxy is up to date to safe test time. In which case I agree!

________________________________________ From: galaxy-dev [galaxy-dev-bounces@lists.galaxyproject.org] on behalf of Christian Brenninkmeijer [christian.brenninkmeijer@manchester.ac.uk] Sent: Friday, November 20, 2015 3:25 PM To: John Chilton Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

Hi John,

Yes i have vitrualenv installed.

I tried planemo both using : brew tap galaxyproject/tap brew install planemo

And using: virtualenv .venv; . .venv/bin/activate pip install planemo

With the exact same results

I also have not trouble running galaxy without the extra. bit in galaxy's requiremets.txt

It is Planamo that fails. But only in the builds since this weekend.

Early Galaxy builds work fine.

Christian

PS. Ubuntu and python 2.7

________________________________________ From: John Chilton [jmchilton@gmail.com] Sent: Friday, November 20, 2015 2:53 PM To: Christian Brenninkmeijer Cc: Peter Cock; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

Do you have virtualenv installed and available to planemo? If not, might be worth reviewing this thread - http://dev.list.galaxyproject.org/Planemo-0-20-0-and-xunit-td4668334.html and installing it.

In dev, virtualenv is a hard dependency for running Galaxy and this is new, in the past Galaxy would run without it. We (Nate) may find a way around this before the next release (16.01) specifically to allow Galaxy to run in anaconda based environments.

If you have virtualenv installed, I really don't think Galaxy's dependency list should require modification especially for Python 2.7, is this for Linux or Mac OS X?

-John

On Fri, Nov 20, 2015 at 2:27 PM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

I have gotten past this stage and an now getting errors which may be my own.

The fix was to edit https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/dependencies/req... and add twill==1.8.0 lxml==3.5.0 cssselect==0.9.1 NoseHTML==0.4.2

There may well be other missing as well so that needs a look. For example in the old version I found an egg NoseTestDiff-0.1-py2.7.egg

Christian ________________________________________ From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 12:41 PM To: Christian Brenninkmeijer Cc: John Chilton; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

Which file are you looking at? There is no nose entry here:

https://github.com/galaxyproject/planemo/blob/master/requirements.txt

Perhaps a more recent version of nose is needed?

Peter

On Fri, Nov 20, 2015 at 11:51 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

...

Hi Peter/ John requirementx.txt already includes the latest version of nose

nose==1.3.7

It is nose that is complaining it does not like the sys,args passed in by FunctionalTest.py

Christian

---

From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 11:30 AM To: Christian Brenninkmeijer; John Chilton Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

On Fri, Nov 20, 2015 at 9:59 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

...

Hi All,

I am unable to run Planemo against the latest dev branch.

1. import twill error

I was able to fix this by adding to requirements.txt twill==1.8.0 lxml==3.5.0 cssselect==0.9.1

But then I get

functional_tests.py: error: no such option: --with-nosehtml functional_tests.py ERROR 2015-11-20 09:57:23,066 Failure running tests Traceback (most recent call last): ... SystemExit: 2

That's not a very helpful error, is it? It reminds me of a missing dependency / installation problem I had long ago when first setting up TravisCI to run Galaxy tool tests:

https://github.com/peterjc/pico_galaxy/commit/c8265f4a42caf62eb3bc8b36b475f1...

@John: My guess is we need to add "nose" to planemo's requirements.txt to ensure this gets installed automatically?

Peter

___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

Hi John,

I did as you suggested wiping out galaxy's .venv folder. And restoring requirements.txt as in gthub.

The results where quite interesting.

The planemo test ran fine. planemo test --galaxy_root=/home/christian/galaxy_new columns.xml (As long as there is no .venv folder in Galaxy)

=== However then I started and stopped the galaxy server using sh run.sh

The exact same planemo test than FAILED. Full output attached but key lines are: unctional_tests.py ERROR 2015-11-23 08:43:42,227 Failure running tests Traceback (most recent call last): File "./scripts/functional_tests.py", line 494, in main success = _run_functional_test( ) File "./scripts/functional_tests.py", line 443, in _run_functional_test import functional.test_toolbox File "/home/christian/galaxy_new/test/functional/test_toolbox.py", line 3, in <module> from base.twilltestcase import TwillTestCase File "/home/christian/galaxy_new/test/base/twilltestcase.py", line 21, in <module> import twill ImportError: No module named twill functional_tests.py INFO 2015-11-23 08:43:42,227 Shutting down

=== I then added the four lines to requirements,txt Ran sh run.sh again Full output attached. But key lines are: Requirement already satisfied (use --upgrade to upgrade): Whoosh==2.4.1+gx1 in ./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 64)) Collecting twill==1.8.0 (from -r requirements.txt (line 67)) /home/christian/galaxy_new/.venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarn.... InsecurePlatformWarning /home/christian/galaxy_new/.venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarn.... InsecurePlatformWarning Using cached twill-1.8.0.tar.gz Collecting lxml==3.5.0 (from -r requirements.txt (line 68)) Using cached lxml-3.5.0.tar.gz Collecting cssselect==0.9.1 (from -r requirements.txt (line 69)) Using cached cssselect-0.9.1.tar.gz Collecting NoseHTML==0.4.2 (from -r requirements.txt (line 70) Using cached NoseHTML-0.4.2-py2-none-any.whl Installing collected packages: twill, lxml, cssselect, NoseHTML Running setup.py install for twill Running setup.py install for lxml Running setup.py install for cssselectt Successfully installed NoseHTML-0.4.2 cssselect-0.9.1 lxml-3.5.0 twill-1.8.0 Activating virtualenv at /home/christian/galaxy_new/.venv No handlers could be found for logger "galaxy.config"

I then reran planemo and it worked fine.

===

So it looks like if galaxy does not have a .venv to use planemo runs fine But if there is a galaxy .venv Planemo uses that one and fails on missing reguirements. Unless these are added in.

====

While this indicates that the issue is not directly with Planemo is is worrying that Plnemo will not run against a galaxy instance that has been used.

Christian University of Manchester

PS. Twill and the other missing requirements have been installed on my machine

________________________________________ From: John Chilton [jmchilton@gmail.com] Sent: Friday, November 20, 2015 5:19 PM To: Christian Brenninkmeijer Cc: Peter Cock; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

DVCSs are weird - 1aecca7889681ad9518e26e2a62b470809d2cdd1 is a single commit that is in dev from a week ago but its parent is from dev on October 11 (that is when the branch was forked). So it actually doesn't have the recent huge switch from eggs to wheels that broken everything.

For instance, the following file in the commit still references Galaxy eggs: https://github.com/galaxyproject/galaxy/blob/1aecca7889681ad9518e26e2a62b470...

I guess what I am saying is I suspect this actually broke when we merged 428 (https://github.com/galaxyproject/galaxy/pull/428) closer to the end of October.

Anyway, if you have virtualenv than I am fairly stumped. Can you wipe out Galaxy's virtualenv and show me the full output of running planemo test.

-John

On Fri, Nov 20, 2015 at 1:30 PM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

I was referring to Galaxy's requirements .txt

This appears to be an issue with changes made to galaxy last weekend.

The very same Planemo work with

commit 1aecca7889681ad9518e26e2a62b470809d2cdd1 Author: Björn Grüning bjoern@gruenings.eu Date: Sat Nov 14 23:56:19 2015 +0100

But fails at or BEFORE

commit b19e71ec465c7145840acf684f8f09eeebb99b5a Author: Nicola Soranzo nicola.soranzo@tgac.ac.uk Date: Mon Nov 16 14:57:53 2015 +0000

Christian ________________________________________ From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 12:41 PM To: Christian Brenninkmeijer Cc: John Chilton; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

Which file are you looking at? There is no nose entry here:

https://github.com/galaxyproject/planemo/blob/master/requirements.txt

Perhaps a more recent version of nose is needed?

Peter

On Fri, Nov 20, 2015 at 11:51 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

...

Hi Peter/ John requirementx.txt already includes the latest version of nose

nose==1.3.7

It is nose that is complaining it does not like the sys,args passed in by FunctionalTest.py

Christian

---

From: Peter Cock [p.j.a.cock@googlemail.com] Sent: Friday, November 20, 2015 11:30 AM To: Christian Brenninkmeijer; John Chilton Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to up Planemo against latest dev

On Fri, Nov 20, 2015 at 9:59 AM, Christian Brenninkmeijer christian.brenninkmeijer@manchester.ac.uk wrote:

...

Hi All,

I am unable to run Planemo against the latest dev branch.

1. import twill error

I was able to fix this by adding to requirements.txt twill==1.8.0 lxml==3.5.0 cssselect==0.9.1

But then I get

functional_tests.py: error: no such option: --with-nosehtml functional_tests.py ERROR 2015-11-20 09:57:23,066 Failure running tests Traceback (most recent call last): ... SystemExit: 2

That's not a very helpful error, is it? It reminds me of a missing dependency / installation problem I had long ago when first setting up TravisCI to run Galaxy tool tests:

https://github.com/peterjc/pico_galaxy/commit/c8265f4a42caf62eb3bc8b36b475f1...

@John: My guess is we need to add "nose" to planemo's requirements.txt to ensure this gets installed automatically?

Peter