Jeremy

First of all, thanks for all your efforts, and for updating the cuffdiff, etc, wrappers.

However, it's a little frustrating to us that a non-backward compatible change in the supported version of the cuffdiff executable was not reflected in a new version number for the cuffdiff (and other cuff) wrappers. This leaves us no way to support both versions of cuffdiff, and makes all histories that were run with the older version "non-reproducible".

I know the general versioning problem is address through the tool-shed mechanism, but for the remaining core wrappers, can maintainers please update wrapper version numbers when making changes?

For this wrapper set, how would you advise us to support both?

It seems to me that I HAVE to leave the cuffdiff v1.3-compatible as wrapper_version=0.0.5, or old histories/workflows break. However, if I change the cuffdiff_v2.1-compatible wrapper to something else, I'll be out of sync with galaxy-central, and will create migration/reproducibility issues in the future.

Would you accept a pull request to update the version number of the cuff wrappers that were changed in the migration to cuff*2.1?

Thanks again for updating the wrappers, and help me figure out how to support them cleanly.

Regards, Curtis

From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Jeremy Goecks Sent: Wednesday, June 12, 2013 3:09 PM To: Karen Chait Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

Hello, After the latest update of Galaxy I saw that the cuffdiff wrapper is for version 2.0. I updated the $PATH to cuffdiff v. 2.0.2 but when I run cuffdiff in Galaxy I get the following error: Error running cuffdiff. cuffdiff: /lib64/libz.so.1: no version information available (required by cuffdiff) cuffdiff: unrecognized option '--library-norm-method' cuffdiff v1.3.0 (3022)

What else do I need to define inorder to run the latest version of cuffdiff.

This depends on your setup. If you're running Galaxy under a particular user, you may have to update that user's $PATH. If you're running on a cluster, you'll need to ensure that the submission script uses the correct $PATH.

If you provide more details on your set up, it may be possible to provide additional insight/suggestions for your problem.

Best, J.

Jeremy

Thanks for the super-quick response. That should resolve things nicely.

We can hack the old wrapper back in, using a hard-coded path to the old cuffdiff. We can pull your new wrapper, and install the new cuffdiff.

Then we should have backward and forward compatibility. Shantanu suggests it might be a better world if we create a tool-shed repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install that. Would that be a better path, or just add confusion?

Regards, Curtis

From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Thursday, September 19, 2013 2:18 PM To: Curtis Hendrickson (Campus) Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

I know the general versioning problem is address through the tool-shed mechanism, but for the remaining core wrappers, can maintainersplease update wrapper version numbers when making changes?

Not bumping wrapper versions is an oversight, not deliberate.

For this wrapper set, how would you advise us to support both?

Fortunately, I refactored the Cuffdiff wrapper recently and correctly bumped the tool version:

https://bitbucket.org/galaxy/galaxy-central/commits/6a32d26aefc758cf576e0b75...

You can either pull this changeset manually or wait until the next distribution in early October to get the changeset.

Best, J.

Jeremy,

We didn't have time to factor out to toolshed just now, so we took the following approach

1. Take 1.3-compatible cuffdiff wrapper (0.0.5) and move it to cuffdiff_1.3_wrapper.py/xml - and add a dependency on cufflinks verson=1.3.0 (and set that up).

2. Apply your patch, changing current cuffdiff wrapper to (0.0.6)

3. Add a line in tool_conf.xml so that both your modern cuffdiff v6 wrapper and our cuffdiff v5 wrappers are included.

When we restarted, and went to the GUI, there was only ONE cuffdiff tool listed, and when you clicked on it, it brought up the 0.0.5 version (the 2nd listed in tool_conf). There was no version pull down to get the 0.0.6 version.,

Any idea what I've missed?

Regards, Curtis

From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Thursday, September 19, 2013 3:17 PM To: Curtis Hendrickson (Campus) Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

We can hack the old wrapper back in, using a hard-coded path to the old cuffdiff.

You could also use Managed Tool Dependencies for multiple versions of cuffdiff:

http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies

Shantanu suggests it might be a better world if we create a tool-shed repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install that. Would that be a better path, or just add confusion?

This is on the right path. The right thing to do is migrate the Cuff* tool wrappers out of the galaxy-central/dist repositories; this is a bit more effort than simply creating the tool-shed repository for the wrappers.

There are two reasons I haven't done this:

(a) time; (b) because the tools-and consequently the wrappers-are still in active development, there needs to be a place for managing development of the wrappers. Hence, the wrappers need a home on bitbucket or github, but we (the Galaxy team) haven't determined where these wrappers should live.

(a) is a much more significant barrier than (b), so if a pull request migrating Cuff* tools out of the distribution were made, I'm sure (b) could be worked out.

Best, J.

Jeremy,

I'm sure that would get both tools to show up, but it seems to me that would miss the goal of past/future reproducibility.

My goal is to have to two versions of the program available at the same time - Cuffdiff_wrapper 0.0.5 (cuffdiff/links 1.3) the historical one that all the current histories/workflows reference, so those don't break, and Cuffdiff_wrapper 0.0.6 (cuffdiff/links 2.1) the new version for use going forward, which has a distinct version so that those histories/workflows are reproducible and the choice of which version to use is the users. I want to avoid someone who wants to re-run last-month's cuffdiff with slightly different parameters being forced (w/o their knowledge) to run a more recent version of cuffdiff - which would then give an apples to oranges comparison.

With 2 versions of the same ID installed, I expected to either see two listings in the tool pane AND/OR a version pulldown on the cuffdiff tool params pane.

Is the only way to get two versions of the same tool id installed via toolshed?

Sorry - I feel like I've mis-understood the tool version/id resolution architecture. Happy to talk on the phone if that would be a better use of your time.

Thanks, Curtis

From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Wednesday, September 25, 2013 8:43 AM To: Curtis Hendrickson (Campus) Cc: galaxy-dev@lists.bx.psu.edu; Shantanu Pavgi (Campus) Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

The problem is likely a collision on the tool_id; update the tool ID for one of the wrappers and that should solve the problem.

Best, J.

On Sep 24, 2013, at 5:30 PM, Curtis Hendrickson (Campus) wrote:

Jeremy,

We didn't have time to factor out to toolshed just now, so we took the following approach 1. Take 1.3-compatible cuffdiff wrapper (0.0.5) and move it to cuffdiff_1.3_wrapper.py/xml - and add a dependency on cufflinks verson=1.3.0 (and set that up). 2. Apply your patch, changing current cuffdiff wrapper to (0.0.6) 3. Add a line in tool_conf.xml so that both your modern cuffdiff v6 wrapper and our cuffdiff v5 wrappers are included.

When we restarted, and went to the GUI, there was only ONE cuffdiff tool listed, and when you clicked on it, it brought up the 0.0.5 version (the 2nd listed in tool_conf). There was no version pull down to get the 0.0.6 version.,

Any idea what I've missed?

Regards, Curtis

From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Thursday, September 19, 2013 3:17 PM To: Curtis Hendrickson (Campus) Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

We can hack the old wrapper back in, using a hard-coded path to the old cuffdiff.

You could also use Managed Tool Dependencies for multiple versions of cuffdiff:

http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies

Shantanu suggests it might be a better world if we create a tool-shed repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install that. Would that be a better path, or just add confusion?

This is on the right path. The right thing to do is migrate the Cuff* tool wrappers out of the galaxy-central/dist repositories; this is a bit more effort than simply creating the tool-shed repository for the wrappers.

There are two reasons I haven't done this:

(a) time; (b) because the tools-and consequently the wrappers-are still in active development, there needs to be a place for managing development of the wrappers. Hence, the wrappers need a home on bitbucket or github, but we (the Galaxy team) haven't determined where these wrappers should live.

(a) is a much more significant barrier than (b), so if a pull request migrating Cuff* tools out of the distribution were made, I'm sure (b) could be worked out.

Best, J.

Jeremy,

Thanks for letting us know so quickly that we're heading down a non-viable path. As I understand it, changing the tool id of the cuff* tools would solve our problem locally, but make workflows created here non-interoperable with Galaxy Central.

We'll focus on moving to latest tool versions and educating uses on non-reproducibility issues/work-arounds - probably providing the older wrappers under a different tool_id, as your average user will want the latest one.

If we're blessed with some extra time, we'll tool-shed-ify it, but I can't commit to that at the moment.

Thanks again for the super quick responses.

Regards, Curtis

From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Wednesday, September 25, 2013 10:43 AM To: Curtis Hendrickson (Campus) Cc: galaxy-dev@lists.bx.psu.edu; Shantanu Pavgi (Campus) Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

Jeremy,

I'm sure that would get both tools to show up, but it seems to me that would miss the goal of past/future reproducibility.

My goal is to have to two versions of the program available at the same time - Cuffdiff_wrapper 0.0.5 (cuffdiff/links 1.3) the historical one that all the current histories/workflows reference, so those don't break, and Cuffdiff_wrapper 0.0.6 (cuffdiff/links 2.1) the new version for use going forward, which has a distinct version so that those histories/workflows are reproducible and the choice of which version to use is the users. I want to avoid someone who wants to re-run last-month's cuffdiff with slightly different parameters being forced (w/o their knowledge) to run a more recent version of cuffdiff - which would then give an apples to oranges comparison.

With 2 versions of the same ID installed, I expected to either see two listings in the tool pane AND/OR a version pulldown on the cuffdiff tool params pane.

Is the only way to get two versions of the same tool id installed via toolshed?

Yes.

Unfortunately, it's not possible to manually install multiple versions of the same tool; the development efforts of the Galaxy team are focused on multiple version installation via only the tool shed for now. However, we'd welcome a pull request that adds this functionality to Galaxy.

Given that you want to provide backward compatibility, it's probably best to keep the tool_id for the older Cuffdiff wrapper and change the new one; mercurial merges should handle this change gracefully as it's small.

Best, J.

Sorry - I feel like I've mis-understood the tool version/id resolution architecture. Happy to talk on the phone if that would be a better use of your time.

Thanks, Curtis

From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Wednesday, September 25, 2013 8:43 AM To: Curtis Hendrickson (Campus) Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu; Shantanu Pavgi (Campus) Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

The problem is likely a collision on the tool_id; update the tool ID for one of the wrappers and that should solve the problem.

Best, J.

On Sep 24, 2013, at 5:30 PM, Curtis Hendrickson (Campus) wrote:

Jeremy,

We didn't have time to factor out to toolshed just now, so we took the following approach 1. Take 1.3-compatible cuffdiff wrapper (0.0.5) and move it to cuffdiff_1.3_wrapper.py/xml - and add a dependency on cufflinks verson=1.3.0 (and set that up). 2. Apply your patch, changing current cuffdiff wrapper to (0.0.6) 3. Add a line in tool_conf.xml so that both your modern cuffdiff v6 wrapper and our cuffdiff v5 wrappers are included.

When we restarted, and went to the GUI, there was only ONE cuffdiff tool listed, and when you clicked on it, it brought up the 0.0.5 version (the 2nd listed in tool_conf). There was no version pull down to get the 0.0.6 version.,

Any idea what I've missed?

Regards, Curtis

From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Thursday, September 19, 2013 3:17 PM To: Curtis Hendrickson (Campus) Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

We can hack the old wrapper back in, using a hard-coded path to the old cuffdiff.

You could also use Managed Tool Dependencies for multiple versions of cuffdiff:

http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies

Shantanu suggests it might be a better world if we create a tool-shed repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install that. Would that be a better path, or just add confusion?

This is on the right path. The right thing to do is migrate the Cuff* tool wrappers out of the galaxy-central/dist repositories; this is a bit more effort than simply creating the tool-shed repository for the wrappers.

There are two reasons I haven't done this:

(a) time; (b) because the tools-and consequently the wrappers-are still in active development, there needs to be a place for managing development of the wrappers. Hence, the wrappers need a home on bitbucket or github, but we (the Galaxy team) haven't determined where these wrappers should live.

(a) is a much more significant barrier than (b), so if a pull request migrating Cuff* tools out of the distribution were made, I'm sure (b) could be worked out.

Best, J.