Jorrit, I've not been able to reproduce the first scenario using a local Galaxy instance against the test tool shed. Could you tell me which revision of Galaxy you're running when you encounter this behavior? I do see that the revision bb20766df19c that you were initially trying to update is immediately followed by another downloadable revision, in which case Galaxy will (correctly) not find any updates for it, due to the reproducibility concerns I mentioned in my previous email. --Dave B. On 3/28/13 17:58:10.000, jorrit poelen wrote:

Hey Dave,

I took some time to try to reproduce the behavior you described using a local test tool shed.

As I understand your previous email, the first scenario (below) outcome of using my local test toolshed is expected, but the second scenario is not. Also, the first scenario outcome using the psu test toolshed seems different compared to running it against my local test tool shed. I was unable to test the second scenario, because of a apparent (recently introduced) bug in the psu test tool shed login.

What am I missing?

Thanks much for you patience, -jorrit

------- First scenario - Change README / update -------- Steps: 1. install repository A from local test tool shed B into local galaxy instance C 2. clone respository A from local test tool into local dev environment **3. apply change to README** 4. hg commit / push changes to local test tool shed B 5. verify that test tool shed received changes by checking content of README and change set hash. 6. in galaxy instance C > admin > manage installed tool shed repositories > repository A > get updates 7. galaxy instance C web page says that updates have been detected and applied **8. verify that README of repo A in galaxy instance C has been changed.**

Outcome: local tool shed instance test - update detected, changes were applied to local galaxy instance C

psu tool shed test - update *not* detected, changes were *not* applied to local galaxy instance C (see initial post)

------ Second scenario - Bump tool version / update ------ Steps: same as first scenario, except for **3. apply change to tool version 0.1 --> 0.11 ** **8. verify that tool version 0.1 has changed to 0.11 of repo A in galaxy instance C has been changed.**

Outcome: local tool shed instance test - update detected, changes were applied: a new version of tool is installed in Galaxy instance.

psu tool shed test - unable to test because I could not login to psu test toolshed

On Mar 28, 2013, at 2:19 PM, Dave Bouvier wrote:

...

Jorrit,

I may have been a bit unclear in my previous response. Allow me to illustrate two scenarios:

First scenario:

1. A repository is created containing a BWA tool with version 0.5.9 2. The repository is installed to Galaxy. 3. New README file is uploaded, tool version is *not* changed.

Second scenario:

1. A repository is created containing a BWA tool with version 0.5.9 2. The repository is installed to Galaxy. 3. New README file is uploaded, and updated BWA tool is uploaded with version 0.6

In the first scenario, Galaxy will find that the repository has been updated, and apply the update.

In the second scenario, Galaxy will not find that the repository has been updated, but will allow you to install both repositories, the one containing version 0.5.9 and the one containing version 0.6 of the BWA tool.

The reason for this behavior is that if a tool has been updated to produce different output, the tool version should be updated, which leads to a separate installable changeset revision being generated. This way, any analyses run on the old tool version can be re-run and produce the same output, but any new analyses can run on the new version of the tool, with whatever new features have been added. This ensures reproducibility throughout the version lineage of the tool, while still allowing Galaxy users to use the most recent version of an available tool for future analyses.

--Dave B.

On 3/28/13 17:03:10.000, jorrit poelen wrote:

...

Hey Dave,

Thanks for pointing me to the wiki that helped demystify the hg changeset <--> repository metadata connection for me.

I'll make sure to change the version numbers in the tool xml definitions whenever a change needs to be make available to galaxy instances through the toolshed. So, even for an update in the README file, I'll make sure to change some version to make the change available. This feels a little clunky for me now, but I am sure there's a good reason to make it work as it does now.

I very much appreciate your prompt response and I'll make sure to dig around the wiki a little longer before posting a question next time ; )

-jorrit

On Mar 26, 2013, at 10:41 AM, Dave Bouvier wrote:

...

Jorrit,

First, allow me to clarify a point of naming:

It's not quite right to say "tools" in this situation, as tools are only one of several utilities that a tool shed repository can contain, along with datatypes, workflows, repository dependencies, and so on. This is an important distinction, because some repositories can (and do) contain no tools, and only datatypes and/or other Galaxy utilities.

In your process description for the current behavior, the tool shed will not report updates to a repository if the tool version has changed. For more details on how and why this happens, see the following wiki page:

http://wiki.galaxyproject.org/RepositoryRevisions#Adding_additional_change_s...

Where the behavior is detailed in these sections:

Adding additional change sets to the initial change set in a repository • What is repository metadata? • What does it mean when repository metadata is associated with a change set in the repository change log? • Adding a change set that does not generate a new repository metadata record • Adding a change set that generates a new repository metadata record • More examples of installable repository change set revisions

--Dave B.

On 3/26/13 12:45:41.000, jorrit poelen wrote:

...

Hey Dave,

Thanks against for your quick response!

I checked my universe_wsgi.ini file and can confirm that the enabled_tool_shed_check was *commented*.

I've uncommented the settings and changed them to:

enable_tool_shed_check = True hours_between_check = 1

Then I followed the steps outlined in my initial email and the observed the same (unexpected) behavior.

Perhaps I am describing a feature request rather than a bug: My expectation is that whenever I explicitly request to update a tool, a immediate check is performed against the relevant tool shed.

To make sure that I correctly understand the existing tool update feature, please confirm that the following describes the currently expected tool update behavior:

0. admin of galaxy instance Z enables the feature to check for tool updates in toolsheds every hour 1. tool X revision 1.0 is installed from tool shed Y into galaxy instance Z by tool developer using hg push 2. tool X is current revision is updated to revision 1.1 in tool shed Y 3. immediately after, admin of galaxy instance Z attempts to (manually) upgrade tool X to version 1.1 using the admin web interface 4. galaxy instance Z reports that no new version of tool X are available, even though a new version exists in the toolshed 5. galaxy instance Z detects version 1.1 of tool X at most one hour after the update was made 6. admin of galaxy instance Z tries again

Thanks for your patience and efforts,

-jorrit

On Mar 26, 2013, at 8:19 AM, Dave Bouvier wrote:

...

Jorrit,

Just to eliminate one possible cause of this behavior, could you confirm that your local Galaxy instance's universe_wsgi.ini file's enable_tool_shed_check setting is uncommented and set to True, and the hours_between_check is also uncommented and set to a value between 1 and 12?

--Dave B.

On 3/25/13 21:26:07.000, jorrit poelen wrote: > Hey y'all, > > I noticed that my local galaxy instance doens't pickup latest updates from test toolshed. > > Steps to reproduce: > > 1. install obotools from test toolshed using hg clone http://jorrit@testtoolshed.g2.bx.psu.edu/repos/jorrit/obotools , revision 45:bb20766df19c > > 2. hg push new change of obotools of galaxy tool to testtoolshed , creating revision 46:532808477800 > > 3. update obotools in galaxy admin > > expected result: galaxy admin detects new version and give an option to update. > > actual result: galaxy admin does not detect new version. > > I've attached some galaxy logging below. One thing I noticed that the reported commit time in testtoolshed seems 7 hours off - just minutes after I pushed the change, testtoolshed says the change was commit "~ 7 hours ago". > > Workaround I am using to do update is to deactivate and remove the tool all together, and re-install straight from test-toolshed. What should I do to make the tool version update work? I am probably missing something. > > Thanks! > > -jorrit > > > Galaxy logging on looking for updates in test toolshed: > --- > > 24.130.122.217 - - [26/Mar/2013:01:12:46 +0000] "GET /admin_toolshed/browse_repositories HTTP/1.1" 200 - "http://galaxy:7474/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" > 24.130.122.217 - - [26/Mar/2013:01:12:51 +0000] "GET /admin_toolshed/browse_repositories?operation=get+updates&id=a799d38679e985db HTTP/1.1" 302 - "http://galaxy:7474/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" > 24.130.122.217 - - [26/Mar/2013:01:12:51 +0000] "GET /admin_toolshed/check_for_updates?id=a799d38679e985db HTTP/1.1" 302 - "http://galaxy:7474/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" > 24.130.122.217 - - [26/Mar/2013:01:12:52 +0000] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=http://testtoolshed.g2.bx.psu.edu/&name=obotools&owner=jorrit&changeset_revision=bb20766df19c&latest_changeset_revision=bb20766df19c&latest_ctx_rev=45 HTTP/1.1" 302 - "http://galaxy:7474/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" > 24.130.122.217 - - [26/Mar/2013:01:12:52 +0000] "GET /admin_toolshed/manage_repository?status=done&message=The+installed+repository+named+%27obotools%27+is+current%2C+there+are+no+updates+available.++&id=a799d38679e985db HTTP/1.1" 200 - "http://galaxy:7474/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10" > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/