Hi, I have been trying out the latest version of Galaxy and a newly installed local toolshed and I have a few questions/suggestions/bug reports:
1. Tool Versioning:
a) As I understand from the wiki, when I create a new version of a tool in a tool shed I must then install the latest version in my Galaxy instance. This then shows up as two separate tools in my tool menu. Presumably it is best to change the xml description tag to reflect the version number and make tool versions distinguishable in the Galaxy tool menu?
b) Would it make more sense to have a single entry for each tool in the tool menu and a drop down to choose which version to run in the tool gui? Similar to when I try to rerun a tool that is no longer installed I get a drop down option to use a derivative of that tool if it exists.
d) Say I create a workflow then install a new version of a tool within the workflow. I would like to be able to run that workflow using either version of the tool without creating a new workflow. Is this possible? To me it would make sense to again provide a drop down to select tool versions on the workflow ui just before execution.
c) The toolshed interface allows me to change the name of my tool. If I do this, then go to my Galaxy admin ui and try to get updates for my tool then it fails. Does it make sense to change a tools name after it is created? Is it possible to track previous tool names to avoid this happening?
e) Email alerts: I could not find an option to receive email alerts about tool updates when installing via the Galaxy admin ui. Perhaps this should be an option during installation.
Running toolshed tools:
I have no problems installing tools from toolsheds via Galaxy admin interface, however I can't get these tools to run. They stay Grey forever and look like this in the manage jobs table:
65 xxxx@xxxx.xx.ac.uk 0 minutes ago toolshed.g2.bx.psu.edu/repos/aaronquinlan/bedtools/bedtools_coveragebed_counts/0.1.0 new None None None
Do I have to set a tool runner or some other option for shed_tools?
Migrating tools:
I upgraded to the latest dist release of galaxy and when I restarted the server I expected to get a message about migrating freebayes tools. I didn't see this message. Will this be because I restart using --daemon? Is it advisable to therefore restart without --daemon after an upgrade to see if there are any tools to be migrated?
Genome indexing:
1. I realise this is just a beta feature and it will be great when fully integrated. I tried to generate bowtie2 indexes for an installed build. I can see the job appear below the table in yellow and it contains the message "...added to the job queue to be indexed with bowtie2". However it seems to just sit there, even after restarting. Should I be able to see the job under 'manage jobs'?
2. Where are the generated indexes stored and is the .loc file automatically updated?
Workflow menus:
Would it be possible to categorise workflows the way we do with tools. I have a long list of workflows and it is sometimes difficult to find the one I need. I realise that workflows are user specific so this is not as easy to implement as with tools. Perhaps it would be possible to have category menus for published workflows at least?
Trackster:
I have noticed a few bugs with trackster in the latest release (tested on Chrome and Firefox):
1. Overview button does not work 2. If I add tracks to a group and make a composite track, then showindividual tracks again I get new menu options (filters, tool, tool parameter space visualisation). None of these seem to work and trackster often stops working after doing this.
Thanks for all the effort put in to these great new features. I hope this lengthy email is helpful!
Shaun Webb Edinburgh University
Hello Shaun,
On 7/31/12 09:37:23.000, SHAUN WEBB wrote:
Genome indexing:
- I realise this is just a beta feature and it will be great when fully
integrated. I tried to generate bowtie2 indexes for an installed build. I can see the job appear below the table in yellow and it contains the message "...added to the job queue to be indexed with bowtie2". However it seems to just sit there, even after restarting. Should I be able to see the job under 'manage jobs'?
No, the indexing jobs are not picked up by that interface. If you click on the job ID under the 'Recent jobs:' heading, you should see a status listing of the indexing job and any sub-jobs it initiated.
- Where are the generated indexes stored and is the .loc file
automatically updated?
This can be specified with the genome_data_path option in your universe_wsgi.ini file. Currently, it defaults to tool-data/genome/<build>/<index name>/
The .loc files are updated when the job returns from a successful run, if there is no entry for that genome and index already present.
--Dave B.
Thanks Dave.
I think I had a permission error on my bowtie2 directory. It seems to be running ok now. However the previous job is still there saying it's running but not. Is there any way to cancel these jobs or do I need to do it via the database?
Thanks for the info. Shaun
Quoting Dave Bouvier dave@bx.psu.edu on Tue, 31 Jul 2012 10:21:07 -0400:
Hello Shaun,
On 7/31/12 09:37:23.000, SHAUN WEBB wrote:
Genome indexing:
- I realise this is just a beta feature and it will be great when fully
integrated. I tried to generate bowtie2 indexes for an installed build. I can see the job appear below the table in yellow and it contains the message "...added to the job queue to be indexed with bowtie2". However it seems to just sit there, even after restarting. Should I be able to see the job under 'manage jobs'?
No, the indexing jobs are not picked up by that interface. If you click on the job ID under the 'Recent jobs:' heading, you should see a status listing of the indexing job and any sub-jobs it initiated.
- Where are the generated indexes stored and is the .loc file
automatically updated?
This can be specified with the genome_data_path option in your universe_wsgi.ini file. Currently, it defaults to tool-data/genome/<build>/<index name>/
The .loc files are updated when the job returns from a successful run, if there is no entry for that genome and index already present.
--Dave B. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Shaun,
On Jul 31, 2012, at 9:37 AM, SHAUN WEBB wrote:
Hi, I have been trying out the latest version of Galaxy and a newly installed local toolshed and I have a few questions/suggestions/bug reports:
- Tool Versioning:
a) As I understand from the wiki, when I create a new version of a tool in a tool shed I must then install the latest version in my Galaxy instance.
This is not correct - you can choose to install any valid version of the tool(s). Can you let me know what content of the wiki resulted in this confusion? I'll be sure to make it more clear.
This then shows up as two separate tools in my tool menu. Presumably it is best to change the xml description tag to reflect the version number and make tool versions distinguishable in the Galaxy tool menu?
b) Would it make more sense to have a single entry for each tool in the tool menu and a drop down to choose which version to run in the tool gui? Similar to when I try to rerun a tool that is no longer installed I get a drop down option to use a derivative of that tool if it exists.
Your above recommendation in (b) is exactly what will occur as soon as I get a chance to implement it.
c) The toolshed interface allows me to change the name of my tool. If I do this, then go to my Galaxy admin ui and try to get updates for my tool then it fails. Does it make sense to change a tools name after it is created? Is it possible to track previous tool names to avoid this happening?
This should not occur. Where are you able to change the name of your tool? You are not even allowed to change the name of the repository if it has been cloned (you can change the name of the repository as long as it has not been cloned). Can you send me a screen shot of hte page that allows you to change the name of your tool in the tool shed? I'll provide a fix as soon as I see where the issue lies.
e) Email alerts: I could not find an option to receive email alerts about tool updates when installing via the Galaxy admin ui. Perhaps this should be an option during installation.
You can currently set email alerts for specified repositories in the tool shed itself. I will add it to my list to enhance the install process with this feature. Thanks for the request!
Running toolshed tools:
I have no problems installing tools from toolsheds via Galaxy admin interface, however I can't get these tools to run. They stay Grey forever and look like this in the manage jobs table:
65 xxxx@xxxx.xx.ac.uk 0 minutes ago toolshed.g2.bx.psu.edu/repos/aaronquinlan/bedtools/bedtools_coveragebed_counts/0.1.0 new None None None
Do I have to set a tool runner or some other option for shed_tools?
No special configuration settings are required in your local Galaxy instance to to run tools installed from a tool shed. If they stay in the queued state, there must be an issue in your local Galaxy instance that causes installed tools as well as tools included in the distribution to remain queued. It has nothing to do with the fact they were installed.
Migrating tools:
I upgraded to the latest dist release of galaxy and when I restarted the server I expected to get a message about migrating freebayes tools. I didn't see this message. Will this be because I restart using --daemon? Is it advisable to therefore restart without --daemon after an upgrade to see if there are any tools to be migrated?
It may be best to restart without --daemon after an upgrade. This process works just like the Galaxy database migration process (but you are not forced to perform the tool installs), so you can do whatever you currently do to handle database migrations after you upgrades. Starting with --daemon should still display the exception in your paster log. Do you run multiple front end web servers? If so, the exception will only be thrown from one of them.
Shaun Webb Edinburgh University
-- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Greg, thanks for the response
Quoting Greg Von Kuster greg@bx.psu.edu on Tue, 31 Jul 2012 11:50:36 -0400:
Hi Shaun,
On Jul 31, 2012, at 9:37 AM, SHAUN WEBB wrote:
Hi, I have been trying out the latest version of Galaxy and a newly installed local toolshed and I have a few questions/suggestions/bug reports:
- Tool Versioning:
a) As I understand from the wiki, when I create a new version of a tool in a tool shed I must then install the latest version in my Galaxy instance.
This is not correct - you can choose to install any valid version of the tool(s). Can you let me know what content of the wiki resulted in this confusion? I'll be sure to make it more clear.
This then shows up as two separate tools in my tool menu. Presumably it is best to change the xml description tag to reflect the version number and make tool versions distinguishable in the Galaxy tool menu?
b) Would it make more sense to have a single entry for each tool in the tool menu and a drop down to choose which version to run in the tool gui? Similar to when I try to rerun a tool that is no longer installed I get a drop down option to use a derivative of that tool if it exists.
Your above recommendation in (b) is exactly what will occur as soon as I get a chance to implement it.
Perhaps my wording was unclear here. What I meant was if a new version of a tool is created in a tool shed I have to install that version to be able to use it, rather than using 'get updates'. This then results in multiple listings of the tool. But it sounds like you plan on improving this process anyway.
c) The toolshed interface allows me to change the name of my tool. If I do this, then go to my Galaxy admin ui and try to get updates for my tool then it fails. Does it make sense to change a tools name after it is created? Is it possible to track previous tool names to avoid this happening?
This should not occur. Where are you able to change the name of your tool? You are not even allowed to change the name of the repository if it has been cloned (you can change the name of the repository as long as it has not been cloned). Can you send me a screen shot of hte page that allows you to change the name of your tool in the tool shed? I'll provide a fix as soon as I see where the issue lies.
OK, I guess I changed the name of my repository not tool. I then installed the tool via my local Galaxy instance. I can still change the name of the repository in my toolshed and when I try to get updates via Galaxy admin panel I get the following server error: "AttributeError: 'NoneType' object has no attribute 'repo_path'"
Running toolshed tools:
I have no problems installing tools from toolsheds via Galaxy admin interface, however I can't get these tools to run. They stay Grey forever and look like this in the manage jobs table:
65 xxxx@xxxx.xx.ac.uk 0 minutes ago toolshed.g2.bx.psu.edu/repos/aaronquinlan/bedtools/bedtools_coveragebed_counts/0.1.0 new None None None
Do I have to set a tool runner or some other option for shed_tools?
No special configuration settings are required in your local Galaxy instance to to run tools installed from a tool shed. If they stay in the queued state, there must be an issue in your local Galaxy instance that causes installed tools as well as tools included in the distribution to remain queued. It has nothing to do with the fact they were installed.
No. All other tools run fine. Tools installed from either my toolshed or the Galaxy toolshed remain queued. If this is not a known issue then I will try to dig a little deeper to find the problem. Any pointers would be useful if you can think of why there should be a difference.
Thanks for your help! Shaun
Hi Shaun,
On Aug 1, 2012, at 6:13 AM, SHAUN WEBB wrote:
c) The toolshed interface allows me to change the name of my tool. If I do this, then go to my Galaxy admin ui and try to get updates for my tool then it fails. Does it make sense to change a tools name after it is created? Is it possible to track previous tool names to avoid this happening?
This should not occur. Where are you able to change the name of your tool? You are not even allowed to change the name of the repository if it has been cloned (you can change the name of the repository as long as it has not been cloned). Can you send me a screen shot of hte page that allows you to change the name of your tool in the tool shed? I'll provide a fix as soon as I see where the issue lies.
OK, I guess I changed the name of my repository not tool. I then installed the tool via my local Galaxy instance. I can still change the name of the repository in my toolshed and when I try to get updates via Galaxy admin panel I get the following server error: "AttributeError: 'NoneType' object has no attribute 'repo_path'"
I cannot reproduce this behavior, so can you let me know the steps you use to do so? I need the page on which you are able to change the name of the tool shed repository in the tool shed after it has been cloned. Here is a screen shot of what should be occurring after the repository is cloned. Notice that the repository name is not alterable as it is no longer a text field.
abyss_tool Clone this repository: hg clone http://greg@toolshed.g2.bx.psu.edu/repos/msjeon/abyss_tool Name: abyss_tool Repository names cannot be changed if the repository has been cloned.
Running toolshed tools:
I have no problems installing tools from toolsheds via Galaxy admin interface, however I can't get these tools to run. They stay Grey forever and look like this in the manage jobs table:
65 xxxx@xxxx.xx.ac.uk 0 minutes ago toolshed.g2.bx.psu.edu/repos/aaronquinlan/bedtools/bedtools_coveragebed_counts/0.1.0 new None None None
Do I have to set a tool runner or some other option for shed_tools?
No special configuration settings are required in your local Galaxy instance to to run tools installed from a tool shed. If they stay in the queued state, there must be an issue in your local Galaxy instance that causes installed tools as well as tools included in the distribution to remain queued. It has nothing to do with the fact they were installed.
No. All other tools run fine. Tools installed from either my toolshed or the Galaxy toolshed remain queued. If this is not a known issue then I will try to dig a little deeper to find the problem. Any pointers would be useful if you can think of why there should be a difference.
I've never seen this behavior before, and tools installed from the Galaxy main tool shed have been running on the main Galaxy server for some time. No changes were made to the job components, so executing tools installed from tool sheds is no different in that regard from running tools included in the distribution. It's difficult to tell you where to begin looking for the cause of the problem.
Thanks for your help! Shaun
-- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
Quoting Greg Von Kuster greg@bx.psu.edu on Wed, 1 Aug 2012 06:23:11 -0400:
Hi Shaun,
On Aug 1, 2012, at 6:13 AM, SHAUN WEBB wrote:
c) The toolshed interface allows me to change the name of my tool. If I do this, then go to my Galaxy admin ui and try to get updates for my tool then it fails. Does it make sense to change a tools name after it is created? Is it possible to track previous tool names to avoid this happening?
This should not occur. Where are you able to change the name of your tool? You are not even allowed to change the name of the repository if it has been cloned (you can change the name of the repository as long as it has not been cloned). Can you send me a screen shot of hte page that allows you to change the name of your tool in the tool shed? I'll provide a fix as soon as I see where the issue lies.
OK, I guess I changed the name of my repository not tool. I then installed the tool via my local Galaxy instance. I can still change the name of the repository in my toolshed and when I try to get updates via Galaxy admin panel I get the following server error: "AttributeError: 'NoneType' object has no attribute 'repo_path'"
I cannot reproduce this behavior, so can you let me know the steps you use to do so? I need the page on which you are able to change the name of the tool shed repository in the tool shed after it has been cloned. Here is a screen shot of what should be occurring after the repository is cloned. Notice that the repository name is not alterable as it is no longer a text field.
abyss_tool Clone this repository: hg clone http://greg@toolshed.g2.bx.psu.edu/repos/msjeon/abyss_tool Name: abyss_tool Repository names cannot be changed if the repository has been cloned.
1. Create new repository in local toolshed and fill in name, synopsis, description and category 2. Upload a .pl script file then upload a .xml wrapper file 3. Go to my local Galaxy and install repository via admin panel 4. Return to toolshed, find repository and name etc are still editable
Screen shot attached. I noticed that "times downloaded" is still 0 after installing the repository. I have cloned the repository using hg on command line and this is still not updated. I'm guessing this is the problem.
As far as getting tools to run I will keep investigating and let you know if I find the issue.
Thanks! Shaun
Hi Shaun,
Thanks very much for the details - they have helped me discover a weakness which I've fixed. if the browser page is opened to the "Manage repository" page in the tool shed when the repository is cloned for the first time, the repository name could be changed after the clone. This has been fixed in change set 7441:513a95abb738.
However, your step 4 (return to toolshed, find repository) implies that this is not what you were doing. Can you update to the tip from Galaxy central and see if my latest fix still allows you to change the repository name after it has been cloned in your environment? If so, I'll need to figure out what you are doing differently that is still allowing for this.
Also, what database are you using? I found some issues with sqlite that I've fixed that would not eve have allowed for the repository to be cloned in the first place.
Thanks again!
Greg
On Aug 1, 2012, at 7:03 AM, SHAUN WEBB wrote:
Quoting Greg Von Kuster greg@bx.psu.edu on Wed, 1 Aug 2012 06:23:11 -0400:
I cannot reproduce this behavior, so can you let me know the steps you use to do so? I need the page on which you are able to change the name of the tool shed repository in the tool shed after it has been cloned.
- Create new repository in local toolshed and fill in name, synopsis, description and category
- Upload a .pl script file then upload a .xml wrapper file
- Go to my local Galaxy and install repository via admin panel
- Return to toolshed, find repository and name etc are still editable
Screen shot attached. I noticed that "times downloaded" is still 0 after installing the repository. I have cloned the repository using hg on command line and this is still not updated. I'm guessing this is the problem.
Hi Greg.
I had the tool shed and Galaxy opened in two tabs so this may have been the issue. I am using a mysql database.
I will check tis again when I next update Galaxy and let you know if there is a problem.
Thanks for your help Shaun
Quoting Greg Von Kuster greg@bx.psu.edu on Wed, 1 Aug 2012 11:24:44 -0400:
Hi Shaun,
Thanks very much for the details - they have helped me discover a weakness which I've fixed. if the browser page is opened to the "Manage repository" page in the tool shed when the repository is cloned for the first time, the repository name could be changed after the clone. This has been fixed in change set 7441:513a95abb738.
However, your step 4 (return to toolshed, find repository) implies that this is not what you were doing. Can you update to the tip from Galaxy central and see if my latest fix still allows you to change the repository name after it has been cloned in your environment? If so, I'll need to figure out what you are doing differently that is still allowing for this.
Also, what database are you using? I found some issues with sqlite that I've fixed that would not eve have allowed for the repository to be cloned in the first place.
Thanks again!
Greg
On Aug 1, 2012, at 7:03 AM, SHAUN WEBB wrote:
Quoting Greg Von Kuster greg@bx.psu.edu on Wed, 1 Aug 2012 06:23:11 -0400:
I cannot reproduce this behavior, so can you let me know the steps you use to do so? I need the page on which you are able to change the name of the tool shed repository in the tool shed after it has been cloned.
- Create new repository in local toolshed and fill in name,
synopsis, description and category 2. Upload a .pl script file then upload a .xml wrapper file 3. Go to my local Galaxy and install repository via admin panel 4. Return to toolshed, find repository and name etc are still editable
Screen shot attached. I noticed that "times downloaded" is still 0 after installing the repository. I have cloned the repository using hg on command line and this is still not updated. I'm guessing this is the problem.
Trackster:
I have noticed a few bugs with trackster in the latest release (tested on Chrome and Firefox):
- Overview button does not work
- If I add tracks to a group and make a composite track, then showindividual tracks again I get new menu options (filters, tool, tool parameter space visualisation). None of these seem to work and trackster often stops working after doing this.
These bugs have been fixed in galaxy-central and will be available in the next distribution.
Many thanks for reporting, J.
galaxy-dev@lists.galaxyproject.org
-
Dave Bouvier -
Greg Von Kuster -
Jeremy Goecks -
SHAUN WEBB