Problem previewing a tool in the Toolshed
Hi, I'm currently developing my first Galaxy tool. I'm working on a wrapper for R's library cummeRbund[1]. I got to the point where the tool although incomplete on functionality, I think is ready to be uploaded to the Toolshed. I was able to do so. I also made sure I could then install it on my local instance using the toolshed repository. Everything is working as expected, except, the tool preview is not working correctly. There is no error, but the inputs, most of them conditional inputs, aren't being displayed correctly. This is the direct link to the tool in the test toolshed: http://testtoolshed.g2.bx.psu.edu/repository/manage_repository?operation=vie... [1]http://compbio.mit.edu/cummeRbund/ Kind regards, Carlos
Hi Carlos, Not every tool form parameter will properly display when previewing the tool in the tool shed because the tool shed doesn't have access to some Galaxy components (e.g., a Galaxy history ) that are required for some tool parameter types. When I get a chance, I'll take a closer look, but if your tool is functional when installed from the tool shed to a local Galaxy instance, all should be ok. Thanks for your contribution! Greg Von Kuster On Feb 14, 2012, at 1:49 PM, Carlos Borroto wrote:
Hi,
I'm currently developing my first Galaxy tool. I'm working on a wrapper for R's library cummeRbund[1]. I got to the point where the tool although incomplete on functionality, I think is ready to be uploaded to the Toolshed. I was able to do so. I also made sure I could then install it on my local instance using the toolshed repository.
Everything is working as expected, except, the tool preview is not working correctly. There is no error, but the inputs, most of them conditional inputs, aren't being displayed correctly. This is the direct link to the tool in the test toolshed: http://testtoolshed.g2.bx.psu.edu/repository/manage_repository?operation=vie...
[1]http://compbio.mit.edu/cummeRbund/
Kind regards, Carlos ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Greg, Thanks for your answer. Would be great to get the tool to preview correctly but is not a big deal. I'm having a few more problems. I'm getting this error from time to time when I do "hg push" or even when I delete all files from the toolshed and upload new ones: "Version information for the tools included in the cummerbund repository is missing. Reset all of this repository's metadata in the tool shed, then set the installed tool versions from the installed repository's Repository Actions menu." Doing a "Reset metadata" in the toolshed doesn't solve this issue. I can't find how to do "then set the installed tool versions from the installed repository's Repository Actions menu". I'm also having problem including a datatype with the tool. One of the outputs of the tool is a sqlite database. I have this datatypes_conf.xml: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes> Still sqlite doesn't appear as an option when I try to upload a file to Galaxy. The new datatype does appear if I choose the edit a dataset attributes. Lastly, when I try to upload a sqlite database I get this error: "An error occurred running this job: The uploaded binary file contains inappropriate content" $ file -i cummerbund.sqlite cummerbund.sqlite: application/octet-stream; charset=binary I was hoping by including a sqlite datatype this would be resolved. Thanks, Carlos On Tue, Feb 14, 2012 at 3:10 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
Not every tool form parameter will properly display when previewing the tool in the tool shed because the tool shed doesn't have access to some Galaxy components (e.g., a Galaxy history ) that are required for some tool parameter types. When I get a chance, I'll take a closer look, but if your tool is functional when installed from the tool shed to a local Galaxy instance, all should be ok.
Thanks for your contribution!
Greg Von Kuster
On Feb 14, 2012, at 1:49 PM, Carlos Borroto wrote:
Hi,
I'm currently developing my first Galaxy tool. I'm working on a wrapper for R's library cummeRbund[1]. I got to the point where the tool although incomplete on functionality, I think is ready to be uploaded to the Toolshed. I was able to do so. I also made sure I could then install it on my local instance using the toolshed repository.
Everything is working as expected, except, the tool preview is not working correctly. There is no error, but the inputs, most of them conditional inputs, aren't being displayed correctly. This is the direct link to the tool in the test toolshed: http://testtoolshed.g2.bx.psu.edu/repository/manage_repository?operation=vie...
[1]http://compbio.mit.edu/cummeRbund/
Kind regards, Carlos ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Carlos, I've uploaded a fix for the problem you saw when attempting to reset all metadata on your cummerbund tool. Thanks for reporting this issue. Looking at your change sets, it seems that you are attempting to use the public tool shed hosted by Penn State as a source code repository for your tool development efforts. If so, this is not the intent the public tool shed - it's intent is rather to share fully functional tools with others in the Galaxy community ("fully functional" implies that the tools was developed in an environment that included a Galaxy instance, and the tool proved to be functional within the Galaxy instance before it was uploaded to the tool shed). The tool shed inspects every change set and performs certain functions based on the content of the change set. For example, if a change set includes a tool, an attempt to load the tool will be made to generate information about the tool. Data types included in a change set are processed as well. Because of this, it is best to upload changes to the tool shed that contain development efforts that are in a "finished" state. See my additional inline comments and questions in your message. Thanks! Greg Von Kuster On Feb 15, 2012, at 2:24 PM, Carlos Borroto wrote:
Hi Greg,
Thanks for your answer. Would be great to get the tool to preview correctly but is not a big deal.
I'm having a few more problems. I'm getting this error from time to time when I do "hg push" or even when I delete all files from the toolshed and upload new ones: "Version information for the tools included in the cummerbund repository is missing. Reset all of this repository's metadata in the tool shed, then set the installed tool versions from the installed repository's Repository Actions menu."
The fix that I've pushed to the tool shed should hopefully resolve this issue.
Doing a "Reset metadata" in the toolshed doesn't solve this issue. I can't find how to do "then set the installed tool versions from the installed repository's Repository Actions menu".
I'm also having problem including a datatype with the tool. One of the outputs of the tool is a sqlite database. I have this datatypes_conf.xml: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes>
Still sqlite doesn't appear as an option when I try to upload a file to Galaxy. The new datatype does appear if I choose the edit a dataset attributes.
I'm not quite sure what your intentions are here, but attempting to make a Galaxy datatypes be a sqlite database sounds like it will be complex and fragile,
Lastly, when I try to upload a sqlite database I get this error: "An error occurred running this job: The uploaded binary file contains inappropriate content"
Uploading a sqlite database to the tool shed will certainly result in this behavior as our code inspectors will not handle this types o content.
$ file -i cummerbund.sqlite cummerbund.sqlite: application/octet-stream; charset=binary
I was hoping by including a sqlite datatype this would be resolved.
Thanks, Carlos
On Tue, Feb 14, 2012 at 3:10 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
Not every tool form parameter will properly display when previewing the tool in the tool shed because the tool shed doesn't have access to some Galaxy components (e.g., a Galaxy history ) that are required for some tool parameter types. When I get a chance, I'll take a closer look, but if your tool is functional when installed from the tool shed to a local Galaxy instance, all should be ok.
Thanks for your contribution!
Greg Von Kuster
On Feb 14, 2012, at 1:49 PM, Carlos Borroto wrote:
Hi,
I'm currently developing my first Galaxy tool. I'm working on a wrapper for R's library cummeRbund[1]. I got to the point where the tool although incomplete on functionality, I think is ready to be uploaded to the Toolshed. I was able to do so. I also made sure I could then install it on my local instance using the toolshed repository.
Everything is working as expected, except, the tool preview is not working correctly. There is no error, but the inputs, most of them conditional inputs, aren't being displayed correctly. This is the direct link to the tool in the test toolshed: http://testtoolshed.g2.bx.psu.edu/repository/manage_repository?operation=vie...
[1]http://compbio.mit.edu/cummeRbund/
Kind regards, Carlos ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Greg, Thanks for looking and resolving this issue. After deleting all the files in the repo and re-adding them, everything is working as expected. Please see my comments inline about the other two issues. On Tue, Feb 28, 2012 at 1:57 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
I've uploaded a fix for the problem you saw when attempting to reset all metadata on your cummerbund tool. Thanks for reporting this issue.
Looking at your change sets, it seems that you are attempting to use the public tool shed hosted by Penn State as a source code repository for your tool development efforts. If so, this is not the intent the public tool shed - it's intent is rather to share fully functional tools with others in the Galaxy community ("fully functional" implies that the tools was developed in an environment that included a Galaxy instance, and the tool proved to be functional within the Galaxy instance before it was uploaded to the tool shed).
The tool shed inspects every change set and performs certain functions based on the content of the change set. For example, if a change set includes a tool, an attempt to load the tool will be made to generate information about the tool. Data types included in a change set are processed as well. Because of this, it is best to upload changes to the tool shed that contain development efforts that are in a "finished" state.
See my additional inline comments and questions in your message.
Thanks!
Greg Von Kuster
On Feb 15, 2012, at 2:24 PM, Carlos Borroto wrote:
Hi Greg,
Thanks for your answer. Would be great to get the tool to preview correctly but is not a big deal.
I'm having a few more problems. I'm getting this error from time to time when I do "hg push" or even when I delete all files from the toolshed and upload new ones: "Version information for the tools included in the cummerbund repository is missing. Reset all of this repository's metadata in the tool shed, then set the installed tool versions from the installed repository's Repository Actions menu."
The fix that I've pushed to the tool shed should hopefully resolve this issue.
Doing a "Reset metadata" in the toolshed doesn't solve this issue. I can't find how to do "then set the installed tool versions from the installed repository's Repository Actions menu".
I'm also having problem including a datatype with the tool. One of the outputs of the tool is a sqlite database. I have this datatypes_conf.xml: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes>
Still sqlite doesn't appear as an option when I try to upload a file to Galaxy. The new datatype does appear if I choose the edit a dataset attributes.
I'm not quite sure what your intentions are here, but attempting to make a Galaxy datatypes be a sqlite database sounds like it will be complex and fragile,
One of the steps for cummeRbund library is the creation of a sqlite database. Generating this database is by far the step taking most of the processing time. cummeRbund author made it possible to pass as an input a previously generated sqlite database, this greatly cuts the processing time for subsequent analysis of the same cuffdiff ouput. I don't see why having a sqlite binary data type would be different than let say BigBed, BAM or h5 binary data types. Could you please comment on why you think it would be complex and fragile?.
Lastly, when I try to upload a sqlite database I get this error: "An error occurred running this job: The uploaded binary file contains inappropriate content"
Uploading a sqlite database to the tool shed will certainly result in this behavior as our code inspectors will not handle this types o content.
Sorry I didn't explained myself clearly, I'm not uploading the sqlite database to the tool shed. I'm uploading it to a history so I can use it with my newly developed tool. Kind regards, Carlos
Hi Carlos, You can certainly add support for a sqlite database datatype to your local galaxy instance. As you say, it may be no more complex / fragile than h5 or other binary datatypes - I've not gone far enough down the analysis path to provide good reasons why it may or may not be reasonable, so perhaps I spoke too soon. If you have questions about how to add support for new datatypes, see our wiki at http://wiki.g2.bx.psu.edu/Admin/Datatypes. Thanks! Greg Von Kuster On Feb 29, 2012, at 11:59 AM, Carlos Borroto wrote:
Hi Greg,
Thanks for looking and resolving this issue. After deleting all the files in the repo and re-adding them, everything is working as expected.
Please see my comments inline about the other two issues.
On Tue, Feb 28, 2012 at 1:57 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
I've uploaded a fix for the problem you saw when attempting to reset all metadata on your cummerbund tool. Thanks for reporting this issue.
Looking at your change sets, it seems that you are attempting to use the public tool shed hosted by Penn State as a source code repository for your tool development efforts. If so, this is not the intent the public tool shed - it's intent is rather to share fully functional tools with others in the Galaxy community ("fully functional" implies that the tools was developed in an environment that included a Galaxy instance, and the tool proved to be functional within the Galaxy instance before it was uploaded to the tool shed).
The tool shed inspects every change set and performs certain functions based on the content of the change set. For example, if a change set includes a tool, an attempt to load the tool will be made to generate information about the tool. Data types included in a change set are processed as well. Because of this, it is best to upload changes to the tool shed that contain development efforts that are in a "finished" state.
See my additional inline comments and questions in your message.
Thanks!
Greg Von Kuster
On Feb 15, 2012, at 2:24 PM, Carlos Borroto wrote:
Hi Greg,
Thanks for your answer. Would be great to get the tool to preview correctly but is not a big deal.
I'm having a few more problems. I'm getting this error from time to time when I do "hg push" or even when I delete all files from the toolshed and upload new ones: "Version information for the tools included in the cummerbund repository is missing. Reset all of this repository's metadata in the tool shed, then set the installed tool versions from the installed repository's Repository Actions menu."
The fix that I've pushed to the tool shed should hopefully resolve this issue.
Doing a "Reset metadata" in the toolshed doesn't solve this issue. I can't find how to do "then set the installed tool versions from the installed repository's Repository Actions menu".
I'm also having problem including a datatype with the tool. One of the outputs of the tool is a sqlite database. I have this datatypes_conf.xml: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes>
Still sqlite doesn't appear as an option when I try to upload a file to Galaxy. The new datatype does appear if I choose the edit a dataset attributes.
I'm not quite sure what your intentions are here, but attempting to make a Galaxy datatypes be a sqlite database sounds like it will be complex and fragile,
One of the steps for cummeRbund library is the creation of a sqlite database. Generating this database is by far the step taking most of the processing time. cummeRbund author made it possible to pass as an input a previously generated sqlite database, this greatly cuts the processing time for subsequent analysis of the same cuffdiff ouput. I don't see why having a sqlite binary data type would be different than let say BigBed, BAM or h5 binary data types. Could you please comment on why you think it would be complex and fragile?.
Lastly, when I try to upload a sqlite database I get this error: "An error occurred running this job: The uploaded binary file contains inappropriate content"
Uploading a sqlite database to the tool shed will certainly result in this behavior as our code inspectors will not handle this types o content.
Sorry I didn't explained myself clearly, I'm not uploading the sqlite database to the tool shed. I'm uploading it to a history so I can use it with my newly developed tool.
Kind regards, Carlos
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On Wed, Feb 29, 2012 at 5:14 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
You can certainly add support for a sqlite database datatype to your local galaxy instance. As you say, it may be no more complex / fragile than h5 or other binary datatypes - I've not gone far enough down the analysis path to provide good reasons why it may or may not be reasonable, so perhaps I spoke too soon. If you have questions about how to add support for new datatypes, see our wiki at http://wiki.g2.bx.psu.edu/Admin/Datatypes.
Thanks!
Greg Von Kuster
My impression is that as long as you treat it as a write once-data-read- many file using an SQLite3 database in Galaxy could work. By that I mean any tool can create and modify SQLite3 files as some of its output files, and any tool can also have any number of SQLite3 files as inputs - but must treat them as read only. The trouble will start if you actually use a pre-existing SQLite3 file like a database and make changes to it. That will break Galaxy's assumptions about how to reproduce the file for workflows etc. Perhaps Galaxy should actively preempt the possibility of tools editing old data files by making all the history data files read only right after the tool creating them has finished? This could catch some existing 'naughty' tools editing things they are not meant to. Peter
On Wed, Feb 29, 2012 at 12:28 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Feb 29, 2012 at 5:14 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
You can certainly add support for a sqlite database datatype to your local galaxy instance. As you say, it may be no more complex / fragile than h5 or other binary datatypes - I've not gone far enough down the analysis path to provide good reasons why it may or may not be reasonable, so perhaps I spoke too soon. If you have questions about how to add support for new datatypes, see our wiki at http://wiki.g2.bx.psu.edu/Admin/Datatypes.
Thanks!
Greg Von Kuster
My impression is that as long as you treat it as a write once-data-read- many file using an SQLite3 database in Galaxy could work. By that I mean any tool can create and modify SQLite3 files as some of its output files, and any tool can also have any number of SQLite3 files as inputs - but must treat them as read only.
The trouble will start if you actually use a pre-existing SQLite3 file like a database and make changes to it. That will break Galaxy's assumptions about how to reproduce the file for workflows etc.
Perhaps Galaxy should actively preempt the possibility of tools editing old data files by making all the history data files read only right after the tool creating them has finished? This could catch some existing 'naughty' tools editing things they are not meant to.
Peter
Hi Peter, This is a good point. I agree making sure the sqlite file is treated as read only would be a good idea. Thanks, Carlos
Hi Greg, I did follow that link and I think the datatypes_conf.xml in the tool repository is correctly written: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes> I thought by adding display_in_upload="true" that would make the datatype to appear as an option when uploading a file, but this is not happening. Do I need to declare type as for example "galaxy.datatypes.binary:Sqlite3" for this to work? If I have to, I'm guessing I would have to do what is explained here: http://wiki.g2.bx.psu.edu/Tool%20Shed#Including_proprietary_data_types_that_... Thanks, Carlos On Wed, Feb 29, 2012 at 12:14 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
You can certainly add support for a sqlite database datatype to your local galaxy instance. As you say, it may be no more complex / fragile than h5 or other binary datatypes - I've not gone far enough down the analysis path to provide good reasons why it may or may not be reasonable, so perhaps I spoke too soon. If you have questions about how to add support for new datatypes, see our wiki at http://wiki.g2.bx.psu.edu/Admin/Datatypes.
Thanks!
Greg Von Kuster
On Feb 29, 2012, at 11:59 AM, Carlos Borroto wrote:
Hi Greg,
Thanks for looking and resolving this issue. After deleting all the files in the repo and re-adding them, everything is working as expected.
Please see my comments inline about the other two issues.
On Tue, Feb 28, 2012 at 1:57 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
I've uploaded a fix for the problem you saw when attempting to reset all metadata on your cummerbund tool. Thanks for reporting this issue.
Looking at your change sets, it seems that you are attempting to use the public tool shed hosted by Penn State as a source code repository for your tool development efforts. If so, this is not the intent the public tool shed - it's intent is rather to share fully functional tools with others in the Galaxy community ("fully functional" implies that the tools was developed in an environment that included a Galaxy instance, and the tool proved to be functional within the Galaxy instance before it was uploaded to the tool shed).
The tool shed inspects every change set and performs certain functions based on the content of the change set. For example, if a change set includes a tool, an attempt to load the tool will be made to generate information about the tool. Data types included in a change set are processed as well. Because of this, it is best to upload changes to the tool shed that contain development efforts that are in a "finished" state.
See my additional inline comments and questions in your message.
Thanks!
Greg Von Kuster
On Feb 15, 2012, at 2:24 PM, Carlos Borroto wrote:
Hi Greg,
Thanks for your answer. Would be great to get the tool to preview correctly but is not a big deal.
I'm having a few more problems. I'm getting this error from time to time when I do "hg push" or even when I delete all files from the toolshed and upload new ones: "Version information for the tools included in the cummerbund repository is missing. Reset all of this repository's metadata in the tool shed, then set the installed tool versions from the installed repository's Repository Actions menu."
The fix that I've pushed to the tool shed should hopefully resolve this issue.
Doing a "Reset metadata" in the toolshed doesn't solve this issue. I can't find how to do "then set the installed tool versions from the installed repository's Repository Actions menu".
I'm also having problem including a datatype with the tool. One of the outputs of the tool is a sqlite database. I have this datatypes_conf.xml: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes>
Still sqlite doesn't appear as an option when I try to upload a file to Galaxy. The new datatype does appear if I choose the edit a dataset attributes.
I'm not quite sure what your intentions are here, but attempting to make a Galaxy datatypes be a sqlite database sounds like it will be complex and fragile,
One of the steps for cummeRbund library is the creation of a sqlite database. Generating this database is by far the step taking most of the processing time. cummeRbund author made it possible to pass as an input a previously generated sqlite database, this greatly cuts the processing time for subsequent analysis of the same cuffdiff ouput. I don't see why having a sqlite binary data type would be different than let say BigBed, BAM or h5 binary data types. Could you please comment on why you think it would be complex and fragile?.
Lastly, when I try to upload a sqlite database I get this error: "An error occurred running this job: The uploaded binary file contains inappropriate content"
Uploading a sqlite database to the tool shed will certainly result in this behavior as our code inspectors will not handle this types o content.
Sorry I didn't explained myself clearly, I'm not uploading the sqlite database to the tool shed. I'm uploading it to a history so I can use it with my newly developed tool.
Kind regards, Carlos
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Is your datatype lading into your Galaxy instance correctly (either when you start your Galaxy instance or when you install your tool from the tool shed)? If so, it should be displaying in the upload form. What does your paster log say? On Feb 29, 2012, at 1:06 PM, Carlos Borroto wrote:
Hi Greg,
I did follow that link and I think the datatypes_conf.xml in the tool repository is correctly written: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes>
I thought by adding display_in_upload="true" that would make the datatype to appear as an option when uploading a file, but this is not happening. Do I need to declare type as for example "galaxy.datatypes.binary:Sqlite3" for this to work? If I have to, I'm guessing I would have to do what is explained here: http://wiki.g2.bx.psu.edu/Tool%20Shed#Including_proprietary_data_types_that_...
Thanks, Carlos
On Wed, Feb 29, 2012 at 12:14 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
You can certainly add support for a sqlite database datatype to your local galaxy instance. As you say, it may be no more complex / fragile than h5 or other binary datatypes - I've not gone far enough down the analysis path to provide good reasons why it may or may not be reasonable, so perhaps I spoke too soon. If you have questions about how to add support for new datatypes, see our wiki at http://wiki.g2.bx.psu.edu/Admin/Datatypes.
Thanks!
Greg Von Kuster
On Feb 29, 2012, at 11:59 AM, Carlos Borroto wrote:
Hi Greg,
Thanks for looking and resolving this issue. After deleting all the files in the repo and re-adding them, everything is working as expected.
Please see my comments inline about the other two issues.
On Tue, Feb 28, 2012 at 1:57 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
I've uploaded a fix for the problem you saw when attempting to reset all metadata on your cummerbund tool. Thanks for reporting this issue.
Looking at your change sets, it seems that you are attempting to use the public tool shed hosted by Penn State as a source code repository for your tool development efforts. If so, this is not the intent the public tool shed - it's intent is rather to share fully functional tools with others in the Galaxy community ("fully functional" implies that the tools was developed in an environment that included a Galaxy instance, and the tool proved to be functional within the Galaxy instance before it was uploaded to the tool shed).
The tool shed inspects every change set and performs certain functions based on the content of the change set. For example, if a change set includes a tool, an attempt to load the tool will be made to generate information about the tool. Data types included in a change set are processed as well. Because of this, it is best to upload changes to the tool shed that contain development efforts that are in a "finished" state.
See my additional inline comments and questions in your message.
Thanks!
Greg Von Kuster
On Feb 15, 2012, at 2:24 PM, Carlos Borroto wrote:
Hi Greg,
Thanks for your answer. Would be great to get the tool to preview correctly but is not a big deal.
I'm having a few more problems. I'm getting this error from time to time when I do "hg push" or even when I delete all files from the toolshed and upload new ones: "Version information for the tools included in the cummerbund repository is missing. Reset all of this repository's metadata in the tool shed, then set the installed tool versions from the installed repository's Repository Actions menu."
The fix that I've pushed to the tool shed should hopefully resolve this issue.
Doing a "Reset metadata" in the toolshed doesn't solve this issue. I can't find how to do "then set the installed tool versions from the installed repository's Repository Actions menu".
I'm also having problem including a datatype with the tool. One of the outputs of the tool is a sqlite database. I have this datatypes_conf.xml: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes>
Still sqlite doesn't appear as an option when I try to upload a file to Galaxy. The new datatype does appear if I choose the edit a dataset attributes.
I'm not quite sure what your intentions are here, but attempting to make a Galaxy datatypes be a sqlite database sounds like it will be complex and fragile,
One of the steps for cummeRbund library is the creation of a sqlite database. Generating this database is by far the step taking most of the processing time. cummeRbund author made it possible to pass as an input a previously generated sqlite database, this greatly cuts the processing time for subsequent analysis of the same cuffdiff ouput. I don't see why having a sqlite binary data type would be different than let say BigBed, BAM or h5 binary data types. Could you please comment on why you think it would be complex and fragile?.
Lastly, when I try to upload a sqlite database I get this error: "An error occurred running this job: The uploaded binary file contains inappropriate content"
Uploading a sqlite database to the tool shed will certainly result in this behavior as our code inspectors will not handle this types o content.
Sorry I didn't explained myself clearly, I'm not uploading the sqlite database to the tool shed. I'm uploading it to a history so I can use it with my newly developed tool.
Kind regards, Carlos
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Greg, After restarting the galaxy instance to get a hold on the starting log, sqlite now shows up in the upload form. I was able to reproduce this behavior in a clean instance. The problem now is I'm back to getting: "An error occurred running this job: The uploaded binary file contains inappropriate content". These are the two relevant lines I can find in my paster log: galaxy.datatypes.registry DEBUG 2012-02-29 13:40:28,320 Loading datatypes from ../galaxy_shed_tools/testtoolshed.g2.bx.psu.edu/repos/cjav/cummerbund/1773e7dc45fe/cummerbund/datatypes_conf.xml galaxy.tools DEBUG 2012-02-29 13:40:38,155 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/cjav/cummerbund/cummerbund/0.0.4, version: 0.0.4. If you want me, I could upload the whole log and shared with you. The datatype does seem to be loading correctly, as I can edit a dataset and change the datatype to 'sqlite', also sqlite is now correctly appearing in the upload form as a possible datatype, although only after restarting the instance. My idea of using a new class modules is because the only different I can find with for example Scf datatype, is the use of a class module for the datatype. I confirmed that by renaming the sqlite file to .scf I can import it as scf without issues. I can them switch it to sqlite by editing the dataset. Thanks, Carlos On Wed, Feb 29, 2012 at 1:16 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Is your datatype lading into your Galaxy instance correctly (either when you start your Galaxy instance or when you install your tool from the tool shed)? If so, it should be displaying in the upload form. What does your paster log say?
On Feb 29, 2012, at 1:06 PM, Carlos Borroto wrote:
Hi Greg,
I did follow that link and I think the datatypes_conf.xml in the tool repository is correctly written: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes>
I thought by adding display_in_upload="true" that would make the datatype to appear as an option when uploading a file, but this is not happening. Do I need to declare type as for example "galaxy.datatypes.binary:Sqlite3" for this to work? If I have to, I'm guessing I would have to do what is explained here: http://wiki.g2.bx.psu.edu/Tool%20Shed#Including_proprietary_data_types_that_...
Thanks, Carlos
On Wed, Feb 29, 2012 at 12:14 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
You can certainly add support for a sqlite database datatype to your local galaxy instance. As you say, it may be no more complex / fragile than h5 or other binary datatypes - I've not gone far enough down the analysis path to provide good reasons why it may or may not be reasonable, so perhaps I spoke too soon. If you have questions about how to add support for new datatypes, see our wiki at http://wiki.g2.bx.psu.edu/Admin/Datatypes.
Thanks!
Greg Von Kuster
On Feb 29, 2012, at 11:59 AM, Carlos Borroto wrote:
Hi Greg,
Thanks for looking and resolving this issue. After deleting all the files in the repo and re-adding them, everything is working as expected.
Please see my comments inline about the other two issues.
On Tue, Feb 28, 2012 at 1:57 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Carlos,
I've uploaded a fix for the problem you saw when attempting to reset all metadata on your cummerbund tool. Thanks for reporting this issue.
Looking at your change sets, it seems that you are attempting to use the public tool shed hosted by Penn State as a source code repository for your tool development efforts. If so, this is not the intent the public tool shed - it's intent is rather to share fully functional tools with others in the Galaxy community ("fully functional" implies that the tools was developed in an environment that included a Galaxy instance, and the tool proved to be functional within the Galaxy instance before it was uploaded to the tool shed).
The tool shed inspects every change set and performs certain functions based on the content of the change set. For example, if a change set includes a tool, an attempt to load the tool will be made to generate information about the tool. Data types included in a change set are processed as well. Because of this, it is best to upload changes to the tool shed that contain development efforts that are in a "finished" state.
See my additional inline comments and questions in your message.
Thanks!
Greg Von Kuster
On Feb 15, 2012, at 2:24 PM, Carlos Borroto wrote:
Hi Greg,
Thanks for your answer. Would be great to get the tool to preview correctly but is not a big deal.
I'm having a few more problems. I'm getting this error from time to time when I do "hg push" or even when I delete all files from the toolshed and upload new ones: "Version information for the tools included in the cummerbund repository is missing. Reset all of this repository's metadata in the tool shed, then set the installed tool versions from the installed repository's Repository Actions menu."
The fix that I've pushed to the tool shed should hopefully resolve this issue.
Doing a "Reset metadata" in the toolshed doesn't solve this issue. I can't find how to do "then set the installed tool versions from the installed repository's Repository Actions menu".
I'm also having problem including a datatype with the tool. One of the outputs of the tool is a sqlite database. I have this datatypes_conf.xml: <?xml version="1.0"?> <datatypes> <registration> <datatype extension="sqlite" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" subclass="True" display_in_upload="true"/> </registration> </datatypes>
Still sqlite doesn't appear as an option when I try to upload a file to Galaxy. The new datatype does appear if I choose the edit a dataset attributes.
I'm not quite sure what your intentions are here, but attempting to make a Galaxy datatypes be a sqlite database sounds like it will be complex and fragile,
One of the steps for cummeRbund library is the creation of a sqlite database. Generating this database is by far the step taking most of the processing time. cummeRbund author made it possible to pass as an input a previously generated sqlite database, this greatly cuts the processing time for subsequent analysis of the same cuffdiff ouput. I don't see why having a sqlite binary data type would be different than let say BigBed, BAM or h5 binary data types. Could you please comment on why you think it would be complex and fragile?.
Lastly, when I try to upload a sqlite database I get this error: "An error occurred running this job: The uploaded binary file contains inappropriate content"
Uploading a sqlite database to the tool shed will certainly result in this behavior as our code inspectors will not handle this types o content.
Sorry I didn't explained myself clearly, I'm not uploading the sqlite database to the tool shed. I'm uploading it to a history so I can use it with my newly developed tool.
Kind regards, Carlos
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (3)
-
Carlos Borroto -
Greg Von Kuster -
Peter Cock