Managing large database lists in tool wrappers?
Does anyone have a workable solution to managing and displaying large lists in tool wrappers? eg. a database list containing hundreds of enteries in the blast+ tool? (we have 1,056 different blast databases). As a work-around we've split our blast databases into species (eg cow, sheep, fungus, clover, fish, rice, etc) then faked a multi-select to pull in those sub-database files with conditional selectors like this: <conditional name="db_opts"> <param name="db_opts_selector" type="select" label="Nucleotide Database Type"> <option value="pub" selected="True">Pub Databases</option> <option value="cattle">Cattle Databases</option> <option value="sheep">Sheep Databases</option> <option value="deer">Deer Databases</option> <option value="ruminant">Ruminant Databases</option> <option value="dog">Dog Databases</option> <option value="fungi">Fungi Databases</option> <option value="invertebrates">Invertebrates Databases</option> <option value="microarrays">Microarrays Databases</option> <option value="marsupials">Marsupials Databases</option> <option value="rodent">Rodent Databases</option> <option value="viruses">Viruses Databases</option> <option value="human">Human Databases</option> <option value="microarrays">Microarrays Databases</option> </param> <when value="ruminant"> <param name="database" type="select" label="Ruminant Nucleotide BLAST database" help=""> <options from_file="blastdb.ruminant.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when> <when value="misc"> <param name="database" type="select" label="Misc Nucleotide BLAST database" help=""> <options from_file="blastdb.misc.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when> We end up with a lot of duplication between tool wrappers (I've never managed to get an <include> tag working) and these aren't exactly simple to maintain so does anyone have a better solution? I had thought about tacking an extra field onto the .loc files with a "species" and doing some grouping with that but haven't progressed that idea very far yet. Any suggestions greatly appreciated :) Thanx, --Russell
Hi Russell, would something like a http://ivaynberg.github.io/select2/ field work for you? What do you mean with <include> tags? Are you aware of Galaxy macros? https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#Direct_XML_Macro... Cheers, Bjoern Am 31.03.2014 22:58, schrieb Smithies, Russell:
Does anyone have a workable solution to managing and displaying large lists in tool wrappers? eg. a database list containing hundreds of enteries in the blast+ tool? (we have 1,056 different blast databases). As a work-around we've split our blast databases into species (eg cow, sheep, fungus, clover, fish, rice, etc) then faked a multi-select to pull in those sub-database files with conditional selectors like this:
<conditional name="db_opts"> <param name="db_opts_selector" type="select" label="Nucleotide Database Type"> <option value="pub" selected="True">Pub Databases</option> <option value="cattle">Cattle Databases</option> <option value="sheep">Sheep Databases</option> <option value="deer">Deer Databases</option> <option value="ruminant">Ruminant Databases</option> <option value="dog">Dog Databases</option> <option value="fungi">Fungi Databases</option> <option value="invertebrates">Invertebrates Databases</option> <option value="microarrays">Microarrays Databases</option> <option value="marsupials">Marsupials Databases</option> <option value="rodent">Rodent Databases</option> <option value="viruses">Viruses Databases</option> <option value="human">Human Databases</option> <option value="microarrays">Microarrays Databases</option> </param>
<when value="ruminant"> <param name="database" type="select" label="Ruminant Nucleotide BLAST database" help=""> <options from_file="blastdb.ruminant.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when>
<when value="misc"> <param name="database" type="select" label="Misc Nucleotide BLAST database" help=""> <options from_file="blastdb.misc.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when>
We end up with a lot of duplication between tool wrappers (I've never managed to get an <include> tag working) and these aren't exactly simple to maintain so does anyone have a better solution? I had thought about tacking an extra field onto the .loc files with a "species" and doing some grouping with that but haven't progressed that idea very far yet.
Any suggestions greatly appreciated :)
Thanx,
--Russell
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Bjoern, Yes I'm aware of the macros but I've never managed to get them to work ;-( What I'd like is something like a php include tag to pull the contents of a whole file into another file so I'm not repeating myself eg. for the database lists in the megablast, blastn, and tblastn wrappers. Those select fields are nice, and in "ordinary" html and JQuery you can do grouping but as far as I can tell it's not possible with Galaxy/Cheetah XML templating - though we are a few versions behind so may have overlooked recent updates. Thanx for your suggestions, --Russell -----Original Message----- From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Björn Grüning Sent: Tuesday, 1 April 2014 10:19 a.m. To: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Managing large database lists in tool wrappers? Hi Russell, would something like a http://ivaynberg.github.io/select2/ field work for you? What do you mean with <include> tags? Are you aware of Galaxy macros? https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#Direct_XML_Macro... Cheers, Bjoern Am 31.03.2014 22:58, schrieb Smithies, Russell:
Does anyone have a workable solution to managing and displaying large lists in tool wrappers? eg. a database list containing hundreds of enteries in the blast+ tool? (we have 1,056 different blast databases). As a work-around we've split our blast databases into species (eg cow, sheep, fungus, clover, fish, rice, etc) then faked a multi-select to pull in those sub-database files with conditional selectors like this:
<conditional name="db_opts"> <param name="db_opts_selector" type="select" label="Nucleotide Database Type"> <option value="pub" selected="True">Pub Databases</option> <option value="cattle">Cattle Databases</option> <option value="sheep">Sheep Databases</option> <option value="deer">Deer Databases</option> <option value="ruminant">Ruminant Databases</option> <option value="dog">Dog Databases</option> <option value="fungi">Fungi Databases</option> <option value="invertebrates">Invertebrates Databases</option> <option value="microarrays">Microarrays Databases</option> <option value="marsupials">Marsupials Databases</option> <option value="rodent">Rodent Databases</option> <option value="viruses">Viruses Databases</option> <option value="human">Human Databases</option> <option value="microarrays">Microarrays Databases</option> </param>
<when value="ruminant"> <param name="database" type="select" label="Ruminant Nucleotide BLAST database" help=""> <options from_file="blastdb.ruminant.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when>
<when value="misc"> <param name="database" type="select" label="Misc Nucleotide BLAST database" help=""> <options from_file="blastdb.misc.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when>
We end up with a lot of duplication between tool wrappers (I've never managed to get an <include> tag working) and these aren't exactly simple to maintain so does anyone have a better solution? I had thought about tacking an extra field onto the .loc files with a "species" and doing some grouping with that but haven't progressed that idea very far yet.
Any suggestions greatly appreciated :)
Thanx,
--Russell
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Russell, Am 31.03.2014 23:46, schrieb Smithies, Russell:
Hi Bjoern, Yes I'm aware of the macros but I've never managed to get them to work ;-(
Lets try to get them to work. It will do exactly what you want. The Homer Tools should be a easy enough to get you started: https://github.com/bgruening/galaxytools/tree/master/homer/tools
What I'd like is something like a php include tag to pull the contents of a whole file into another file so I'm not repeating myself eg. for the database lists in the megablast, blastn, and tblastn wrappers.
Those select fields are nice, and in "ordinary" html and JQuery you can do grouping but as far as I can tell it's not possible with Galaxy/Cheetah XML templating - though we are a few versions behind so may have overlooked recent updates.
Not yet, but I think it would be easy to implement it. It's used in other areas of Galaxy a lot. Ciao, Bjoern
Thanx for your suggestions,
--Russell
-----Original Message----- From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Björn Grüning Sent: Tuesday, 1 April 2014 10:19 a.m. To: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Managing large database lists in tool wrappers?
Hi Russell,
would something like a http://ivaynberg.github.io/select2/ field work for you? What do you mean with <include> tags? Are you aware of Galaxy macros? https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#Direct_XML_Macro...
Cheers, Bjoern
Am 31.03.2014 22:58, schrieb Smithies, Russell:
Does anyone have a workable solution to managing and displaying large lists in tool wrappers? eg. a database list containing hundreds of enteries in the blast+ tool? (we have 1,056 different blast databases). As a work-around we've split our blast databases into species (eg cow, sheep, fungus, clover, fish, rice, etc) then faked a multi-select to pull in those sub-database files with conditional selectors like this:
<conditional name="db_opts"> <param name="db_opts_selector" type="select" label="Nucleotide Database Type"> <option value="pub" selected="True">Pub Databases</option> <option value="cattle">Cattle Databases</option> <option value="sheep">Sheep Databases</option> <option value="deer">Deer Databases</option> <option value="ruminant">Ruminant Databases</option> <option value="dog">Dog Databases</option> <option value="fungi">Fungi Databases</option> <option value="invertebrates">Invertebrates Databases</option> <option value="microarrays">Microarrays Databases</option> <option value="marsupials">Marsupials Databases</option> <option value="rodent">Rodent Databases</option> <option value="viruses">Viruses Databases</option> <option value="human">Human Databases</option> <option value="microarrays">Microarrays Databases</option> </param>
<when value="ruminant"> <param name="database" type="select" label="Ruminant Nucleotide BLAST database" help=""> <options from_file="blastdb.ruminant.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when>
<when value="misc"> <param name="database" type="select" label="Misc Nucleotide BLAST database" help=""> <options from_file="blastdb.misc.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when>
We end up with a lot of duplication between tool wrappers (I've never managed to get an <include> tag working) and these aren't exactly simple to maintain so does anyone have a better solution? I had thought about tacking an extra field onto the .loc files with a "species" and doing some grouping with that but haven't progressed that idea very far yet.
Any suggestions greatly appreciated :)
Thanx,
--Russell
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Those macros look great and are exactly what I want :-) Thanx, --Russell -----Original Message----- From: Björn Grüning [mailto:bjoern.gruening@gmail.com] Sent: Tuesday, 1 April 2014 11:07 a.m. To: Smithies, Russell; Björn Grüning; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Managing large database lists in tool wrappers? Hi Russell, Am 31.03.2014 23:46, schrieb Smithies, Russell:
Hi Bjoern, Yes I'm aware of the macros but I've never managed to get them to work ;-(
Lets try to get them to work. It will do exactly what you want. The Homer Tools should be a easy enough to get you started: https://github.com/bgruening/galaxytools/tree/master/homer/tools
What I'd like is something like a php include tag to pull the contents of a whole file into another file so I'm not repeating myself eg. for the database lists in the megablast, blastn, and tblastn wrappers.
Those select fields are nice, and in "ordinary" html and JQuery you can do grouping but as far as I can tell it's not possible with Galaxy/Cheetah XML templating - though we are a few versions behind so may have overlooked recent updates.
Not yet, but I think it would be easy to implement it. It's used in other areas of Galaxy a lot. Ciao, Bjoern
Thanx for your suggestions,
--Russell
-----Original Message----- From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Björn Grüning Sent: Tuesday, 1 April 2014 10:19 a.m. To: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Managing large database lists in tool wrappers?
Hi Russell,
would something like a http://ivaynberg.github.io/select2/ field work for you? What do you mean with <include> tags? Are you aware of Galaxy macros? https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#Direct_XML _Macros
Cheers, Bjoern
Am 31.03.2014 22:58, schrieb Smithies, Russell:
Does anyone have a workable solution to managing and displaying large lists in tool wrappers? eg. a database list containing hundreds of enteries in the blast+ tool? (we have 1,056 different blast databases). As a work-around we've split our blast databases into species (eg cow, sheep, fungus, clover, fish, rice, etc) then faked a multi-select to pull in those sub-database files with conditional selectors like this:
<conditional name="db_opts"> <param name="db_opts_selector" type="select" label="Nucleotide Database Type"> <option value="pub" selected="True">Pub Databases</option> <option value="cattle">Cattle Databases</option> <option value="sheep">Sheep Databases</option> <option value="deer">Deer Databases</option> <option value="ruminant">Ruminant Databases</option> <option value="dog">Dog Databases</option> <option value="fungi">Fungi Databases</option> <option value="invertebrates">Invertebrates Databases</option> <option value="microarrays">Microarrays Databases</option> <option value="marsupials">Marsupials Databases</option> <option value="rodent">Rodent Databases</option> <option value="viruses">Viruses Databases</option> <option value="human">Human Databases</option> <option value="microarrays">Microarrays Databases</option> </param>
<when value="ruminant"> <param name="database" type="select" label="Ruminant Nucleotide BLAST database" help=""> <options from_file="blastdb.ruminant.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when>
<when value="misc"> <param name="database" type="select" label="Misc Nucleotide BLAST database" help=""> <options from_file="blastdb.misc.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> <param name="subject" type="hidden" value="" /> </when>
We end up with a lot of duplication between tool wrappers (I've never managed to get an <include> tag working) and these aren't exactly simple to maintain so does anyone have a better solution? I had thought about tacking an extra field onto the .loc files with a "species" and doing some grouping with that but haven't progressed that idea very far yet.
Any suggestions greatly appreciated :)
Thanx,
--Russell
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On 31.03.2014 22:58, Smithies, Russell:
Does anyone have a workable solution to managing and displaying large lists in tool wrappers? eg. a database list containing hundreds of enteries in the blast+ tool? (we have 1,056 different blast databases).
On Mon, Mar 31, 2014 at 10:18 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi Russell,
would something like a http://ivaynberg.github.io/select2/ field work for you?
Which bit of that do you mean Bjoern? Galaxy already does searching within the drop down box (when the list is above some threshold in length) which is very handy. Mind you, we currently have under a hundred BLAST databases defined in our *.loc files, so this is an order of magnitude less than Russell's setup.
What do you mean with <include> tags? Are you aware of Galaxy macros? https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#Direct_XML_Macro...
Cheers, Bjoern
Since it looks like you are modifying the BLAST+ wrappers, they are already using Galaxy macros - you probably only need to change the ncbi_macros.xml file once and have this work on all of BLASTP, BLASTN, etc. i.e. Modify the input_conditional_nucleotide_db and input_conditional_protein_db definitions (and potentially the token @BLAST_DB_SUBJECT@ where those parameters are inserted in the the Cheetah command line template): https://github.com/peterjc/galaxy_blast/blob/master/tools/ncbi_blast_plus/nc... Regards, Peter
Hi,
On 31.03.2014 22:58, Smithies, Russell:
Does anyone have a workable solution to managing and displaying large lists in tool wrappers? eg. a database list containing hundreds of enteries in the blast+ tool? (we have 1,056 different blast databases).
On Mon, Mar 31, 2014 at 10:18 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi Russell,
would something like a http://ivaynberg.github.io/select2/ field work for you?
Which bit of that do you mean Bjoern? Galaxy already does searching within the drop down box (when the list is above some threshold in length) which is very handy.
Yes, that was my intention. To point out that something like that should be possible. It new for me that there is a threshold to enable that mechanism.
Mind you, we currently have under a hundred BLAST databases defined in our *.loc files, so this is an order of magnitude less than Russell's setup.
We also have only 3x. So no problem until now. Ciao, Bjoern
What do you mean with <include> tags? Are you aware of Galaxy macros? https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#Direct_XML_Macro...
Cheers, Bjoern
Since it looks like you are modifying the BLAST+ wrappers, they are already using Galaxy macros - you probably only need to change the ncbi_macros.xml file once and have this work on all of BLASTP, BLASTN, etc.
i.e. Modify the input_conditional_nucleotide_db and input_conditional_protein_db definitions (and potentially the token @BLAST_DB_SUBJECT@ where those parameters are inserted in the the Cheetah command line template):
https://github.com/peterjc/galaxy_blast/blob/master/tools/ncbi_blast_plus/nc...
Regards,
Peter
participants (3)
-
Björn Grüning -
Peter Cock -
Smithies, Russell