It's actually in related to a new galaxy shed tool we developed called meme-chip, we uploaded it to the main Tool Shed site.

We provided tool_data_table_conf.xml.sample and xxx.loc.sample along with the tool. After the installation, the loc file has been populated automatically, but the entry inside tool_data_table_conf.xml.sample didn't get populated.

Although it can be fixed easily by adding entries to the runtime tool_data_table_conf.xml manually, it's good to know if it's the right way or we are missing something in the shed tool repos.

Cheers, D

On Thu, Nov 8, 2012 at 10:20 AM, Derrick Lin klin938@gmail.com wrote:

Hi guys,

After installing the BWA from the Tools Shed, I find there is a new tool_data_table_conf.xml got created under tool-data directory. The file contains the entries of BWA's loc files.

I believe the installed BWA still relies on the tool_data_table_conf.xml under galaxy root dir. Anyone can clarify what the one under tool-data dir is for?

Regards, Derrick

Thanks Greg,

Very nice to hear they are all fixed now. Will try and verify when they are released.

Cheers, D

On Fri, Nov 9, 2012 at 2:10 AM, Greg Von Kuster greg@bx.psu.edu wrote:

Hello Derrick,

Your message discusses 2 separate issues:

1. When you install your meme_chip repository from the tool shed, the file

named tool_data_table_conf.xml.sample contained in the repository is copied to the ~/tool-data directory in the local Galaxy instance. This issue has been corrected in change set b5ce9451c5d1 which I committed this morning to the galaxy-central repository.

2. When you install your meme_chip repository from the tool shed, the

entries defined in the file named tool_data_table_conf.xml.sample contained in the repository are not entered into the tool_data_table_conf.xml file in the local Galaxy instance. This issue was corrected in change set bc5fa254bafc which I committed to the galaxy-central repository yesterday. With this change set, entries for data tables that are defined in tool_data_table_conf.xml.sample files included in the tool shed repository will be automatically added to a new separate file named shed_tool_data_table_conf.xml. This will ensure that manual edits made to the original tool_data_table_conf.xml file will not be munged when Galaxy's tool shed repository installation process automatically adds entries into the file.

Unfortunately both of these fixes have not made it into the Galaxy release that is currently planned for this week, so they will be available in the release currently scheduled for 2 weeks from now.

Thanks very much for reporting both of these issues, and sorry for the inconvenience they may have caused.

Greg Von Kuster

On Nov 7, 2012, at 8:19 PM, Derrick Lin wrote:

It's actually in related to a new galaxy shed tool we developed called meme-chip, we uploaded it to the main Tool Shed site.

We provided tool_data_table_conf.xml.sample and xxx.loc.sample along with the tool. After the installation, the loc file has been populated automatically, but the entry inside tool_data_table_conf.xml.sample didn't get populated.

Although it can be fixed easily by adding entries to the runtime tool_data_table_conf.xml manually, it's good to know if it's the right way or we are missing something in the shed tool repos.

Cheers, D

On Thu, Nov 8, 2012 at 10:20 AM, Derrick Lin klin938@gmail.com wrote:

...

Hi guys,

After installing the BWA from the Tools Shed, I find there is a new tool_data_table_conf.xml got created under tool-data directory. The file contains the entries of BWA's loc files.

I believe the installed BWA still relies on the tool_data_table_conf.xml under galaxy root dir. Anyone can clarify what the one under tool-data dir is for?

Regards, Derrick

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

Hi Derrick,

On Nov 9, 2012, at 12:56 AM, Derrick Lin wrote:

For issue 1), so I think it's safe to remove the tool-data/tool_data_table_conf.xml & tool-data/tool_data_table_conf.xml.sample since they are not used?

Yes, the ~/tool-data/tool_data_table_conf.xml.sample and ~/tool-data/tool_data_table_conf.xml.sample files can be removed.

For issue 2) if the galaxy will automatically pick up the new file shed_tool_data_table_conf.xml without configuring the ini file?

Yes, no changes are necessarey in your universe_wsgi.ini file unless you want to name these particular files differently than these default names. The change set I committed includes entries in universe_wsgi.ini.sample for both of these files (the tool_data_table_conf.xml file was never added to the sample ini), but adding the new settings to the universe_wsgi.ini file is not required as the default names will be used.

Also how does this change affect the existing shed tools like BWA that is using tool_data_table_conf.xml?

The existing shed tools will contiue to function as usual, but entries for them will be in the tool_data_table_conf.xml file. ANy new tools installed from the tool shed that use entries like this will be added to the new shed_tool_data_table_conf.xml file. Entries from both of these files are loaded into the ToolDataTableManager. The intent is for the Galaxy admin to be able to manuall make changes to the tool_data_table_conf.xml file and not have them munged by the tool shed's installation process. This is the same approach used for the tool_conf.xml file, where the Galaxy tool shed repository installation process uses the shed_tool_conf.xml file.

Thanks Derrick,

Greg Von Kuster

Cheers, D

On Fri, Nov 9, 2012 at 2:10 AM, Greg Von Kuster greg@bx.psu.edu wrote: Hello Derrick,

Your message discusses 2 separate issues:

1. When you install your meme_chip repository from the tool shed, the file named tool_data_table_conf.xml.sample contained in the repository is copied to the ~/tool-data directory in the local Galaxy instance. This issue has been corrected in change set b5ce9451c5d1 which I committed this morning to the galaxy-central repository.

2. When you install your meme_chip repository from the tool shed, the entries defined in the file named tool_data_table_conf.xml.sample contained in the repository are not entered into the tool_data_table_conf.xml file in the local Galaxy instance. This issue was corrected in change set bc5fa254bafc which I committed to the galaxy-central repository yesterday. With this change set, entries for data tables that are defined in tool_data_table_conf.xml.sample files included in the tool shed repository will be automatically added to a new separate file named shed_tool_data_table_conf.xml. This will ensure that manual edits made to the original tool_data_table_conf.xml file will not be munged when Galaxy's tool shed repository installation process automatically adds entries into the file.

Unfortunately both of these fixes have not made it into the Galaxy release that is currently planned for this week, so they will be available in the release currently scheduled for 2 weeks from now.

Thanks very much for reporting both of these issues, and sorry for the inconvenience they may have caused.

Greg Von Kuster

On Nov 7, 2012, at 8:19 PM, Derrick Lin wrote:

...

It's actually in related to a new galaxy shed tool we developed called meme-chip, we uploaded it to the main Tool Shed site.

We provided tool_data_table_conf.xml.sample and xxx.loc.sample along with the tool. After the installation, the loc file has been populated automatically, but the entry inside tool_data_table_conf.xml.sample didn't get populated.

Although it can be fixed easily by adding entries to the runtime tool_data_table_conf.xml manually, it's good to know if it's the right way or we are missing something in the shed tool repos.

Cheers, D

On Thu, Nov 8, 2012 at 10:20 AM, Derrick Lin klin938@gmail.com wrote: Hi guys,

After installing the BWA from the Tools Shed, I find there is a new tool_data_table_conf.xml got created under tool-data directory. The file contains the entries of BWA's loc files.

I believe the installed BWA still relies on the tool_data_table_conf.xml under galaxy root dir. Anyone can clarify what the one under tool-data dir is for?

Regards, Derrick

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

Greg Von Kuster <greg@...> writes:

On Nov 9, 2012, at 12:56 AM, Derrick Lin wrote:

...

Also how does this change affect the existing shed tools like BWA that

is using tool_data_table_conf.xml?

The existing shed tools will contiue to function as usual, but entries for

them will be in the tool_data_table_conf.xml file.  ANy new tools installed from the tool shed that use entries like this will be added to the new shed_tool_data_table_conf.xml file.  Entries from both of these files are loaded into the ToolDataTableManager.  The intent is for the Galaxy admin to be able to manuall make changes to the tool_data_table_conf.xml file and not have them munged by the tool shed's installation process.  This is the same approach used for the tool_conf.xml file, where the Galaxy tool shed repository installation process uses the shed_tool_conf.xml file.

Reviving an old thread...

I noticed that when I install a repository from the Tool Shed which contains a tool_data_table_conf.xml.sample , the sample files mentioned therein are copied to: - tool-data/ (with AND without .sample extension) - tool-data/toolshed.g2.bx.psu.edu/repos/OWNER/REPO/REVISION/ (without .sample extension) The tool_data_table_conf.xml file in tool-data/toolshed.g2.bx.psu.edu/repos/OWNER/REPO/REVISION/ references the files in the same directory, so I find the duplication confusing.

Moreover, there is an increasing number of repositories which install the same .loc files, e.g.: - the 4 cufflinks tools plus sam_pileup, sam_to_bam and samtools_mpileup install fasta_indexes.loc ; - blat, lastz and lastz_paired_reads install lastz_seqs.loc . I suppose it was made to prevent repositories from getting in each other's way, but I see 2 problems: - system administrators have to manually hard-link these files; - duplications in shed_tool_data_table_conf.xml generate errors like: "galaxy.tools.data ERROR 2014-01-17 15:29:41,409 Attempted to add fields (['hg19full', 'hg19', 'Human Feb. 2009 (GRCh37/hg19) (hg19) Full', '/mnt/genome/hg19/sam_index/hg19full/hg19full.fa']) to data table 'fasta_indexes', but this entry already exists and allow_duplicates is False."

Suggestions?

Thanks, Nicola