Hi all,
I've retitled this from "Composite datatype output for Cuffdiff" to "Implementing dataset collections" to try and link into Jeremy's thread. References: http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-February/013574.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-March/013634.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-March/013636.html
On Mon, Mar 4, 2013 at 4:08 AM, Alex.Khassapov@csiro.au wrote:
Hi John,
Are you saying that "composite multiple file dataset" isn't required and won't be implemented?
We are using your implementation of multifiles dataset ("m:xxx" type) and hope that eventually it will be pushed into main Galaxy implementation.
As we are using Galaxy for CT reconstruction tools, where input and output can consist of a couple thousand files, other options are not feasible, i.e. grouping datasets.
-Alex
On Mon, Mar 4, 2013 at 5:42 AM, John Chilton chilton@msi.umn.edu wrote:
Hi Alex,
Thanks for the comments. The galaxy team has made it clear here and to me privately that this will NOT be included in the Galaxy main code base. I hope and am I confident that they will make grouping datasets work, hopefully even to thousands of files.
I do not believe the two ideas are mutually exclusive and I will be maintaining a fork of galaxy-central with these additions, I will set this up this week hopefully. I will do my best to respond to support requests and make multiple file datasets and composite types in general as robust as possible, keep up with Galaxy updates, etc.... Obviously, it is risky to let a code base drift so far from galaxy main's however and you, me, and others who might want to use them will have to carefully weigh the risks when determining if multiple file datasets are worth the headache.
Thanks for all your help and inputs. I am sorry this did not turn out differently, I feel I have really failed here.
-John
Hi John,
Does your multiple file system work on composite datatypes? For example, given a 'blastdbn' file (several files making up a BLAST nucleotide database) could we use 'm:blastdbn' to refer to several BLAST databases? Is this at least possible in principle based on your work so far?
How different is your m:xxx grouping of multiple files of a given datatype at odds with the dataset collections idea as outlined by Jeremy Goecks here?
http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-February/013574.html https://trello.com/c/325AXIEr
Jeremy's idea mentions the special case of a unique dataset collection object for paired-end reads - well if that was defined using the existing core Galaxy functionality as a 'pairedfastq' composite datatype (made of two FASTQ files), then could we use 'm:pairedfastq' (with your enhancement) for a whole bunch of paired FASTQ files. Nice? :)
Regards,
Peter