Is dynamic associated information per dataset possible?
We'd like to be able to associate fixed things (project, Sample, sequencer used) with user's FASTQ files, and we'd also like to allow users to associate dynamic, site-specific stuff with the sequencing run. Currently, users track their runs using a CSV sample sheet, and often they add columns to that sample sheet for their own information. Is it possible to associate that information with the FASTQ file when it is placed in Galaxy? I know about metadata, but the supported fields look like they are fixed in the code. I was hoping for a solution where the users do not need to modify the Galaxy code to pull this off. John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com<mailto:jduddy@illumina.com>
Duddy, John wrote:
We'd like to be able to associate fixed things (project, Sample, sequencer used) with user's FASTQ files, and we'd also like to allow users to associate dynamic, site-specific stuff with the sequencing run. Currently, users track their runs using a CSV sample sheet, and often they add columns to that sample sheet for their own information.
Is it possible to associate that information with the FASTQ file when it is placed in Galaxy? I know about metadata, but the supported fields look like they are fixed in the code. I was hoping for a solution where the users do not need to modify the Galaxy code to pull this off.
Hi John, You could accomplish this with library templates: https://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/LibraryTempla... --nate
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com<mailto:jduddy@illumina.com>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
In addition to Data Library templates, which are useful after the sequencer data has arrived in a Galaxy Data Library, Galaxy's sample tracking system includes sample run templates which are very similar to the Data Library templates, but are associated with a sample as it progresses through it's sequence run lifecycle in the facility. Sample run details templates are defined by the facility administrator. They can be created in the Admin view via the Manage form definitions menu link. On May 26, 2011, at 4:38 AM, Nate Coraor wrote:
Duddy, John wrote:
We'd like to be able to associate fixed things (project, Sample, sequencer used) with user's FASTQ files, and we'd also like to allow users to associate dynamic, site-specific stuff with the sequencing run. Currently, users track their runs using a CSV sample sheet, and often they add columns to that sample sheet for their own information.
Is it possible to associate that information with the FASTQ file when it is placed in Galaxy? I know about metadata, but the supported fields look like they are fixed in the code. I was hoping for a solution where the users do not need to modify the Galaxy code to pull this off.
Hi John,
You could accomplish this with library templates:
https://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/LibraryTempla...
--nate
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com<mailto:jduddy@illumina.com>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Greg Von Kuster Galaxy Development Team greg@bx.psu.edu
The sample tracking system looks interesting. It looks like this is designed to have the sequencers modeled in Galaxy, have Galaxy pull the data from the sequencers, and might assume 1-1 pairing between sequencing run and samples. I'd like to be able to support pushing files from a central location and variably multiplexed runs, all via the API (or extensions to the API). Is there any up-to-date documentation on this feature? John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com -----Original Message----- From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Thursday, May 26, 2011 2:18 AM To: Duddy, John Cc: galaxy-dev Subject: Re: [galaxy-dev] Is dynamic associated information per dataset possible? In addition to Data Library templates, which are useful after the sequencer data has arrived in a Galaxy Data Library, Galaxy's sample tracking system includes sample run templates which are very similar to the Data Library templates, but are associated with a sample as it progresses through it's sequence run lifecycle in the facility. Sample run details templates are defined by the facility administrator. They can be created in the Admin view via the Manage form definitions menu link.
Duddy, John wrote:
The sample tracking system looks interesting. It looks like this is designed to have the sequencers modeled in Galaxy, have Galaxy pull the data from the sequencers, and might assume 1-1 pairing between sequencing run and samples.
I'd like to be able to support pushing files from a central location and variably multiplexed runs, all via the API (or extensions to the API).
Is there any up-to-date documentation on this feature?
The sample tracking system is undergoing a bunch of changes, so I'm thinking the documentation will be rewritten after that. --nate
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com
-----Original Message----- From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Thursday, May 26, 2011 2:18 AM To: Duddy, John Cc: galaxy-dev Subject: Re: [galaxy-dev] Is dynamic associated information per dataset possible?
In addition to Data Library templates, which are useful after the sequencer data has arrived in a Galaxy Data Library, Galaxy's sample tracking system includes sample run templates which are very similar to the Data Library templates, but are associated with a sample as it progresses through it's sequence run lifecycle in the facility. Sample run details templates are defined by the facility administrator. They can be created in the Admin view via the Manage form definitions menu link.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
The sample tracking system looks interesting. It looks like this is designed to have the sequencers modeled in Galaxy, have Galaxy pull the data from the sequencers, and might assume 1-1 pairing between sequencing run and samples. I'd like to be able to support pushing files from a central location and variably multiplexed runs, all via the API (or extensions to the API).Is there any up-to-date documentation on this feature? The sample tracking system is undergoing a bunch of changes, so I'm
Duddy, John wrote: thinking the documentation will be rewritten after that.
There's some code from Brad Chapman at https://github.com/chapmanb/bcbb/tree/master/nextgen/scripts that specifically addresses some of your downstream use cases and more. Currently involves deep galaxy introspection into the sequencing system. As Nate said, that's undergoing replumbing, so that interface part will need changes - but the rest is likely a great starting point for putting together your own local common tasks like (eg) splitting and analysing a multiplexed lane and calling variants and other routine core lab kinds of operations.
Nate, is it just me or all the .png links on that wikipage are broken/missing ? :-S On 2011-05-26 10:38, Nate Coraor wrote:
Duddy, John wrote:
We'd like to be able to associate fixed things (project, Sample, sequencer used) with user's FASTQ files, and we'd also like to allow users to associate dynamic, site-specific stuff with the sequencing run. Currently, users track their runs using a CSV sample sheet, and often they add columns to that sample sheet for their own information.
Is it possible to associate that information with the FASTQ file when it is placed in Galaxy? I know about metadata, but the supported fields look like they are fixed in the code. I was hoping for a solution where the users do not need to modify the Galaxy code to pull this off.
Hi John,
You could accomplish this with library templates:
https://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/LibraryTempla...
--nate
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com<mailto:jduddy@illumina.com>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On Thu, May 26, 2011 at 10:34 AM, Roman Valls <brainstorm@nopcode.org> wrote:
Nate, is it just me or all the .png links on that wikipage are broken/missing ? :-S
I've noticed this on other pages, I think some stuff got moved a while back. They just need the folder name prefixed... Peter
Roman Valls wrote:
Nate, is it just me or all the .png links on that wikipage are broken/missing ? :-S
Hi Roman, We've had occasional problems with the images in the wiki, and I think sometimes they'll show up, sometimes they won't. We're in the process of migrating the wiki to our own Moin install, so this should be fixed then. --nate
On 2011-05-26 10:38, Nate Coraor wrote:
Duddy, John wrote:
We'd like to be able to associate fixed things (project, Sample, sequencer used) with user's FASTQ files, and we'd also like to allow users to associate dynamic, site-specific stuff with the sequencing run. Currently, users track their runs using a CSV sample sheet, and often they add columns to that sample sheet for their own information.
Is it possible to associate that information with the FASTQ file when it is placed in Galaxy? I know about metadata, but the supported fields look like they are fixed in the code. I was hoping for a solution where the users do not need to modify the Galaxy code to pull this off.
Hi John,
You could accomplish this with library templates:
https://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/LibraryTempla...
--nate
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com<mailto:jduddy@illumina.com>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
That's good news. Is this available via the API as well? I didn't see examples of this anywhere in the code, but I thought it might be available by passing additional values in the dictionary. -----Original Message----- From: Nate Coraor [mailto:nate@bx.psu.edu] Sent: Thursday, May 26, 2011 1:38 AM To: Duddy, John Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Is dynamic associated information per dataset possible? Duddy, John wrote:
We'd like to be able to associate fixed things (project, Sample, sequencer used) with user's FASTQ files, and we'd also like to allow users to associate dynamic, site-specific stuff with the sequencing run. Currently, users track their runs using a CSV sample sheet, and often they add columns to that sample sheet for their own information.
Is it possible to associate that information with the FASTQ file when it is placed in Galaxy? I know about metadata, but the supported fields look like they are fixed in the code. I was hoping for a solution where the users do not need to modify the Galaxy code to pull this off.
Hi John, You could accomplish this with library templates: https://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/LibraryTempla... --nate
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com<mailto:jduddy@illumina.com>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Duddy, John wrote:
That's good news. Is this available via the API as well? I didn't see examples of this anywhere in the code, but I thought it might be available by passing additional values in the dictionary.
I don't think it has been done, but as you say, it may be possible just via an arg in the dictionary. --nate
-----Original Message----- From: Nate Coraor [mailto:nate@bx.psu.edu] Sent: Thursday, May 26, 2011 1:38 AM To: Duddy, John Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Is dynamic associated information per dataset possible?
Duddy, John wrote:
We'd like to be able to associate fixed things (project, Sample, sequencer used) with user's FASTQ files, and we'd also like to allow users to associate dynamic, site-specific stuff with the sequencing run. Currently, users track their runs using a CSV sample sheet, and often they add columns to that sample sheet for their own information.
Is it possible to associate that information with the FASTQ file when it is placed in Galaxy? I know about metadata, but the supported fields look like they are fixed in the code. I was hoping for a solution where the users do not need to modify the Galaxy code to pull this off.
Hi John,
You could accomplish this with library templates:
https://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/LibraryTempla...
--nate
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com<mailto:jduddy@illumina.com>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (6)
-
Duddy, John -
Greg Von Kuster -
Nate Coraor -
Peter Cock -
Roman Valls -
Ross