I'm not following you - it's been 6 months since I wrote that code ;-} IT looks to me like a DatasetPath() object is always placed in that array, and with one exception near then, it looks like the change I made generates those objects the same way. Do you have a stack trace for the merge problem I can look at? John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com -----Original Message----- From: Nate Coraor (nate@bx.psu.edu) [mailto:nate@bx.psu.edu] Sent: Thursday, November 03, 2011 2:22 PM To: Duddy, John Cc: Chorny, Ilya; galaxy-dev@lists.bx.psu.edu Subject: Re: Looks like actual user breaks splitting Hi John, It looks like the first issue is related to the change from get_output_fnames() -> compute_outputs(). When outputs_to_working_directory = False (default) this method stores/returns a HistoryDatasetAssociation, but when True, stores/returns a Dataset (the original method's behavior). Thus, accessing the object's .datatype attribute in the splitter's do_merge() fails. Thanks, --nate Duddy, John wrote:
I'll submit a pull request shortly...
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com
-----Original Message----- From: Nate Coraor (nate@bx.psu.edu) [mailto:nate@bx.psu.edu] Sent: Wednesday, November 02, 2011 12:24 PM To: Duddy, John Cc: Chorny, Ilya; galaxy-dev@lists.bx.psu.edu Subject: Re: Looks like actual user breaks splitting
John, Ilya,
I get further with sequence type inputs but it looks like JobWrapper.get_output_datasets_and_fnames() is not returning the right thing when outputs_to_working_directory = True
BTW, the base Data.split() method is broken after the updates to Sequence.split() since it wasn't updated to expect HistoryDatasetAssociations rather than filenames. Could you take a look at that when you get a chance?
--nate
Duddy, John wrote:
The datatype you are using does not define a split method. Are you working with our in-progress gz type or fastqillumina?
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com<mailto:jduddy@illumina.com>
From: Chorny, Ilya Sent: Wednesday, November 02, 2011 11:50 AM To: Duddy, John Cc: Nate Coraor (nate@bx.psu.edu); galaxy-dev@lists.bx.psu.edu Subject: Looks like actual user breaks splitting
Hey John,
Any thoughts?
Ilya
Traceback (most recent call last): File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/runners/tasks.py", line 73, in run_job tasks = splitter.do_split(job_wrapper) File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/jobs/splitters/multi.py", line 73, in do_split input_type.split(input_datasets, get_new_working_directory_name, parallel_settings) File "/home/galaxy/ichorny/galaxy-central/lib/galaxy/datatypes/data.py", line 473, in split raise Exception("Text file splitting does not support multiple files") Exception: Text file splitting does not support multiple files
Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny@illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>