My next question is, I have found that VCF files don't get split properly as the header is not included in the second file as is usually required by tools (such as vcf-subset). I have read the code and am happy to implement this functionality but am not to sure where this would best be done.

On Wed, Oct 31, 2012 at 12:35 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:

On Tue, Oct 30, 2012 at 11:20 PM, Edward Hills <ehills666@gmail.com> wrote:
> Hi Galaxy-Team,
>
> After reading a message on this mailing list about the job splitter I began
> to investigate what and how this is used. Unfortunately I have been unable
> to find any documentation on your website for it.
>
> Am I blind and missing it or is it yet to be properly documented?
>
> Sorry if this turns to be out a pointless exercise, but it would be
> extremely useful for my Galaxy development.
>
> Cheers,
> Ed

Hi Ed,

To enable this you need to add a <parallelism> tag to the
tool's XML file, and enable the feature in universe_wsgi.ini
with something like this:

use_tasked_jobs = True
local_task_queue_workers = 4

I'm not aware of any documentation, I've been mostly working
from the Python source code in order to get it to work on the
BLAST+ wrappers and some of my other tool wrappers. In all
the cases I've used there is a single FASTA file being split,
sometimes some common input files which are unchanged,
and a single output file being merged.

Peter