Re: [galaxy-dev] Running a workflow programatically
Hi Dannon, Thanks very much for answering these questions. I think this all I need to get it going. I'll report back soon. Darren On Mar 15, 2011 6:06 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
Hi Dannon, I am still just trying to proof of concept the workflow_execute script and running into a little trouble. I am tracking my issue into workflows.py in the api. In debugging, I am confident that the datasets are correctly specified, as I can specify a bad dataset and get the http workflow interface to return an error. However, the input dataset that I am specifying is not being forwarded to the workflow step. When I execute the workflow from the web ui, I get a command like this for the first step: executing: python /mnt/galaxy/galaxy_dist/tools/fastq/fastq_groomer.py '/mnt/galaxy/galaxy_dist/database/files/000/dataset_48.dat' 'illumina' '/mnt/galaxy/galaxy_dist/database/job_working_directory/169/galaxy_dataset_169.dat' 'sanger' 'ascii' 'summarize_input' However, when I run a workflow from the command line: python /mnt/galaxy/galaxy_dist/scripts/api/workflow_execute.py <api-key> <url>/api/workflows 38247d270c7cb1bb 'hist_id=38247d270c7cb1bb' '1=hda=30fc17ce78176bfb' this is the command I get for the first step: executing: python /mnt/galaxy/galaxy_dist/tools/fastq/fastq_groomer.py 'None' 'illumina' '/mnt/galaxy/galaxy_dist/database/job_working_directory/164/galaxy_dataset_164.dat' 'sanger' 'ascii' 'summarize_input' So the dataset passes all of the asserts, but is not passed to the first step of the workflow. The api key and workflow id are working because it's not failing right away, and the history id is working because the failed steps are populated in it. The dataset seems not to be throwing any errors. So I am finding myself very confused! Is there something obvious that you see? Take care, Darren On Tue, Mar 15, 2011 at 7:50 PM, Darren Brown <brown@centerspace.net> wrote:
Hi Dannon,
Thanks very much for answering these questions. I think this all I need to get it going. I'll report back soon.
Darren
On Mar 15, 2011 6:06 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
On Mar 16, 2011, at 5:43 PM, Darren Brown wrote:
However, when I run a workflow from the command line:
python /mnt/galaxy/galaxy_dist/scripts/api/workflow_execute.py <api-key> <url>/api/workflows 38247d270c7cb1bb 'hist_id=38247d270c7cb1bb' '1=hda=30fc17ce78176bfb'
My hunch is that the step id '1' in this line is probably your issue. To list workflow details in the api, you can use the scripts/api/display.py So, for your workflow there, python /mnt/galaxy/galaxy_dist/scripts/api/display.py <api_key> http://localhost:8080/api/workflows/38247d270c7cb1bb Check that with the workflow in question and see what it returns as the inputs dictionary. -Dannon
Ah ok, and you are right there is something wrong there. The inputs dictionary is empty: inputs: {} However this seems kind of odd, because I do need to specify history inputs for the workflow through the web ui. On Wed, Mar 16, 2011 at 2:55 PM, Dannon Baker <dannonbaker@me.com> wrote:
On Mar 16, 2011, at 5:43 PM, Darren Brown wrote:
However, when I run a workflow from the command line:
python /mnt/galaxy/galaxy_dist/scripts/api/workflow_execute.py <api-key> <url>/api/workflows 38247d270c7cb1bb 'hist_id=38247d270c7cb1bb' '1=hda=30fc17ce78176bfb'
My hunch is that the step id '1' in this line is probably your issue. To list workflow details in the api, you can use the scripts/api/display.py
So, for your workflow there,
python /mnt/galaxy/galaxy_dist/scripts/api/display.py <api_key> http://localhost:8080/api/workflows/38247d270c7cb1bb
Check that with the workflow in question and see what it returns as the inputs dictionary.
-Dannon
That's not actually what I expected, but I see what's going on here. The workflows API expects explicit "Input Dataset" steps for mapping datasets, add one of those and connect it to your fastq step and see if it has inputs showing up for you to map to at that point. Do note that the input step id expected is actually the workflow step_id, not the "Step 1" "Step 2", etc numbers that you'll see in the run workflow page. -Dannon On Mar 16, 2011, at 6:01 PM, Darren Brown wrote:
Ah ok, and you are right there is something wrong there. The inputs dictionary is empty:
inputs: {}
However this seems kind of odd, because I do need to specify history inputs for the workflow through the web ui.
On Wed, Mar 16, 2011 at 2:55 PM, Dannon Baker <dannonbaker@me.com> wrote:
On Mar 16, 2011, at 5:43 PM, Darren Brown wrote:
However, when I run a workflow from the command line:
python /mnt/galaxy/galaxy_dist/scripts/api/workflow_execute.py <api-key> <url>/api/workflows 38247d270c7cb1bb 'hist_id=38247d270c7cb1bb' '1=hda=30fc17ce78176bfb'
My hunch is that the step id '1' in this line is probably your issue. To list workflow details in the api, you can use the scripts/api/display.py
So, for your workflow there,
python /mnt/galaxy/galaxy_dist/scripts/api/display.py <api_key> http://localhost:8080/api/workflows/38247d270c7cb1bb
Check that with the workflow in question and see what it returns as the inputs dictionary.
-Dannon
Ah I see what you mean Dannon. I need the id of the step from the database essentially (or maybe there is a better way to do it 'right'). But anyway, I have been able to run a workflow from the command-line! Thanks so much for your help, Darren On Wed, Mar 16, 2011 at 3:07 PM, Dannon Baker <dannonbaker@me.com> wrote:
That's not actually what I expected, but I see what's going on here. The workflows API expects explicit "Input Dataset" steps for mapping datasets, add one of those and connect it to your fastq step and see if it has inputs showing up for you to map to at that point.
Do note that the input step id expected is actually the workflow step_id, not the "Step 1" "Step 2", etc numbers that you'll see in the run workflow page.
-Dannon
On Mar 16, 2011, at 6:01 PM, Darren Brown wrote:
Ah ok, and you are right there is something wrong there. The inputs dictionary is empty:
inputs: {}
However this seems kind of odd, because I do need to specify history inputs for the workflow through the web ui.
On Wed, Mar 16, 2011 at 2:55 PM, Dannon Baker <dannonbaker@me.com> wrote:
On Mar 16, 2011, at 5:43 PM, Darren Brown wrote:
However, when I run a workflow from the command line:
python /mnt/galaxy/galaxy_dist/scripts/api/workflow_execute.py <api-key> <url>/api/workflows 38247d270c7cb1bb 'hist_id=38247d270c7cb1bb' '1=hda=30fc17ce78176bfb'
My hunch is that the step id '1' in this line is probably your issue. To list workflow details in the api, you can use the scripts/api/display.py
So, for your workflow there,
python /mnt/galaxy/galaxy_dist/scripts/api/display.py <api_key> http://localhost:8080/api/workflows/38247d270c7cb1bb
Check that with the workflow in question and see what it returns as the inputs dictionary.
-Dannon
When is this functionality expected to make it into galaxy-dist? Thanks, Dave On 3/16/11 6:07 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
That's not actually what I expected, but I see what's going on here. The workflows API expects explicit "Input Dataset" steps for mapping datasets, add one of those and connect it to your fastq step and see if it has inputs showing up for you to map to at that point.
Do note that the input step id expected is actually the workflow step_id, not the "Step 1" "Step 2", etc numbers that you'll see in the run workflow page.
-Dannon
On Mar 16, 2011, at 6:01 PM, Darren Brown wrote:
Ah ok, and you are right there is something wrong there. The inputs dictionary is empty:
inputs: {}
However this seems kind of odd, because I do need to specify history inputs for the workflow through the web ui.
On Wed, Mar 16, 2011 at 2:55 PM, Dannon Baker <dannonbaker@me.com> wrote:
On Mar 16, 2011, at 5:43 PM, Darren Brown wrote:
However, when I run a workflow from the command line:
python /mnt/galaxy/galaxy_dist/scripts/api/workflow_execute.py <api-key> <url>/api/workflows 38247d270c7cb1bb 'hist_id=38247d270c7cb1bb' '1=hda=30fc17ce78176bfb'
My hunch is that the step id '1' in this line is probably your issue. To list workflow details in the api, you can use the scripts/api/display.py
So, for your workflow there,
python /mnt/galaxy/galaxy_dist/scripts/api/display.py <api_key> http://localhost:8080/api/workflows/38247d270c7cb1bb
Check that with the workflow in question and see what it returns as the inputs dictionary.
-Dannon
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Dave, Most likely within a week. If you wanted to go ahead and try it, the latest changeset that affects the API is 5223:82f9422864ff. You can snag that now (the subsequent update from galaxy-dist will still work fine, when it comes out) with: % hg pull -u -r 82f9422864ff http://bitbucket.org/galaxy/galaxy-central/ -Dannon On Mar 28, 2011, at 3:42 PM, Dave Walton wrote:
When is this functionality expected to make it into galaxy-dist?
Thanks,
Dave
On 3/16/11 6:07 PM, "Dannon Baker" <dannonbaker@me.com> wrote:
That's not actually what I expected, but I see what's going on here. The workflows API expects explicit "Input Dataset" steps for mapping datasets, add one of those and connect it to your fastq step and see if it has inputs showing up for you to map to at that point.
Do note that the input step id expected is actually the workflow step_id, not the "Step 1" "Step 2", etc numbers that you'll see in the run workflow page.
-Dannon
On Mar 16, 2011, at 6:01 PM, Darren Brown wrote:
Ah ok, and you are right there is something wrong there. The inputs dictionary is empty:
inputs: {}
However this seems kind of odd, because I do need to specify history inputs for the workflow through the web ui.
On Wed, Mar 16, 2011 at 2:55 PM, Dannon Baker <dannonbaker@me.com> wrote:
On Mar 16, 2011, at 5:43 PM, Darren Brown wrote:
However, when I run a workflow from the command line:
python /mnt/galaxy/galaxy_dist/scripts/api/workflow_execute.py <api-key> <url>/api/workflows 38247d270c7cb1bb 'hist_id=38247d270c7cb1bb' '1=hda=30fc17ce78176bfb'
My hunch is that the step id '1' in this line is probably your issue. To list workflow details in the api, you can use the scripts/api/display.py
So, for your workflow there,
python /mnt/galaxy/galaxy_dist/scripts/api/display.py <api_key> http://localhost:8080/api/workflows/38247d270c7cb1bb
Check that with the workflow in question and see what it returns as the inputs dictionary.
-Dannon
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (3)
-
Dannon Baker -
Darren Brown -
Dave Walton