Re: [galaxy-dev] galaxy-dev Digest, Vol 33, Issue 21
I may be missing something, but right now, and without any changes to Galaxy, there's no reason why a new tool should not wrap an existing (or new) command line executable program that uses web services to do something interesting! I have an rgenetics tool that uses the Haploview java executable in command line mode to prepare image files of LD patterns in the Hapmap populations with an inhouse sample over a region, and then stitches them together into a single image so they can be visually compared - Galaxy doesn't know or care if the tool executable uses web services - the job runner sets up the call and then deals with the resulting tool outputs correctly - as long as the cluster node executing the tool has internet access, it can interact with the web services transparently - as Greg says, the tool will be fragile to changes in the web service call syntax but that's always a problem in the long term. I agree with Greg that bringing web services interfaces *directly* into the existing Galaxy framework does not appear to have a high priority at this time and I'm not even sure that it makes an awful lot of sense in the long term - especially since we can let web service interaction live out in tool space where it already works? Is there a specific use-case anyone has in mind where the web services interface really needs to be brought inside the framework itself? On Tue, Sep 30, 2008 at 2:00 AM, <galaxy-dev-request@bx.psu.edu> wrote:
Today's Topics:
1. Re: Galaxy with web services (Greg Von Kuster)
----------------------------------------------------------------------
Message: 1 Date: Mon, 29 Sep 2008 09:52:39 -0400 From: Greg Von Kuster <ghv2@psu.edu> Subject: Re: [galaxy-dev] Galaxy with web services To: Praveen Agrawal <praveen@scfbio-iitd.res.in> Cc: galaxy-dev@bx.psu.edu Message-ID: <48E0DDA7.1090200@psu.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hello Praveen,
Currently web-based interaction is only used for external data sources (biomart, UCSC table browser, most other things under "Get Data"). For the other tools, we generate a command line based on the user input, which Galaxy then runs. The motivation for this is control and trust. One of our major goals with Galaxy is to ensure reproducibility, and relying on web services which can change their interfaces or behavior makes this very challenging. We are certainly interested in supporting web services in the Galaxy framework, but implementation is not yet under way. Several of the methods currently in ~/tools/__init__.py ( among others ) will need to evolve and grow in order to support a web services interface for remote tools. We'll certainly keep you informed as we begin to support this, but it may be a while...
Greg Von Kuster Galaxy Development Team
Praveen Agrawal wrote:
Hi, I am trying to use Galaxy at my site and cann't figure out Galaxy with web services. I understand that it's possible to integrate a data source using WSDL/SOAP. Is the same possible for a remote tool as well?
-- python -c "foo = map(None,'moc.liamg@surazal.ssor'); foo.reverse(); print ''.join(foo)"
Hi Ross, Hi Greg, Thanks for the elaborate answers. Certainly, having programs installed locally gives lot more control over their execution. In my case I am going to have tools which would be running across Indian "grid" and then the scientist would have the freedom to use the workflow capability to join those tools. I couldn't understand "we can let web service interaction live out in tool space where it already works". Please elaborate. Thanks, Praveen
I may be missing something, but right now, and without any changes to Galaxy, there's no reason why a new tool should not wrap an existing (or new) command line executable program that uses web services to do something interesting!
I have an rgenetics tool that uses the Haploview java executable in command line mode to prepare image files of LD patterns in the Hapmap populations with an inhouse sample over a region, and then stitches them together into a single image so they can be visually compared - Galaxy doesn't know or care if the tool executable uses web services - the job runner sets up the call and then deals with the resulting tool outputs correctly - as long as the cluster node executing the tool has internet access, it can interact with the web services transparently - as Greg says, the tool will be fragile to changes in the web service call syntax but that's always a problem in the long term.
I agree with Greg that bringing web services interfaces *directly* into the existing Galaxy framework does not appear to have a high priority at this time and I'm not even sure that it makes an awful lot of sense in the long term - especially since we can let web service interaction live out in tool space where it already works? Is there a specific use-case anyone has in mind where the web services interface really needs to be brought inside the framework itself?
On Tue, Sep 30, 2008 at 2:00 AM, <galaxy-dev-request@bx.psu.edu> wrote:
Today's Topics:
1. Re: Galaxy with web services (Greg Von Kuster)
----------------------------------------------------------------------
Message: 1 Date: Mon, 29 Sep 2008 09:52:39 -0400 From: Greg Von Kuster <ghv2@psu.edu> Subject: Re: [galaxy-dev] Galaxy with web services To: Praveen Agrawal <praveen@scfbio-iitd.res.in> Cc: galaxy-dev@bx.psu.edu Message-ID: <48E0DDA7.1090200@psu.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hello Praveen,
Currently web-based interaction is only used for external data sources (biomart, UCSC table browser, most other things under "Get Data"). For the other tools, we generate a command line based on the user input, which Galaxy then runs. The motivation for this is control and trust. One of our major goals with Galaxy is to ensure reproducibility, and relying on web services which can change their interfaces or behavior makes this very challenging. We are certainly interested in supporting web services in the Galaxy framework, but implementation is not yet under way. Several of the methods currently in ~/tools/__init__.py ( among others ) will need to evolve and grow in order to support a web services interface for remote tools. We'll certainly keep you informed as we begin to support this, but it may be a while...
Greg Von Kuster Galaxy Development Team
Praveen Agrawal wrote:
Hi, I am trying to use Galaxy at my site and cann't figure out Galaxy with web services. I understand that it's possible to integrate a data source using WSDL/SOAP. Is the same possible for a remote tool as well?
-- python -c "foo = map(None,'moc.liamg@surazal.ssor'); foo.reverse(); print ''.join(foo)"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://www.scfbio-iitd.res.in ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On Tue, Sep 30, 2008 at 6:32 PM, Praveen Agrawal <praveen@scfbio-iitd.res.in> wrote:
Hi Ross, Hi Greg, Thanks for the elaborate answers. Certainly, having programs installed locally gives lot more control over their execution.
In my case I am going to have tools which would be running across Indian "grid" and then the scientist would have the freedom to use the workflow capability to join those tools. I couldn't understand "we can let web service interaction live out in tool space where it already works". Please elaborate.
I'm trying to say that changes to the core of Galaxy to allow (eg) remote web services interaction that Greg mentioned may eventually appear, but you can already make use of existing web services in new tools that you write for Galaxy. The Galaxy framework already allows you to wrap a web services based application into a tool - like the Haploview example I mentioned. From the point of view of the user (and Galaxy!), it's just another Galaxy tool - it happens to use web services (eg xmlrpc or SOAP) and they are being executed out in "tool space" - meaning that the Galaxy framework doesn't know or care about the web services calls made by whatever executable the tool has invoked. In other words, there's nothing stopping you writing tools that use web services exposed on the Indian grid or any other web services - you don't need any changes to the Galaxy core - in the example I gave, we have a Java executable (haploview) which uses xml-rpc to talk to the hapmap web site, wrapped into a Galaxy tool (using Python code that I wrote to do some tricks with the Haploview output files and images). If you're interested in seeing how I made it work, take a look at http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView.... (and http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView.... and http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView_...) The code is brutally ugly but works pretty well. I hope this helps?
Thanks, Praveen
I may be missing something, but right now, and without any changes to Galaxy, there's no reason why a new tool should not wrap an existing (or new) command line executable program that uses web services to do something interesting!
I have an rgenetics tool that uses the Haploview java executable in command line mode to prepare image files of LD patterns in the Hapmap populations with an inhouse sample over a region, and then stitches them together into a single image so they can be visually compared - Galaxy doesn't know or care if the tool executable uses web services - the job runner sets up the call and then deals with the resulting tool outputs correctly - as long as the cluster node executing the tool has internet access, it can interact with the web services transparently - as Greg says, the tool will be fragile to changes in the web service call syntax but that's always a problem in the long term.
I agree with Greg that bringing web services interfaces *directly* into the existing Galaxy framework does not appear to have a high priority at this time and I'm not even sure that it makes an awful lot of sense in the long term - especially since we can let web service interaction live out in tool space where it already works? Is there a specific use-case anyone has in mind where the web services interface really needs to be brought inside the framework itself?
On Tue, Sep 30, 2008 at 2:00 AM, <galaxy-dev-request@bx.psu.edu> wrote:
Today's Topics:
1. Re: Galaxy with web services (Greg Von Kuster)
----------------------------------------------------------------------
Message: 1 Date: Mon, 29 Sep 2008 09:52:39 -0400 From: Greg Von Kuster <ghv2@psu.edu> Subject: Re: [galaxy-dev] Galaxy with web services To: Praveen Agrawal <praveen@scfbio-iitd.res.in> Cc: galaxy-dev@bx.psu.edu Message-ID: <48E0DDA7.1090200@psu.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hello Praveen,
Currently web-based interaction is only used for external data sources (biomart, UCSC table browser, most other things under "Get Data"). For the other tools, we generate a command line based on the user input, which Galaxy then runs. The motivation for this is control and trust. One of our major goals with Galaxy is to ensure reproducibility, and relying on web services which can change their interfaces or behavior makes this very challenging. We are certainly interested in supporting web services in the Galaxy framework, but implementation is not yet under way. Several of the methods currently in ~/tools/__init__.py ( among others ) will need to evolve and grow in order to support a web services interface for remote tools. We'll certainly keep you informed as we begin to support this, but it may be a while...
Greg Von Kuster Galaxy Development Team
Praveen Agrawal wrote:
Hi, I am trying to use Galaxy at my site and cann't figure out Galaxy with web services. I understand that it's possible to integrate a data source using WSDL/SOAP. Is the same possible for a remote tool as well?
-- python -c "foo = map(None,'moc.liamg@surazal.ssor'); foo.reverse(); print ''.join(foo)"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://www.scfbio-iitd.res.in ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- python -c "foo = map(None,'moc.liamg@surazal.ssor'); foo.reverse(); print ''.join(foo)"
Thanks Ross, that solves my purpose. Regards, Praveen
On Tue, Sep 30, 2008 at 6:32 PM, Praveen Agrawal <praveen@scfbio-iitd.res.in> wrote:
Hi Ross, Hi Greg, Thanks for the elaborate answers. Certainly, having programs installed locally gives lot more control over their execution.
In my case I am going to have tools which would be running across Indian "grid" and then the scientist would have the freedom to use the workflow capability to join those tools. I couldn't understand "we can let web service interaction live out in tool space where it already works". Please elaborate.
I'm trying to say that changes to the core of Galaxy to allow (eg) remote web services interaction that Greg mentioned may eventually appear, but you can already make use of existing web services in new tools that you write for Galaxy.
The Galaxy framework already allows you to wrap a web services based application into a tool - like the Haploview example I mentioned. From the point of view of the user (and Galaxy!), it's just another Galaxy tool - it happens to use web services (eg xmlrpc or SOAP) and they are being executed out in "tool space" - meaning that the Galaxy framework doesn't know or care about the web services calls made by whatever executable the tool has invoked.
In other words, there's nothing stopping you writing tools that use web services exposed on the Indian grid or any other web services - you don't need any changes to the Galaxy core - in the example I gave, we have a Java executable (haploview) which uses xml-rpc to talk to the hapmap web site, wrapped into a Galaxy tool (using Python code that I wrote to do some tricks with the Haploview output files and images).
If you're interested in seeing how I made it work, take a look at http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView.... (and http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView.... and http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView_...) The code is brutally ugly but works pretty well.
I hope this helps?
Thanks, Praveen
I may be missing something, but right now, and without any changes to Galaxy, there's no reason why a new tool should not wrap an existing (or new) command line executable program that uses web services to do something interesting!
I have an rgenetics tool that uses the Haploview java executable in command line mode to prepare image files of LD patterns in the Hapmap populations with an inhouse sample over a region, and then stitches them together into a single image so they can be visually compared - Galaxy doesn't know or care if the tool executable uses web services - the job runner sets up the call and then deals with the resulting tool outputs correctly - as long as the cluster node executing the tool has internet access, it can interact with the web services transparently - as Greg says, the tool will be fragile to changes in the web service call syntax but that's always a problem in the long term.
I agree with Greg that bringing web services interfaces *directly* into the existing Galaxy framework does not appear to have a high priority at this time and I'm not even sure that it makes an awful lot of sense in the long term - especially since we can let web service interaction live out in tool space where it already works? Is there a specific use-case anyone has in mind where the web services interface really needs to be brought inside the framework itself?
On Tue, Sep 30, 2008 at 2:00 AM, <galaxy-dev-request@bx.psu.edu> wrote:
Today's Topics:
1. Re: Galaxy with web services (Greg Von Kuster)
----------------------------------------------------------------------
Message: 1 Date: Mon, 29 Sep 2008 09:52:39 -0400 From: Greg Von Kuster <ghv2@psu.edu> Subject: Re: [galaxy-dev] Galaxy with web services To: Praveen Agrawal <praveen@scfbio-iitd.res.in> Cc: galaxy-dev@bx.psu.edu Message-ID: <48E0DDA7.1090200@psu.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hello Praveen,
Currently web-based interaction is only used for external data sources (biomart, UCSC table browser, most other things under "Get Data"). For the other tools, we generate a command line based on the user input, which Galaxy then runs. The motivation for this is control and trust. One of our major goals with Galaxy is to ensure reproducibility, and relying on web services which can change their interfaces or behavior makes this very challenging. We are certainly interested in supporting web services in the Galaxy framework, but implementation is not yet under way. Several of the methods currently in ~/tools/__init__.py ( among others ) will need to evolve and grow in order to support a web services interface for remote tools. We'll certainly keep you informed as we begin to support this, but it may be a while...
Greg Von Kuster Galaxy Development Team
Praveen Agrawal wrote:
Hi, I am trying to use Galaxy at my site and cann't figure out Galaxy with web services. I understand that it's possible to integrate a data source using WSDL/SOAP. Is the same possible for a remote tool as well?
-- python -c "foo = map(None,'moc.liamg@surazal.ssor'); foo.reverse(); print ''.join(foo)"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://www.scfbio-iitd.res.in ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- python -c "foo = map(None,'moc.liamg@surazal.ssor'); foo.reverse(); print ''.join(foo)"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://www.scfbio-iitd.res.in ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
participants (2)
-
Praveen Agrawal -
Ross