Dear Dave, This was very useful. Thank you so much. Specifically, I am trying to integrate Galaxy with Condor, which supports DRMAA. Is there an existing Condor job runner implementation, or any useful code for submitting Galaxy jobs to Condor? Thanks! Oren On 9/20/2011 12:21 PM, Dave Clements wrote:

Hi Oren,

This book chapter http://userwww.service.emory.edu/~eafgan/content/Galaxy_eScienceGateway.pdf <http://userwww.service.emory.edu/%7Eeafgan/content/Galaxy_eScienceGateway.pdf>, includes a high-level overview of Galaxy. (Note: that chapter will be copied to the wiki soon.)

There is some API documentation. ~/scripts/api/README, ~/scripts/api/workflow_ execute.py and ~/scripts/api/example_watch_folder.py in the distribution http://wiki.g2.bx.psu.edu/News%20Briefs/2010_07_16#Initial_implementation_of... http://wiki.g2.bx.psu.edu/News%20Briefs/2011_08_30#API http://wiki.g2.bx.psu.edu/Events/GCC2011 (/Galaxy Deployment and API/ presentation) http://wiki.g2.bx.psu.edu/Admin/Sample%20Tracking/Next%20Gen http://wiki.g2.bx.psu.edu/Events/GCC2011/Workflows%20and%20API

Sometime in the next week I'll compile all this into a single wiki page.

As far as I know there is no "How to write a runner for Galaxy" document. The closest we have is http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster and http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Web%20Application%20Scali...

Hope this helps,

Dave C.

On Mon, Sep 19, 2011 at 9:51 AM, Oren Livne <livne@uchicago.edu <mailto:livne@uchicago.edu>> wrote:

Dear All,

I am a software developer, new to Galaxy. Is there a detailed document about the Galaxy code architecture, API, what runners are, and how to write a runner?

I am especially interested in integrating Galaxy with an external scheduler software (I am aware of Galaxy's scheduling capabilities). I'd like to intercept Galaxy workflows and send their object model to my scheduler. I could not find this information on the Galaxy Wiki.

Thank you so much in advance, Oren Livne ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org <http://usegalaxy.org>. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

-- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

Hi Dave, That was me! I'm currently talking with the condor folks here in Madison WI. It would be great to get galaxy going with Condor since condor nows forms part of red hat. Besides the existing available docs that you listed below who do you think would be best as a contact for questions regarding this implementation? best, Victor On Sep 21, 2011, at 12:16 PM, Dave Clements wrote:

Hi Oren,

I had no idea so I went to the Galaxy mailing list archive @ Nabble (http://gmod.827538.n3.nabble.com/Galaxy-f829901.html) and searched for Condor (http://gmod.827538.n3.nabble.com/template/NamlServlet.jtp?macro=search_page&node=829901&query=condor). It appears that Ravi Madduri @ Argonne has done some work with this.

I was also at a meeting in March with someone from Wisconsin (where I think Condor is currently developed) and they indicated their might be interest in helping Galaxy work with Condor. I/you can pursue this option as well.

Dave C.

On Wed, Sep 21, 2011 at 9:38 AM, Oren Livne <livne@uchicago.edu> wrote: Dear Dave,

This was very useful. Thank you so much. Specifically, I am trying to integrate Galaxy with Condor, which supports DRMAA. Is there an existing Condor job runner implementation, or any useful code for submitting Galaxy jobs to Condor?

Thanks!

Oren

On 9/20/2011 12:21 PM, Dave Clements wrote:

...

Hi Oren,

This book chapter http://userwww.service.emory.edu/~eafgan/content/Galaxy_eScienceGateway.pdf, includes a high-level overview of Galaxy. (Note: that chapter will be copied to the wiki soon.)

There is some API documentation. ~/scripts/api/README, ~/scripts/api/workflow_ execute.py and ~/scripts/api/example_watch_folder.py in the distribution http://wiki.g2.bx.psu.edu/News%20Briefs/2010_07_16#Initial_implementation_of... http://wiki.g2.bx.psu.edu/News%20Briefs/2011_08_30#API http://wiki.g2.bx.psu.edu/Events/GCC2011 (Galaxy Deployment and API presentation) http://wiki.g2.bx.psu.edu/Admin/Sample%20Tracking/Next%20Gen http://wiki.g2.bx.psu.edu/Events/GCC2011/Workflows%20and%20API

Sometime in the next week I'll compile all this into a single wiki page.

As far as I know there is no "How to write a runner for Galaxy" document. The closest we have is http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster and http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Web%20Application%20Scali...

Hope this helps,

Dave C.

On Mon, Sep 19, 2011 at 9:51 AM, Oren Livne <livne@uchicago.edu> wrote: Dear All,

I am a software developer, new to Galaxy. Is there a detailed document about the Galaxy code architecture, API, what runners are, and how to write a runner?

I am especially interested in integrating Galaxy with an external scheduler software (I am aware of Galaxy's scheduling capabilities). I'd like to intercept Galaxy workflows and send their object model to my scheduler. I could not find this information on the Galaxy Wiki.

Thank you so much in advance, Oren Livne ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

-- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

-- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

Oren, I do believe some folks have done this, I am a bit surprised none have replied. But yes, since Condor supports DRMAA, you should be able to point $DRMAA_LIBRARY_PATH at Condor's libdrmaa.so, enable the drmaa runner, and try it out. --nate Oren Livne wrote:

Dear Dave, Victor, Is it possible to simply use the drmma job runner with the condor drmma lib.so? Thanks! Oren

On 9/22/2011 3:21 PM, Victor Ruotti wrote:

...

Hi Dave, That was me! I'm currently talking with the condor folks here in Madison WI. It would be great to get galaxy going with Condor since condor nows forms part of red hat.

Besides the existing available docs that you listed below who do you think would be best as a contact for questions regarding this implementation? best, Victor

On Sep 21, 2011, at 12:16 PM, Dave Clements wrote:

...

Hi Oren,

I had no idea so I went to the Galaxy mailing list archive @ Nabble (http://gmod.827538.n3.nabble.com/Galaxy-f829901.html) and searched for Condor (http://gmod.827538.n3.nabble.com/template/NamlServlet.jtp?macro=search_page&node=829901&query=condor <http://gmod.827538.n3.nabble.com/template/NamlServlet.jtp?macro=search_page&node=829901&query=condor>). It appears that Ravi Madduri @ Argonne has done some work with this.

I was also at a meeting in March with someone from Wisconsin (where I think Condor is currently developed) and they indicated their might be interest in helping Galaxy work with Condor. I/you can pursue this option as well.

Dave C.

On Wed, Sep 21, 2011 at 9:38 AM, Oren Livne <livne@uchicago.edu <mailto:livne@uchicago.edu>> wrote:

Dear Dave,

This was very useful. Thank you so much. Specifically, I am trying to integrate Galaxy with Condor, which supports DRMAA. Is there an existing Condor job runner implementation, or any useful code for submitting Galaxy jobs to Condor?

Thanks!

Oren

On 9/20/2011 12:21 PM, Dave Clements wrote:

...

Hi Oren,

This book chapter http://userwww.service.emory.edu/~eafgan/content/Galaxy_eScienceGateway.pdf <http://userwww.service.emory.edu/%7Eeafgan/content/Galaxy_eScienceGateway.pdf>, includes a high-level overview of Galaxy. (Note: that chapter will be copied to the wiki soon.)

There is some API documentation. ~/scripts/api/README, ~/scripts/api/workflow_ execute.py and ~/scripts/api/example_watch_folder.py in the distribution http://wiki.g2.bx.psu.edu/News%20Briefs/2010_07_16#Initial_implementation_of... http://wiki.g2.bx.psu.edu/News%20Briefs/2011_08_30#API http://wiki.g2.bx.psu.edu/Events/GCC2011 (/Galaxy Deployment and API/ presentation) http://wiki.g2.bx.psu.edu/Admin/Sample%20Tracking/Next%20Gen http://wiki.g2.bx.psu.edu/Events/GCC2011/Workflows%20and%20API

Sometime in the next week I'll compile all this into a single wiki page.

As far as I know there is no "How to write a runner for Galaxy" document. The closest we have is http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster and http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Web%20Application%20Scali...

Hope this helps,

Dave C.

On Mon, Sep 19, 2011 at 9:51 AM, Oren Livne <livne@uchicago.edu <mailto:livne@uchicago.edu>> wrote:

Dear All,

I am a software developer, new to Galaxy. Is there a detailed document about the Galaxy code architecture, API, what runners are, and how to write a runner?

I am especially interested in integrating Galaxy with an external scheduler software (I am aware of Galaxy's scheduling capabilities). I'd like to intercept Galaxy workflows and send their object model to my scheduler. I could not find this information on the Galaxy Wiki.

Thank you so much in advance, Oren Livne ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org <http://usegalaxy.org/>. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

-- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

-- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org <http://usegalaxy.org>. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

Oren Livne wrote:

Dear All,

We have a lot of sequencing data files whose locations and identifiers are managed by our home-grown webapp. We'd like to enable users to single-sign-on from our webapp into Galaxy (we will be using open ID on both systems for that). When the user opens a Galaxy window, we'd like them to be able to access the data files on our system which they have permissions to see.

How should we go about that without copying or moving files? Should we make a new data library, or (preferably) create our identifier service and make it a remote data source/library that galaxy can see and that can be browsed like other data libraries? Is there an API we should write our service against, or wrap files with appropriate galaxy meta data, etc.?

Hi Oren, Without knowing the full architecture it's a bit hard to say what the best solution is. However, Galaxy's API does allow for "upload" from locations on the filesystem or from an FTP/HTTP/HTTPS url to a data library, so you should be able to use this to do what you're talking about. Thanks, --nate

Thank you so much, Oren