Re: [galaxy-dev] rpy or perl
On Tue, Oct 12, 2010 at 10:43 AM, Freddy <freddy.debree@wur.nl> wrote:
Hi Peter,
Running R from xml is not so straight forward to me. Is the interpreter "bash" or is it "R"? If "bash" then it should be running "R --vanilla --slave input < Rscript > output" if "R" then it should be running ? smthg like "source(Rscript)"
Hope you can help...
Freddy
Consider a simple R script, hello_world.R like this: print('Hello world'); print(commandArgs()); Then in Galaxy I think you'd need the default interpreter ("bash") and the command to be: R --vanilla --file=hello_world.R --args "Args go here" I'm assuming your script will write useful output to a file, and stdout can be discarded. However, what I originally had in mind was setting the hash bang and executable bit with chmod so that you can do this at the command line: ./hello_world.R "Args go here" Once that works, calling it from Galaxy should be trivial. I think one way to do this is with the little r package: http://code.google.com/p/littler/ Peter
On Tue, Oct 12, 2010 at 11:15 AM, Peter <peter@maubp.freeserve.co.uk> wrote:
On Tue, Oct 12, 2010 at 10:43 AM, Freddy <freddy.debree@wur.nl> wrote:
Hi Peter,
Running R from xml is not so straight forward to me. Is the interpreter "bash" or is it "R"? If "bash" then it should be running "R --vanilla --slave input < Rscript > output" if "R" then it should be running ? smthg like "source(Rscript)"
Hope you can help...
Freddy
Consider a simple R script, hello_world.R like this:
print('Hello world'); print(commandArgs());
Then in Galaxy I think you'd need the default interpreter ("bash") and the command to be:
R --vanilla --file=hello_world.R --args "Args go here"
I'm assuming your script will write useful output to a file, and stdout can be discarded.
However, what I originally had in mind was setting the hash bang and executable bit with chmod so that you can do this at the command line:
./hello_world.R "Args go here"
Once that works, calling it from Galaxy should be trivial. I think one way to do this is with the little r package: http://code.google.com/p/littler/
Peter
Hi Freddy, I missed your earlier email about the r_wrapper.sh script that comes with Galaxy, used like this where an R script is created at run time from the XML wrapper using a moderately complex Cheater template: $ grep r_wrapper.sh tools/*/*.xml tools/plotting/xy_plot.xml: <command interpreter="bash">r_wrapper.sh $script_file</command> tools/stats/lda_analy.xml: <command interpreter="sh">r_wrapper.sh $script_file</command> tools/stats/plot_from_lda.xml: <command interpreter="sh">r_wrapper.sh $script_file</command> In these examples only the script name is passed (all the variable information is inside the dynamically generated script file). However, reading the shell script it *should* accept more than one input, but the first input is interpreted as the script name. Maybe that doesn't work since you said you tried it? Clearly there are several ways to approach this (r_wrapper.sh, rpy from python, etc). Guidance from the Galaxy developers on what they consider to be best practice would be welcome. Peter
Exactly, That is what confused me. Earlier I simply tried running it 'straight' and that didn't work so well. Later I saw the wrappers thanks to James Taylor, but then it simply didn't produce output. so basically, I thought, I'm having a problem with in/output, so system problems rahter than R itself. Thus, I was puzzled that galaxy itself was never using ANY R-script and only rpy/xml. But I will give R-straight another good try. I've tried the things you mentioned, but I guess I have never used the "--file=Rscript" option. Will let you know how that goes. Cheers, Freddy On 10/12/2010 12:28 PM, Peter wrote:
On Tue, Oct 12, 2010 at 11:15 AM, Peter<peter@maubp.freeserve.co.uk> wrote:
On Tue, Oct 12, 2010 at 10:43 AM, Freddy<freddy.debree@wur.nl> wrote:
Hi Peter,
Running R from xml is not so straight forward to me. Is the interpreter "bash" or is it "R"? If "bash" then it should be running "R --vanilla --slave input< Rscript> output" if "R" then it should be running ? smthg like "source(Rscript)"
Hope you can help...
Freddy
Consider a simple R script, hello_world.R like this:
print('Hello world'); print(commandArgs());
Then in Galaxy I think you'd need the default interpreter ("bash") and the command to be:
R --vanilla --file=hello_world.R --args "Args go here"
I'm assuming your script will write useful output to a file, and stdout can be discarded.
However, what I originally had in mind was setting the hash bang and executable bit with chmod so that you can do this at the command line:
./hello_world.R "Args go here"
Once that works, calling it from Galaxy should be trivial. I think one way to do this is with the little r package: http://code.google.com/p/littler/
Peter
Hi Freddy,
I missed your earlier email about the r_wrapper.sh script that comes with Galaxy, used like this where an R script is created at run time from the XML wrapper using a moderately complex Cheater template:
$ grep r_wrapper.sh tools/*/*.xml tools/plotting/xy_plot.xml:<command interpreter="bash">r_wrapper.sh $script_file</command> tools/stats/lda_analy.xml: <command interpreter="sh">r_wrapper.sh $script_file</command> tools/stats/plot_from_lda.xml: <command interpreter="sh">r_wrapper.sh $script_file</command>
In these examples only the script name is passed (all the variable information is inside the dynamically generated script file).
However, reading the shell script it *should* accept more than one input, but the first input is interpreted as the script name. Maybe that doesn't work since you said you tried it?
Clearly there are several ways to approach this (r_wrapper.sh, rpy from python, etc). Guidance from the Galaxy developers on what they consider to be best practice would be welcome.
Peter
On Tue, Oct 12, 2010 at 11:40 AM, Freddy <freddy.debree@wur.nl> wrote:
Exactly, That is what confused me.
Earlier I simply tried running it 'straight' and that didn't work so well. Later I saw the wrappers thanks to James Taylor, but then it simply didn't produce output. so basically, I thought, I'm having a problem with in/output, so system problems rahter than R itself.
Thus, I was puzzled that galaxy itself was never using ANY R-script and only rpy/xml. But I will give R-straight another good try. I've tried the things you mentioned, but I guess I have never used the "--file=Rscript" option. Will let you know how that goes.
Cheers, Freddy
As I understand it: * Galaxy will capture stdout/stderr (unless you have redirected them), stdout is discarded, and any text on stderr is treated as an error. * If you want output from a script, it should be sent to a file Since R tends to be very noisy on stdout, it is probably best to ignore stdout and make any output explicit to a file (whose name should be a command line argument which Galaxy will assign). Please keep us posted - I do sometimes write plain R scripts myself, although I have tended to use a Python script and rpy more often. Peter
Just to note for those Perl people developing tools with Galaxy and wanting to interface with R there are a couple of libraries out there to do this for you: RSPerl http://www.omegahat.org/RSPerl/ Statistics::R http://search.cpan.org/search?query=Statistics%3A%3AR
I am not using perl, but bash, which makes things much easier on the cmd line. However, maybe you know how to retrieve the R output or: what exactly is the problem for Galaxy for how to handle the system output from R through the shell? We have seen it here with other tools as well. R should work straight from the cmd line as well. And everything - incl R - does when use bash in a shell, but as soon as you put galaxy in between you and the program the output, is nowhere to be seen so far, we've been able to simply 'mv' the output from the program towards bash and that will push it to galaxy. But with R this didn't work. (so far) On 10/13/2010 12:47 PM, Leandro Hermida wrote:
Just to note for those Perl people developing tools with Galaxy and wanting to interface with R there are a couple of libraries out there to do this for you:
RSPerl http://www.omegahat.org/RSPerl/
Statistics::R http://search.cpan.org/search?query=Statistics%3A%3AR
If I understand you correctly what you want to do you should be able to just do that by a tool.xml config calling an R script and providing a data file. The R script can open that file and process it into some output file known by galaxy.... But maybe I oversee something. In your tool_config grab an R-file and some datafile: <command> R --slave -f $in_r --args $in_data $out_data </command> <inputs> <param name="in_data" type="data" format="tabular" label="Test data file" /> <param name="in_r" type="data" format="text" label="R script to load and execute" /> </inputs> <outputs> <data name="out_data" type="data" format="tabular" label="R script output" /> </outputs> The R script provided will grab the args from the cmd line as you indiciated earlier: # R script file to grab input and output filenames from cmdline and just copy args <- commandArgs() output <- read.table(args[6], header=T) write.table(output,sep="\t",file=args[7],row.names=F) #end script Hope this helped, Alex ________________________________________ Van: galaxy-dev-bounces@lists.bx.psu.edu [galaxy-dev-bounces@lists.bx.psu.edu] namens Freddy [freddy.debree@wur.nl] Verzonden: woensdag 13 oktober 2010 14:19 Aan: galaxy-dev@lists.bx.psu.edu Onderwerp: Re: [galaxy-dev] rpy or perl I am not using perl, but bash, which makes things much easier on the cmd line. However, maybe you know how to retrieve the R output or: what exactly is the problem for Galaxy for how to handle the system output from R through the shell? We have seen it here with other tools as well. R should work straight from the cmd line as well. And everything - incl R - does when use bash in a shell, but as soon as you put galaxy in between you and the program the output, is nowhere to be seen so far, we've been able to simply 'mv' the output from the program towards bash and that will push it to galaxy. But with R this didn't work. (so far) On 10/13/2010 12:47 PM, Leandro Hermida wrote:
Just to note for those Perl people developing tools with Galaxy and wanting to interface with R there are a couple of libraries out there to do this for you:
RSPerl http://www.omegahat.org/RSPerl/
Statistics::R http://search.cpan.org/search?query=Statistics%3A%3AR
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
On Thu, Oct 14, 2010 at 6:44 AM, Bossers, Alex <Alex.Bossers@wur.nl> wrote:
If I understand you correctly what you want to do you should be able to just do that by a tool.xml config calling an R script and providing a data file. The R script can open that file and process it into some output file known by galaxy.... But maybe I oversee something.
In your tool_config grab an R-file and some datafile:
<command> R --slave -f $in_r --args $in_data $out_data </command> <inputs> <param name="in_data" type="data" format="tabular" label="Test data file" /> <param name="in_r" type="data" format="text" label="R script to load and execute" /> </inputs> <outputs> <data name="out_data" type="data" format="tabular" label="R script output" /> </outputs>
The R script provided will grab the args from the cmd line as you indiciated earlier:
# R script file to grab input and output filenames from cmdline and just copy args <- commandArgs() output <- read.table(args[6], header=T) write.table(output,sep="\t",file=args[7],row.names=F) #end script
Hope this helped,
Alex
Wow - very similar to the version I wrote (off list discussion yesterday). You can also try something like this if the script writes to stdout: <command> R --slave -f $in_r --args $in_data > $out_data </command> Peter
Hi, I have made a lot of progress getting the LIMS system up and running and not I'm at the point where I was able to connect to the folders having my sequence data from galaxy. I can select a sample from a list of submitted samples in the tracking form and then match it to a file in our sequence directory. I get an error when trying to transfer the sample to galaxy. Does anybody have any ideas where should I start looking to fix this? Error below. Thanks in advance. Victor Last couple of lines are, NoSectionError: No section: 'data_transfer_user_login_info' On Oct 14, 2010, at 12:44 AM, Bossers, Alex wrote:
If I understand you correctly what you want to do you should be able to just do that by a tool.xml config calling an R script and providing a data file. The R script can open that file and process it into some output file known by galaxy.... But maybe I oversee something.
In your tool_config grab an R-file and some datafile:
<command> R --slave -f $in_r --args $in_data $out_data </command> <inputs> <param name="in_data" type="data" format="tabular" label="Test data file" /> <param name="in_r" type="data" format="text" label="R script to load and execute" /> </inputs> <outputs> <data name="out_data" type="data" format="tabular" label="R script output" /> </outputs>
The R script provided will grab the args from the cmd line as you indiciated earlier:
# R script file to grab input and output filenames from cmdline and just copy args <- commandArgs() output <- read.table(args[6], header=T) write.table(output,sep="\t",file=args[7],row.names=F) #end script
Hope this helped,
Alex
________________________________________ Van: galaxy-dev-bounces@lists.bx.psu.edu [galaxy-dev-bounces@lists.bx.psu.edu] namens Freddy [freddy.debree@wur.nl] Verzonden: woensdag 13 oktober 2010 14:19 Aan: galaxy-dev@lists.bx.psu.edu Onderwerp: Re: [galaxy-dev] rpy or perl
I am not using perl, but bash, which makes things much easier on the cmd line. However, maybe you know how to retrieve the R output or: what exactly is the problem for Galaxy for how to handle the system output from R through the shell? We have seen it here with other tools as well. R should work straight from the cmd line as well. And everything - incl R - does when use bash in a shell, but as soon as you put galaxy in between you and the program the output, is nowhere to be seen
so far, we've been able to simply 'mv' the output from the program towards bash and that will push it to galaxy. But with R this didn't work. (so far)
On 10/13/2010 12:47 PM, Leandro Hermida wrote:
Just to note for those Perl people developing tools with Galaxy and wanting to interface with R there are a couple of libraries out there to do this for you:
RSPerl http://www.omegahat.org/RSPerl/
Statistics::R http://search.cpan.org/search?query=Statistics%3A%3AR
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Error Traceback: View as: Interactive | Text | XML (full) ⇝ NoSectionError: No section: 'data_transfer_user_login_info' URL: http://mir-13:8087/requests_admin/manage_datasets?sort=-create_time&operation=Start+Transfer&id=df7a1f0c02a5b08e Module weberror.evalexception.middleware:364 in respond view
app_iter = self.application(environ, detect_start_response) Module paste.debug.prints:98 in __call__ view environ, self.app) Module paste.wsgilib:539 in intercept_output view app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ view return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ view return self.application(environ, start_response) Module galaxy.web.framework.base:145 in __call__ view body = method( trans, **kwargs ) Module galaxy.web.framework:139 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.requests_admin:469 in manage_datasets view self.__start_datatx(trans, sample_dataset.sample, id_list) Module galaxy.web.controllers.requests_admin:837 in __start_datatx view datatx_user = self.__setup_datatx_user(trans, sample.library, sample.folder) Module galaxy.web.controllers.requests_admin:739 in __setup_datatx_user view email = config.get("data_transfer_user_login_info", "email") Module ConfigParser:511 in get view << except KeyError: if section != DEFAULTSECT: raise NoSectionError(section) # Update with the entry specific variables if vars: NoSectionError: No section: 'data_transfer_user_login_info'
Victor, You will need to setup data transfer in sample tracking before you start using it. This page (http://main.g2.bx.psu.edu/u/rkchak/p/sts-setup-data-transfer ) describes it step by step. Let me know if you have more questions. thanks rc On Oct 14, 2010, at 1:00 PM, Victor Ruotti wrote:
Hi, I have made a lot of progress getting the LIMS system up and running and not I'm at the point where I was able to connect to the folders having my sequence data from galaxy. I can select a sample from a list of submitted samples in the tracking form and then match it to a file in our sequence directory.
I get an error when trying to transfer the sample to galaxy. Does anybody have any ideas where should I start looking to fix this? Error below. Thanks in advance. Victor Last couple of lines are, NoSectionError: No section: 'data_transfer_user_login_info'
On Oct 14, 2010, at 12:44 AM, Bossers, Alex wrote:
If I understand you correctly what you want to do you should be able to just do that by a tool.xml config calling an R script and providing a data file. The R script can open that file and process it into some output file known by galaxy.... But maybe I oversee something.
In your tool_config grab an R-file and some datafile:
<command> R --slave -f $in_r --args $in_data $out_data </command> <inputs> <param name="in_data" type="data" format="tabular" label="Test data file" /> <param name="in_r" type="data" format="text" label="R script to load and execute" /> </inputs> <outputs> <data name="out_data" type="data" format="tabular" label="R script output" /> </outputs>
The R script provided will grab the args from the cmd line as you indiciated earlier:
# R script file to grab input and output filenames from cmdline and just copy args <- commandArgs() output <- read.table(args[6], header=T) write.table(output,sep="\t",file=args[7],row.names=F) #end script
Hope this helped,
Alex
________________________________________ Van: galaxy-dev-bounces@lists.bx.psu.edu [galaxy-dev-bounces@lists.bx.psu.edu ] namens Freddy [freddy.debree@wur.nl] Verzonden: woensdag 13 oktober 2010 14:19 Aan: galaxy-dev@lists.bx.psu.edu Onderwerp: Re: [galaxy-dev] rpy or perl
I am not using perl, but bash, which makes things much easier on the cmd line. However, maybe you know how to retrieve the R output or: what exactly is the problem for Galaxy for how to handle the system output from R through the shell? We have seen it here with other tools as well. R should work straight from the cmd line as well. And everything - incl R - does when use bash in a shell, but as soon as you put galaxy in between you and the program the output, is nowhere to be seen
so far, we've been able to simply 'mv' the output from the program towards bash and that will push it to galaxy. But with R this didn't work. (so far)
On 10/13/2010 12:47 PM, Leandro Hermida wrote:
Just to note for those Perl people developing tools with Galaxy and wanting to interface with R there are a couple of libraries out there to do this for you:
RSPerl http://www.omegahat.org/RSPerl/
Statistics::R http://search.cpan.org/search?query=Statistics%3A%3AR
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Error Traceback: View as: Interactive | Text | XML (full) ⇝ NoSectionError: No section: 'data_transfer_user_login_info' URL: http://mir-13:8087/requests_admin/manage_datasets?sort=- create_time&operation=Start+Transfer&id=df7a1f0c02a5b08e Module weberror.evalexception.middleware:364 in respond <plus.jpg> view
app_iter = self.application(environ, detect_start_response) Module paste.debug.prints:98 in __call__ <plus.jpg> view environ, self.app) Module paste.wsgilib:539 in intercept_output <plus.jpg> view app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ <plus.jpg> view return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ <plus.jpg> view return self.application(environ, start_response) Module galaxy.web.framework.base:145 in __call__ <plus.jpg> view body = method( trans, **kwargs ) Module galaxy.web.framework:139 in decorator <plus.jpg> view return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.requests_admin:469 in manage_datasets <plus.jpg> view self.__start_datatx(trans, sample_dataset.sample, id_list) Module galaxy.web.controllers.requests_admin:837 in __start_datatx <plus.jpg> view datatx_user = self.__setup_datatx_user(trans, sample.library, sample.folder) Module galaxy.web.controllers.requests_admin:739 in __setup_datatx_user <plus.jpg> view email = config.get("data_transfer_user_login_info", "email") Module ConfigParser:511 in get <plus.jpg> view << except KeyError: if section != DEFAULTSECT: raise NoSectionError(section) # Update with the entry specific variables if vars: NoSectionError: No section: 'data_transfer_user_login_info'
galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
participants (6)
-
Bossers, Alex -
Freddy -
Leandro Hermida -
Peter -
Ramkrishna Chakrabarty -
Victor Ruotti