August 2013 - galaxy-dev - lists.galaxyproject.org

Integration with remote SAAS
by Joe Stubbs 20 Aug '13

20 Aug '13

Hello, I'm working on a project to integrate a Galaxy instance with a remote software execution service. The service maintains a catalogue of applications, storage and execution hosts, files (input and output) and jobs (submitted, executing, finished, etc) all manageable through REST endpoints. For the integration, I'd like apps, datasets and jobs in the service to be available through Galaxy. RIght now, I'm querying the service to retrieve the catalogue of apps available to a user and, after some conversion, inserting them into the Galaxy toolbox. For these apps, jobs are submitted via the REST API and executed on the execution host registered in the service. I'm currently polling a status endpoint in the service to keep Galaxy updated with the job's status. I'd like the I/O to be seamless for the users, so that datasets generated from "native" Galaxy tools/jobs can be used as inputs for service apps and vice versa. Similarly, I would like to take advantage of Galaxy's data viewer for datasets generated by remote jobs. Since the service can handle files published to URLs, I was looking into that approach. It looks like Galaxy has some support for this -- perhaps using redirect params? Is that right, and if so, is there documentation on what is currently supported in Galaxy? Does this seem like the best approach? Among other wish list items, it would be nice if the toolbox (or at least the catalogue of apps in the service) could be refreshed real-time. Right now they are only being added at server start up. Quite generally, I would welcome any comments or suggestions on the overall integration approach as I am fairly new to Galaxy. Thanks in advance, Joe -- Joe Stubbs Research Engineering Scientist Associate Texas Advanced Computing Center University of Texas, Austin

1 0

Bullet points not showing in README.rst on Tool Shed
by Peter Cock 20 Aug '13

20 Aug '13

Hi all, I'm having a problem with bullet point lists in my README.rst file not rendering nicely on the (main and Test) Tool Shed. This example of mine shows the bullet points fine on GitHub's rendering (two bullet lists examples): https://github.com/peterjc/picobio/blob/424747ecccf5cce6ce327754e6f8e57a594… However, on the Tool Shed the bullet points are missing: http://testtoolshed.g2.bx.psu.edu/view/peterjc/rxlr_venn_workflow/de29af5da… Here is the relevant reStructuredText documentation: http://docutils.sourceforge.net/docs/user/rst/quickref.html#bullet-lists As per the documentation, I have a blank line before and after the bullets, and I am using one other three valid bullet characters ("*", rather than "-" or "+"), e.g. * 89 RXLRs using Whisson et al. (2007) * 124 RXLRs using Win et al. (2007) * 162 RXLRs using Bhattacharjee et al. (2006) In fact, looking at the HTML source on the Tool Shed it has correctly rendered as an unordered list: <ul class="simple"> <li>89 RXLRs using Whisson et al. (2007)</li> <li>124 RXLRs using Win et al. (2007)</li> <li>162 RXLRs using Bhattacharjee et al. (2006)</li> </ul> Is there a bug in the default Tool Shed CSS which is hiding the bullet points? Thanks. Peter

2 1

Error when I will set "set at runtime" in a workflow
by Ramon Tiburski 20 Aug '13

20 Aug '13

I have a tool that uses two dates to fetch data in a database. I am using a parameter type called "datetime" in the field "type". It uses javascript and allows a date to be selected through a calendar. When I use it as a tool works perfectly, but when I try to use the workflow got an error. When I check the "Set at runtime" I got the following error: Server Error URL: http://localhost:8080/workflow/editor_form_post Module paste.exceptions.errormiddleware:144 in __call__ >> app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ >> environ, self.app) Module paste.wsgilib:543 in intercept_output >> app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:185 in handle_request >> body = method( trans, **kwargs ) Module galaxy.web.framework:73 in decorator >> return simplejson.dumps( func( self, trans, *args, **kwargs ) ) Module galaxy.webapps.galaxy.controllers.workflow:607 in editor_form_post >> 'tool_state': incoming.pop("tool_state") KeyError: 'tool_state' and I can not create the workflow. What can I do? Thank you

1 0

Getting data from a database to create a dynamic combo box
by Ramon Tiburski 20 Aug '13

20 Aug '13

Hi, I need to create a combo box that will take data from a database dynamically. How could I do this? Thank you

4 8

Licensing Galaxy workflows; are they software or documents?
by Peter Cock 20 Aug '13

20 Aug '13

Hello all, A philosophical question - for my Galaxy tools and wrappers, I have been using open source software (OSS) licences, e.g. the MIT license, or GPL. For licensing my Galaxy workflows, should I also treat them as software and do the same, or as a protocol document and go for something like one of the Creative Commons licenses? e.g. CC BY, or CC BY-SA Thanks, Peter

4 6

Problem when I submit 'gatk_wrapper.py' task with my job scheduling system arranged on my sever.
by Wei Liu 20 Aug '13

20 Aug '13

Dear everyone, These days I want to localize Galaxy on our server. When I tried to arrange gatk, I met a problem as follow. Firstly, I run the script 'gatk_wrapper.py' like this: python /bio-db/apps/api/galaxy-galaxy-dist-e6444e7a1685/tools/gatk/gatk_wrapper.py -d -I /share/home/wuyilei/bioc/test/111/test1.bbioc/test/111/test1.bam .bam test1 -p /bio-db/apps/pool/java1.6/jre1.6.0_24/bin/java -jar /bio-db/apps/api/galaxy-galaxy-dist-e6444e7a1685/shared/jars/gatk/GenomeAnalysisTK.jar -R /bio-db/reference-genome/hg19.fa -T DepthOfCoverage But I got an error message like this: gatk_wrapper.py: error: no such option: -j Then I tried to add a couple of single quotes on the parameters behind of -d and -p, just like the following: python /bio-db/apps/api/galaxy-galaxy-dist-e6444e7a1685/tools/gatk/gatk_wrapper.py -d '-I' /share/home/wuyilei/bioc/test/111/test1.bbioc/test/111/test1.bam .bam test1 -p '/bio-db/apps/pool/java1.6/jre1.6.0_24/bin/java -jar /bio-db/apps/api/galaxy-galaxy-dist-e6444e7a1685/shared/jars/gatk/GenomeAnalysisTK.jar -R /bio-db/reference-genome/hg19.fa -T DepthOfCoverage ' Then 'gatk_wrapper.py' can work well! But if I use the ob scheduling system which was arranged on my server, I can not run the 'gatk_wrapper.py' script any more. After a long time research, I found the problem, all because of the single quotes behind of -d and -p. Can anyone help me, for example, rewrite the script 'gatk_wrapper.py' for me, then I can delete the single quotes! Or give me some other advice. Looking forward your reply! Best, Wei Liu.

1 1

Re: [galaxy-dev] OSError: [Errno 5] Input/output error
by Hans-Rudolf Hotz 20 Aug '13

20 Aug '13

Please keep all replies on the list by using "reply all" I am not really a tool shed expert, hence I can only direct you to the wiki pages: http://wiki.galaxyproject.org/Tool%20Shed Hans-Rudolf On 08/20/2013 12:12 PM, Richard Kuo wrote: > Hello Hans-Rudolf, > > I am wondering if the issue could have been caused by the tool shed? I > had been playing around with adding new tools via the tool shed just > before this happened. The sys admin has restored our Galaxy instance to > a previous time point before I did all this and it seems to be working > alright now. Is there any documentation of how the tool shed interacts > with the conf files within the galaxy folder? > > Thank you! > Richard > > > On Mon, Aug 19, 2013 at 12:32 PM, Hans-Rudolf Hotz <hrh(a)fmi.ch > <mailto:hrh@fmi.ch>> wrote: > > Hi Richard > > this looks like a system error to me (and has nothing to do with the > galaxy server itself). Something is wrong with your disk? which > might also be the original reason why you got complaints about the > upload speed. > > Sorry, I can't help you any further. I suggest you contact your sys > admin. > > > Regards, Hans-Rudolf > > > > On 08/19/2013 01:07 PM, Richard Kuo wrote: > > Hello, > > I tried restarting my local Galaxy instance today bit it won't > start up > properly. I keep getting this message when I run "sh run.sh": > > Traceback (most recent call last): > File > "/mnt/ris-fas1a/linux_users/__avnglxy/ris-lx10/galaxy-dist/__lib/galaxy/webapps/galaxy/__buildapp.py", > line 35, in app_factory > app = UniverseApplication( global_conf = global_conf, > **kwargs ) > File > "/mnt/ris-fas1a/linux_users/__avnglxy/ris-lx10/galaxy-dist/__lib/galaxy/app.py", > line 79, in __init__ > self.datatypes_registry.load___datatypes( self.config.root, > self.config.datatypes_config ) > File > "/mnt/ris-fas1a/linux_users/__avnglxy/ris-lx10/galaxy-dist/__lib/galaxy/datatypes/registry.__py", > line 273, in load_datatypes > self.to_xml_file() > File > "/mnt/ris-fas1a/linux_users/__avnglxy/ris-lx10/galaxy-dist/__lib/galaxy/datatypes/registry.__py", > line 681, in to_xml_file > os.close( fd ) > OSError: [Errno 5] Input/output error > > > The reason for my restart was dur to a users complaint about > upload speeds. > > Any help is much appreciated! > Richard > > > _____________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/__search/mailinglists/ > <http://galaxyproject.org/search/mailinglists/> > >

1 0

Showing images in README.rst on Tool Shed
by Peter Cock 20 Aug '13

20 Aug '13

Hi all, Thanks to Bjoern's recent enhancement, most of the Tool Shed pages now render a repository's README file as reStructuredText if the filename ends *.rst [I've noticed a bug where during repository installation the README file is just shown as plain test - I'm not sure if this a Galaxy bug or a Tool Shed bug though.] What I wanted to ask today is can we include images in the rendered README file? For instance a small PNG showing sample output, or any other kind of documentation: http://docutils.sourceforge.net/docs/ref/rst/directives.html#image I've tried it here but it fails: http://testtoolshed.g2.bx.psu.edu/view/peterjc/rxlr_venn_workflow/2dfe8b788… Assuming I want to show a local image included within the Tool Shed repository, I presume this would require tweaks to get the image served by the Tool Shed at the appropriate relative URL? I can work-around this by using a full http URL to a publicly hosted image, but that may raise privacy concerns: http://testtoolshed.g2.bx.psu.edu/view/peterjc/rxlr_venn_workflow/d89313312… Thoughts? Peter

2 2

Note about running functional tests.
by Dave Bouvier 20 Aug '13

20 Aug '13

Dear Galaxy admins, As of 10408:6822f41bc9bb stable/10409:2ca0b65d5005 default, a bug has been fixed where running functional tests could overwrite some files in your Galaxy path. If you intend to run functional tests, it is recommended to update to the relevant revision on the branch you are running. Additionally, it is not recommended to run functional tests on a production instance, since the test suite alters a number of configuration settings on the fly, with the result being that it is not a reliable test of the exact production environment, which may give misleading results. -- --Dave B.

3 5

Apache authentication UCSC error
by Shane Sturrock 20 Aug '13

20 Aug '13

I've followed the instructions on setting up Apache Authentication (have to authenticate against an Active Directory server) here http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy and it doesn't work for external display sites unfortunately. The section on display sites suggests the following: <Location "/root/display_as"> Satisfy Any Order deny,allow Deny from all Allow from hgw1.cse.ucsc.edu Allow from hgw2.cse.ucsc.edu Allow from hgw3.cse.ucsc.edu Allow from hgw4.cse.ucsc.edu Allow from hgw5.cse.ucsc.edu Allow from hgw6.cse.ucsc.edu Allow from hgw7.cse.ucsc.edu Allow from hgw8.cse.ucsc.edu </Location> but that doesn't stop Apache requiring authentication. Only putting the Allow from hgw1.csc.ucsc.edu lines in the Location "/" directive gets past that but then I hit another issue which didn't previously exist where I get an error "redirected to non-http(s): /static/user_disabled.html" which is new since I have had it work with this current install. It is possible that CentOS has pushed an update onto my server that has changed some subtle behaviour I guess but it is eluding me entirely and Google isn't doing me much good (Samsung, why did you have to exist?!) For the moment I've disabled the UCSC sites but that isn't a long term solution and I'm stuck because I have to use Apache and AD authentication so I would appreciate hearing from anyone who has figured out how to get this to work. Shane -- For urgent cases, contact support(a)biomatters.com Dr Shane Sturrock shane.sturrock(a)biomatters.com Senior Scientist Tel: +64 (0) 9 379 5064 76 Anzac Ave Auckland New Zealand

1 0