- galaxy-dev - lists.galaxyproject.org

Are there examples of tools that run processes in the background?
by Melissa Cline 06 Jun '14

06 Jun '14

Hi folks, We're working on developing a VM which we'd like to be tightly coordinated with Galaxy, such that VM users can export data from the VM into Galaxy, perform further analyses on that data, and then import the analysis results into the VM for storage (from where it will serve additional applications). Right now, I'm picturing this as a set of Galaxy tools that launch the VM, manage the data import and export, and shut down the VM at the appropriate time. Or is there a better approach I didn't consider? Assuming the answer is "no", then are there any tools in the toolshed that launch background processes? If so, I'd like to look at them. And assuming that my VM will be running in the background, I'd like it to shut down when Galaxy shuts down. Is there a way to configure a process to be run when it's time to quit Galaxy? Thanks! Melissa

2 1

to put or not to put a tool suite into the toolshed
by Wolfgang Maier 06 Jun '14

06 Jun '14

Dear all, I am maintaining a (still young) suite of command line tools (written in Python) for identification of mutations in model organism genomes through whole-genome sequencing (https://sourceforge.net/projects/mimodd/) MiModD aims at geneticists that do not have much background in bioinformatics, and it's supposed to make WGS analysis for small model organism genomes (anything from yeast to fish) possible on regular PCs, so it's not a cloud/cluster solution. We do support Galaxy though through a complete set of tool wrappers for three reasons: - to provide a graphical user interface - to offer labs the possibility to install the software on one dedicated machine, but run analyses from any machine (typically with Windows installed) - to keep analysis workflows documented and reproducible. Currently, we advise users to take advantage of these features and install a local instance of Galaxy even though it will be running then only on a single desktop PC (or even just a notebook). After installation of Galaxy our software simply copies its wrapper xmls over to the tools folder and modifies the tool_conf file to integrate itself. In addition, users have to install a bit of other third-party software (samtools, snap aligner, optionally snpeff) that our code relies on. [End of lengthy introduction] So my question is: in what way could the package profit from being uploaded to a Galaxy toolshed ? I guess it would mean quite some extra work from my side since I'm not familiar with the whole procedure, so are there benefits (visibility, ease of installation, etc.) that are worth the effort ? Thanks a real lot for any feedback, Wolfgang

3 3

ToolShed for Visualisation Registry?
by Saket Choudhary 06 Jun '14

06 Jun '14

I was wondering if there is a possibility of a 'Tool'Shed for Visualisation registry plugins(custom plugins, developed by users/developers)? The motivation for this comes from a small 'tool' I wrote for visualiing fastq phred scores using box plots[Still under development] here: http://saketkc.github.io/biojs/src/test/javascript/TestPhredScoreViewer.html I am relying on d3.text to read my fastq, but I believe I could handle this in a much better way had I written it for Galaxy's Visualisation registry specifically. For e.g. I could fetch fastq scores line by line using Galaxy's API, which would make my task a bit easier. I could always keep this to my local instance of Galaxy, but a 'toolshed' or 'registry' for visualisation would help. Saket

2 1

Re: [galaxy-dev] to put or not to put a tool suite into the
by Kandalaft, Iyad 06 Jun '14

06 Jun '14

Hello Wolfang, I agree that developing the Galaxy "way" is not always "easy". The main advantage of galaxy's toolshed is a point that you stated in your introduction "reproducibility". Galaxy's toolshed enforces proper versioning of tool wrappers and its dependency on underlying software. Hence, reproducibility is taken care of automatically between galaxy and the toolshed. In addition, it automates one of the steps for your users: "dependencies on samtools, etc, etc". Some of these dependencies are already packages in the galaxy central toolshed (such as samtools). Therefore, using the tool shed is advantageous. Pushing the repositories to the central tool shed is a way of giving back to the galaxy/research community that you and I depend on. Regards, Iyad Kandalaft Bioinformatics Application Developer Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada KW Neatby Bldg | éd. KW Neatby 960 Carling Ave| 960, avenue Carling Ottawa, ON | Ottawa (ON) K1A 0C6 E-mail Address / Adresse courriel: Iyad.Kandalaft(a)agr.gc.ca Telephone | Téléphone 613- 759-1228 Facsimile | Télécopieur 613-759-1701 Government of Canada | Gouvernement du Canada

1 0

Installing galaxy with Apache
by Matthias Enders 06 Jun '14

06 Jun '14

Hi, this thread is likely related to this one: http://user.list.galaxyproject.org/Installing-galaxy-with-Apache-td4654499.… I installed Galaxy on a Ubuntu Server 12.04 system. I followed the instruction on the link mentioned above and this one: https://wiki.galaxyproject.org/Admin/Config/ApacheProxy Now my issue is: When I got this configuration in my /etc/apache2/sites-available/default : RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8080$1 [P] Galaxy is visible at www.my_server_url.de<http://www.my_server_url.de> but not at www.my_server_url.de/galaxy<http://www.my_server_url.de/galaxy> Following the later URL I got this message: Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET /galaxy<http://www.gsa.npz.de/galaxy>. Reason: DNS lookup failure for: localhost:8080http: ________________________________ Apache/2.2.22 (Ubuntu) Server at www.my_server_url.de Port 80 When I change the configuration like this: (make the second line a comment): RewriteEngine on #RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8080$1 [P] Galaxy is visible both at www.my_server_url.de<http://www.my_server_url.de> and at www.my_server_url.de/galaxy<http://www.my_server_url.de/galaxy> When I completely remove this part, the "It works!" standard page of apache is displayed. The configuration in my universe.ini is as follows: host = 127.0.0.1 [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /galaxy [app:main] filter-with = proxy-prefix cookie_path = /galaxy Can you please help me with this issue? I want to see the "It works!" page at www.my_server_url.de<http://www.my_server_url.de> and galaxy only at www.my_server_urla.de/galaxy. Thanks in advance! Kind regards Matthias Enders

2 2

galaxy is not sourcing tool environment
by Isabelle Phan 05 Jun '14

05 Jun '14

I am using env.sh to manage my tool dependencies: # tool_dependency_dir/hhsuite/2.0.16/env.sh export HHPRED_DIR=/opt/galaxy-dist/tool_dependencies/hhsuite/default export HHLIB=${HHPRED_DIR}/lib/hh export PATH=${HHPRED_DIR}/bin:${HHLIB}/scripts:$PATH Galaxy is not sourcing it. I have to manually source it before I restart Galaxy. I am following "Using the env.sh file" and cannot get it to work. What am I missing? Any help greatly appreciated, thanks, Isabelle

2 2

Manual Tool and dependency installation
by Ravi Alla 05 Jun '14

05 Jun '14

Hi All, Due to the nature of our cluster we are not allowed to have compilers on our webserver, which is where our galaxy instance runs. Because of this tool dependencies and tools that need to be compiled fail to install through the galaxy tool shed webpage. Is there a way for me to install dependencies and tools manually? For example I am trying to install the GATK2 wrapper. This depends on samtools-0.1.19 from iuc which in turn requires ncurses-5.9. Is there instructions somewhere as to where to install these tools such that they are tied to the GATK2 wrapper? Pardon me if this is a naive question, but I am really out of sorts here. Thank you Ravi

3 2

Re: [galaxy-dev] tool dependencies fail
by Isabelle Phan 05 Jun '14

05 Jun '14

I tried <requirements> <requirement type="package">hhsuite</requirement> </requirements> and that had no effect. I. ------------------------------ Message: 3 Date: Sat, 31 May 2014 03:51:55 +0200 From: Bj?rn Gr?ning <bjoern.gruening(a)gmail.com> To: galaxy-dev(a)lists.bx.psu.edu Subject: Re: [galaxy-dev] tool dependencies fail Message-ID: <538935BB.7020905(a)gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi, can you try: <requirements> <requirement type="package">hhsuite</requirement> </requirements> Hope that helps, sorry for the short answer I'm too tired :) Bjoern

3 4

About GALAXY customisation
by Patrick Tran Van 05 Jun '14

05 Jun '14

Hi, I've done a local installation of Galaxy. Everything work but I just have some questions about customisation: 1)How can I modify the main page (for example: new image, twitter integration etc.) as seen on the main page of galaxy (https://usegalaxy.org/) ? 2)How can I write title on the toolbox menu (eg : NGS Toolbox Beta) 3)Is it possible to upload a file without copy it and without being as admin but as simple user ? I know we can upload a file without copy in Galaxy (https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles?ac…) but I want to do it without being admin. Thanks for your help Patrick Tran Van INSA Lyon 4ème année Bio-Informatique et Modélisation

2 1

Pre pull-request: CLI and hidden data
by Ganote, Carrie L 04 Jun '14

04 Jun '14

Hi Devs, I'd like to open up some discussion about incorporating some code bits into the Galaxy distribution. My code is here: https://bitbucket.org/cganote/osg-blast-galaxy First off, I'd like to say that these changes were made initially as hacks to get Galaxy working with a grid interface for our nefarious purposes. For us, the results have been spiffy, in that we can offload a bunch of Blast work off of our own clusters and onto the grid, which processes them fast on a distributed set of computers. In order to do this, I wanted to be able to take as much control over the process as I could. The destination uses Condor, but it used condor_dag to submit jobs - that means I would have to modify the condor job runner. The destination needed to have the files shipped over to it first - so I had to be able to stage. This made lwr attractive, but then I would need to guarantee that the server at the other end was running lwr, and since I don't have control of that server, this seemed less likely to be a good option. The easiest thing for me to understand was the cli runner. I could do ssh, I could do scp, so this seemed the best place to start. So I started by trying to figure out which files needed to be sent to the server, and then implementing a way to send them. I start with stdout, stderr and exit code files. I also want to stage any datasets that are in the param_dict, and anything that is in extra_files_path. Then we alter the command line that is run such that all the paths make sense on the remote server, and to make sure that the right things are run remotely vs. locally (i.e., metadata.sh is run locally after job is done). Right now, this is done by splitting the command line on a specific string, which is not robust for future changes to the command_factory, but I'm open to suggestions. So, here's one hack. The hidden data tool parameter is something I hijacked - as far as I can tell, hidden data is only used for Cufflinks, so it seemed safe. I use it to send the shell script that will be run on the server (but NOT sent to the worker nodes). It needed to be a DATA type so that my stager would pick it up and send it over. I wanted it to be hidden because it was only used by the tool and it should not need to be an HDA. I made changes to allow the value of the hidden data to be set in the tool - this would become the false_path of the data, which would then become its actual path. Please have a look, and ask questions, and if there are improvements needed before anything is considered for pulling, let me know. I'd like to present this at the Galaxy conference without having vegetables thrown at me. Thanks! -Carrie Ganote National Center for Genome Analysis Support Indiana University

2 1