April 2014 - galaxy-dev - lists.galaxyproject.org

Using Galaxy ToolShed API for uploading a tar-ball
by Peter Cock 10 Apr '14

10 Apr '14

Hi all, I've just skimmed Greg's latest blog post which is about the REST API for the ToolShed: http://gregvonkuster.org/galaxy-tool-shed-restful-api/ I would be interested in automating pushing updates to the ToolShed, which I currently do manually by uploading a tar-ball. Is it possible yet to upload a tar-ball via the API? Thanks, Peter

2 1

Jobs deleted staying in 'dr' status
by Mathieu Bahin 10 Apr '14

10 Apr '14

Hi all, We have been developing our own Galaxy instance for a while now. We have a cluster on which the job are sent to be executed, it is managed through SGE. Usually, communication between SGE and DRMAA is ok and we don't have any problem with that. When a job is deleted by the user, most of the times, the job disappears but sometimes, we don't know why, the job stays and has the status 'dr' within SGE. If we don't kill it 'manually', it stays forever. It is not always the same tools which produces this error . Have you any idea why how manage it ? We have another problem, a display one. Since the last update, we experienced problem with the history. Launching a job, it was very long to appear in the history and we had to refresh. We got a more recent version of firefox and it seems that we don't have the problem anymore. But some user of ours can't update their firefox. Are you aware of that problem ? Cheers, Genouest Platform, Rennes - FRANCE

4 3

Error running workflow via API
by John Eppley 09 Apr '14

09 Apr '14

The API interface is translating my parameters into python repr strings instead of actual strings. A command that should look like this: createPrimerFile.py /minilims/galaxy-data/primerTemplates/nextera.2x.primers.fasta NNNNNN NNNNNN > /slipstream/galaxy/production/data/files/020/dataset_20664.dat is getting run as this: createPrimerFile.py /minilims/galaxy-data/primerTemplates/nextera.2x.primers.fasta __lt__galaxy.tools.parameters.basic.RuntimeValue object at 0x792be10__gt__ NNNNNN > /slipstream/galaxy/production/data/files/020/dataset_20728.dat The input table in the tool info page is: Input Parameter Value Note for rerun chemistry type: <galaxy.tools.parameters.basic.RuntimeValue object at 0x7fee4425ca10> First barcode NNNNNN Second (optional) barcode NNNNNN I’m running a slightly out of date version of Galaxy (I think it is change set 11080:87b586afb054) because this is BioTeam’s Slipstream appliance. Is there any chance this is a known bug? I didn’t see anything in Trello, but I’m really not sure I’ve got the hang of Trello yet. Thanks -john

2 2

Launching a workflow via API, multiple params for one tool
by John Marmaduke Eppley 09 Apr '14

09 Apr '14

As far as I can figure out, the parameter map for running a workflow via the API is supposed to look like this: {'blastn': {'param': 'evalue', 'value': '1e-06’}} This doesn’t seem to allow for multiple parameters to a single tool. Is there a way to submit multiple parameters that I’m missing? For example: {'blastn': {'param': 'evalue', 'value': '1e-06’, 'param': ‘identity_cutoff’, ‘value’: ’70’}} …will not work. The dict() container doesn’t allow duplicate keys, it just overwrites the values. Thanks -john

1 1

jobs stuck in new state
by David Hoover 09 Apr '14

09 Apr '14

I have many jobs stuck in the 'new' state on our local Galaxy instance. The jobs can't be stopped using the Admin->Manage jobs tool. First, does anyone know why a job would get stuck in the 'new' state for weeks? I have cleaned things up by manually setting their states to 'error' in the MySQL database. Is there a better way of dealing with 'new' jobs? BTW, our Galaxy instance was updated about two weeks ago. Wondering, David Hoover Helix Systems Staff

4 4

Error getting Detrprok_wf working on local Galaxy instance
by Bernardo Bello 09 Apr '14

09 Apr '14

Hi, While problems are been solved in my other Galaxy instance (MAC OSX), I tried to install Galaxy on local Linux machine running Ubuntu 12.04. Surprisingly, seems I have been able to get Galaxy toolshed work. The problem I have now is that I'm trying to run a published workflow, named ' Detrprok_wf <http://127.0.0.1:8080/workflow/run?id=f597429621d6eb2b>', that requires S-Mart installed and Detrprok scripts installed. Both of them are installed, however although I've also been able to import the corresponding wokflow 'Detrprok_wf', seems there is something wrong. See attached files. https://drive.google.com/file/d/0B8-ZAuZe8jldNE5ncktjTFZ6SG8/edit?usp=shari… https://drive.google.com/file/d/0B8-ZAuZe8jldbTVxTTM3Ujluems/edit?usp=shari… Thanks for your help, Bernardo Command and actions up to now: python --version hg clone -v https://bitbucket.org/galaxy/galaxy-dist/ cd galaxy-dist hg update stable sh run.sh #7. Workflow installation instructions #The workflow should be run on a local Galaxy instance or one hosted on a distant server. Refer to the tutorial ( http://wiki.galaxyproject.org/Admin/Get_Galaxy) for creation of a #Galaxy instance. As the workflow has several steps running in parallel, the Galaxy instance should use a database management system supporting concurential access such as #postgreSQL, instead of the default SQLite system. The DETR'PROK pipeline is obtained from the Galaxy main tool shed ( http://toolshed.g2.bx.psu.edu/) or my experiment (http://# www.myexperiment.org) searching by name "detrprok_wf", and should be installed by the Galaxy instance administrator. DETR'PROK requires installation of the S-MART toolbox [13] and 3the "detrprok_scripts", both available in the Galaxy main tool shed. done # replace admin = none by admin = popnard(a)gmail.com in 'universe_wsgi.ini' file lets see if this is not essential # postgreSQL installation ( https://help.ubuntu.com/community/PostgreSQL) done # The DETR'PROK pipeline is obtained from the Galaxy main tool shed ( http://toolshed.g2.bx.psu.edu/ done ## DETR'PROK requires installation of the S-MART toolbox [13] and the "detrprok_scripts" # S-MART needs the folowing packages R --slave --no-save --no-restore --quiet -e 'if("RColorBrewer" %in% rownames(installed.packages()) == FALSE){install.packages("RColorBrewer", repos = c("http://cran.rstudio.com/"), dependencies = TRUE)}' R --slave --no-save --no-restore --quiet -e 'if("Hmisc" %in% rownames(installed.packages()) == FALSE){install.packages("Hmisc", repos = c("http://cran.rstudio.com/"), dependencies = TRUE)}' # set 'tool_dependency_dir' mkdir tool_dependency_dir # http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies tool_dependency_dir = /home/usuario/tool_dependency_dir # install 'detrprok_wf' nad from 'repository actions' import workflow to Galaxy NOTE -> Workflow Detrprok_wf imported successfully. The following tools required by this workflow are missing from this Galaxy instance: name: bam_to_sam, id: BAM-to-SAM. done # install 'bam_to_sam' Install S-MART dependencies -- *Bernardo Bello Ortí* PhD student CReSA-IRTA Campus de Bellaterra-Universitat Autònoma de Barcelona Edifici CReSA 08193 Bellaterra (Barcelona, Spain) Tel.: 647 42 52 63 *www.cresa.es <http://www.cresa.es> *

1 1

Installing S-Mart on Mac OSX local instance (updated)
by Bernardo Bello 09 Apr '14

09 Apr '14

Hi, I am having problems to install a tool from tool shed named S_MART on my local Galaxy. In fact, its the first tool I'm trying to install. The reason to install S-MART is to use DETR'PROK in Galaxy. Here is the 'error' Galaxy shows after installing some programs from toolshed.<https://drive.google.com/file/d/0B8-ZAuZe8jldN3ptYzE5N0p2ZGM/edit?usp=shari…> I've seen this related post, but I don't know if we are talking about the same problem.<http://dev.list.galaxyproject.org/Problem-installing-tool-Galaxy-local-td46…> Here are the commands aI've used up to now: Thanks, Bernardo P.S. Sorry for the duplicated email, I mistyped the subject in the previous one. hg clone https://bitbucket.org/galaxy/galaxy-dist/ cd galaxy-dist hg update stable sh run.sh # gave errors, so see below # solve problem at startup ( https://www.mail-archive.com/galaxy-dev@lists.bx.psu.edu/msg13354.html) % export LC_ALL='en_US.UTF-8' % sh run.sh done # Go to http://localhost:8080/ and register as popnard(a)gmail.com # Setup admin user (https://wiki.galaxyproject.org/Admin/Interface) To give a user Galaxy admin privileges, add their Galaxy login ( email ) to the list in the following config setting in the Galaxy configuration file universe_wsgi.ini. admin_users = popnard(a)gmail.com # Set 'Tool Dependencies' https://wiki.galaxyproject.org/Admin/Config/ToolDependencies?action=show&re… mkdir tool_dependency_dir done # add tool dependencies folder to 'universe_wsgi.ini' -> Galaxy provides a method for managing the dependencies of Galaxy tools installed from the Tool Shed. In this case, it is simply necessary to set the tool_dependency_dir option of universe_wsgi.ini to a path writable by the Galaxy server. done # restart server First, I modified the universe_wsgi.ini file by removing the "#" before the tool_config_file and tool_path lines. # http://user.list.galaxyproject.org/Installing-new-tools-from-the-tool-shed-… # Then, I created a shed_tools folder in my $HOME directory before modifying the shed_tool_conf.xml file <?xml version="1.0"?> <toolbox tool_path="$HOME/shed_tools"> </toolbox> done # INSTALL R done # ADD R PACKAGES R --slave --no-save --no-restore --quiet -e 'if("RColorBrewer" %in% rownames(installed.packages()) == FALSE){install.packages("RColorBrewer", repos = c("http://cran.rstudio.com/"), dependencies = TRUE)}' R --slave --no-save --no-restore --quiet -e 'if("Hmisc" %in% rownames(installed.packages()) == FALSE){install.packages("Hmisc", repos = c("http://cran.rstudio.com/"), dependencies = TRUE)}' -- *Bernardo Bello Ortí* PhD student CReSA-IRTA Campus de Bellaterra-Universitat Autònoma de Barcelona Edifici CReSA 08193 Bellaterra (Barcelona, Spain) Tel.: 647 42 52 63 *www.cresa.es <http://www.cresa.es> *

2 2

Data Tables and *.loc files: Using named columns versus from_data_table
by Peter Cock 09 Apr '14

09 Apr '14

Hi all, In discussion about adding an NCBI BLAST data manager https://github.com/peterjc/galaxy_blast/issues/22 based on Dan's example, Michael Li has suggested using the new(ish) Data Table functionality of Galaxy for using *.loc files: https://wiki.galaxyproject.org/Admin/Tools/Data%20Tables Currently the BLAST+ wrappers access the blastdb.loc file for picking a system installed nucleotide BLAST database like this: <param name="database" type="select" label="Nucleotide BLAST database"> <options from_file="blastdb.loc"> <column name="value" index="0"/> <column name="name" index="1"/> <column name="path" index="2"/> </options> </param> See https://github.com/peterjc/galaxy_blast/blob/master/tools/ncbi_blast_plus/n… With the from_data_table feature this which would be much shorter: <param name="database" type="select" label="Nucleotide BLAST database"> <options from_data_table="blastdb" /> </param> For this to work, the column information must instead be defined centrally in ``tool_data_table_conf.xml`` (via a ``tool_data_table_conf.xml.sample`` file), e.g. <table name="blastdb" comment_char="#"> <columns>value, name, path</columns> <file path="tool-data/blastdb.loc" /> </table> For simple tools this seems quite neat, but within a single tool suite using XML macros seems equally effective for centrally defining the columns in the *.loc files (we do this currently). However, what worries me is the data table XML configuration file adds a new complexity for dependency management between different ToolShed repositories using a *.loc file (like the *.loc files for BLAST databases). For the BLAST database *.loc files, the simplest solution seems to be not to use the Data Tables feature (as we do now). The next best solution seems to be to put the sample *.loc files and associated data table definition XML files into a shared ToolShed repository (called called blast_data_tables, or blast_databases?) which would be declared as a dependency of anything using the BLAST database *.loc files (e.g. the BLAST+ wrappers and any data managers). [This would be like the existing blast_datatypes ToolShed repository which is a declared dependency of many tools using BLAST] Is that a good plan? What benefits does it have over simply not using the Data Table functionality? Thanks, Peter

2 2

automatically adding fasta files to "built-in" databases and selecting files questions
by Nikhil Joshi 09 Apr '14

09 Apr '14

Hi all, We have a local install of Galaxy that we use for testing for our bioinformatics courses. I am creating some new tool interfaces for the install. I want to create a tool that will allow a user to automatically add a fasta database to the "built-in" databases (without needing to reboot the server). Is this possible? Along with that issue is that the number of databases for this tool can number in the thousands. Is there a way to easily search the built-in databases for the database you are looking for, and then use that database in a tool? Right now, it is an enormous drop down list which is really annoying and unwieldy. Finally, these tools can take multiple input files, but I really don't like the way galaxy handles that where you have to continually click on "Add another file". Is there a method or work-around in galaxy where you can have a multi-file select? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

2 1

visualization button
by Philippe Moncuquet 09 Apr '14

09 Apr '14

Hi, I have developed a simple script that give some statistics on an assemble. It generates a tabular output. The visualization button is not available when I extend the output in the history pane. Is there any code that I need to put in my xml wrapper to get this button available ? Regards, Philip

2 5