April 2013 - galaxy-dev - lists.galaxyproject.org

Using conditional parameters in unit tests
by Peter Cock 30 Apr '13

30 Apr '13

Hi all, I've not previously need to use a conditional parameter in a unit test, but for the MIRA wrapper this is essential. Here's my first attempt and the error when running the test: https://bitbucket.org/peterjc/galaxy-central/commits/794ebc6cafecd3db9388f5… <tests> <test> <param name="job_method" value="denovo" /> <param name="job_type" value="est" /> <param name="job_qual" value="accurate" /> <param name="condBackbone.use" value="false" /> <param name="condSanger.use" value="true" /> <param name="condSanger.filename" value="tvc_mini.fastq" ftype="fastq" /> <param name="condRoche.use" value="false" /> <param name="condIllumina.use" value="false" /> <param name="condIonTorrent.use" value="false" /> <output name="out_fasta" file="tvc_contigs.fasta" ftype="fasta" /> </test> </tests> Note that I can't refer to the parameters by the final part of their name ("use") as this is ambiguous. The dot syntax is natural to match that used in the Cheetah templates, but does not appear to work in the tool tests: $ ./run_functional_tests.sh -id mira_assembler ... ====================================================================== ERROR: Assemble with MIRA v3.4 ( mira_assembler ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/mnt/galaxy/galaxy-central/test/functional/test_toolbox.py", line 76, in do_it page_inputs = self.__expand_grouping(testdef.tool.inputs_by_page[0], all_inputs) File "/mnt/galaxy/galaxy-central/test/functional/test_toolbox.py", line 118, in __expand_grouping if declared_inputs[ value.test_param.name ] == case.value: KeyError: 'use' Is there something wrong with my test, or is this not yet supported? Thanks, Peter

3 7

Setting torque queues
by Scott Hazelhurst 30 Apr '13

30 Apr '13

Dear dev I would be grateful for some assistance in setting up our galaxy server to submit jobs to our torque server.. We are using release_2013.04.01 I have tried going through the install information but clearly I’ve misunderstood something, and unfortunately most examples come from earlier versions galaxy.jobs DEBUG 2013-04-30 15:28:13,481 (33) Working directory for job is: /home/galaxy/galaxy-dist/database/job_working_directory/000/33 galaxy.jobs.handler DEBUG 2013-04-30 15:28:13,527 (33) Dispatching to pbs runner 146.141.75.218, 146.141.1.81 - - [30/Apr/2013:15:28:12 +0200] "POST /galaxy/tool_runner/index HTTP/1.1" 200 - "http://galaxy.bioinf.wits.ac.za/galaxy/tool_runner/rerun?id=9752b387803d3e1e" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:20.0) Gecko/20100101 Firefox/20.0" galaxy.jobs DEBUG 2013-04-30 15:28:14,213 (33) Persisting job destination (destination id: WitsLong) Exception AssertionError: AssertionError('State <sqlalchemy.orm.state.InstanceState object at 0x7f79682e5150> is not present in this identity map',) in <bound method InstanceState._cleanup of <sqlalchemy.orm.state.InstanceState object at 0x7f79682e5150>> ignored galaxy.jobs.handler INFO 2013-04-30 15:28:14,522 (33) Job dispatched galaxy.tools DEBUG 2013-04-30 15:28:15,189 Building dependency shell command for dependency 'clustalw2' galaxy.tools WARNING 2013-04-30 15:28:15,192 Failed to resolve dependency on 'clustalw2', ignoring galaxy.jobs.runners.pbs DEBUG 2013-04-30 15:28:15,957 (33) submitting file /home/galaxy/galaxy-dist/database/pbs/33.sh galaxy.jobs.runners.pbs DEBUG 2013-04-30 15:28:15,958 (33) command is: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i "/home/galaxy/galaxy-dist/database/files/000/dataset_2.dat" -o "/home/galaxy/galaxy-dist/database/files/000/dataset_60.dat" -s "ALIGNED" -l "/home/galaxy/galaxy-dist/database/files/000/dataset_61.dat" -t "Clustal_run" -d "DNA" -f "CLUSTAL" galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:15,963 (33) pbs_submit failed (try 1/5), PBS error 15039: Route rejected by all destinations galaxy.jobs DEBUG 2013-04-30 15:28:16,400 (34) Working directory for job is: /home/galaxy/galaxy-dist/database/job_working_directory/000/34 galaxy.jobs.handler DEBUG 2013-04-30 15:28:16,416 (34) Dispatching to pbs runner 146.141.75.218, 146.141.1.81 - - [30/Apr/2013:15:28:13 +0200] "POST /galaxy/tool_runner/index HTTP/1.1" 200 - "http://galaxy.bioinf.wits.ac.za/galaxy/tool_runner/rerun?id=9752b387803d3e1e" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:20.0) Gecko/20100101 Firefox/20.0" galaxy.jobs DEBUG 2013-04-30 15:28:16,668 (34) Persisting job destination (destination id: WitsLong) galaxy.jobs.handler INFO 2013-04-30 15:28:16,689 (34) Job dispatched galaxy.tools DEBUG 2013-04-30 15:28:16,937 Building dependency shell command for dependency 'clustalw2' galaxy.tools WARNING 2013-04-30 15:28:16,938 Failed to resolve dependency on 'clustalw2', ignoring galaxy.jobs.runners.pbs DEBUG 2013-04-30 15:28:17,160 (34) submitting file /home/galaxy/galaxy-dist/database/pbs/34.sh galaxy.jobs.runners.pbs DEBUG 2013-04-30 15:28:17,161 (34) command is: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i "/home/galaxy/galaxy-dist/database/files/000/dataset_2.dat" -o "/home/galaxy/galaxy-dist/database/files/000/dataset_62.dat" -s "ALIGNED" -l "/home/galaxy/galaxy-dist/database/files/000/dataset_63.dat" -t "Clustal_run" -d "DNA" -f "CLUSTAL" galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:17,162 (34) pbs_submit failed (try 1/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:17,971 (33) pbs_submit failed (try 2/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:19,166 (34) pbs_submit failed (try 2/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:19,975 (33) pbs_submit failed (try 3/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:21,170 (34) pbs_submit failed (try 3/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:21,978 (33) pbs_submit failed (try 4/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:23,173 (34) pbs_submit failed (try 4/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:23,982 (33) pbs_submit failed (try 5/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:25,176 (34) pbs_submit failed (try 5/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs ERROR 2013-04-30 15:28:25,984 (33) All attempts to submit job failed galaxy.jobs.runners.pbs ERROR 2013-04-30 15:28:27,179 (34) All attempts to submit job failed <table width="100%" border="0" cellspacing="0" cellpadding="0" style="width:100%;"> <tr> <td align="left" style="text-align:justify;"><font face="arial,sans-serif" size="1" color="#999999"><span style="font-size:11px;">This communication is intended for the addressee only. It is confidential. If you have received this communication in error, please notify us immediately and destroy the original message. You may not copy or disseminate this communication without the permission of the University. Only authorised signatories are competent to enter into agreements on behalf of the University and recipients are thus advised that the content of this message may not be legally binding on the University and may contain the personal views and opinions of the author, which are not necessarily the views and opinions of The University of the Witwatersrand, Johannesburg. All agreements between the University and outsiders are subject to South African Law unless the University agrees in writing to the contrary. </span></font></td> </tr> </table

1 0

Visually differentiate the main and test Tool Shed
by Peter Cock 30 Apr '13

30 Apr '13

Hello all, I can't be the only Galaxy Tool author to have accidentally made a change on the main Tool Shed my mistake, wrongly assuming I was on the test Tool Shed (sandbox). In this case the tool update was harmless, but it could easily have resulted in users of my tool downloading a broken file. To minimise the chances of this happening again, please: (1) Change the page title of the http://testtoolshed.g2.bx.psu.edu/ which is currently also "Galaxy Tool Shed". I'm talking about the title attribute shown in the Web Browser tab or chrome. (2) Change the colour scheme of http://testtoolshed.g2.bx.psu.edu/ which is identical to the main tool shed. How about red? Related to this, you could replace Galaxy icon with a "beta" or something? http://testtoolshed.g2.bx.psu.edu/static/images/galaxyIcon_noText.png Thanks, Peter

2 2

BLAST Database error: No alias or index file found
by Mike Dyall-Smith 30 Apr '13

30 Apr '13

This follows on from Kenneth Auerbach's question Sep 12, 2012: Galaxy error running NCBI BLAST: blastx http://dev.list.galaxyproject.org/Galaxy-error-running-NCBI-BLAST-blastx-td… I wish to use a local copy of the genbank database for searches. The genbank db files are sitting in a folder on an external drive, and I have altered the blastdb_p.loc to point to them. I can see and select this database within galaxy, but when I try using blast+ blastp tool, I get the error ""BLAST Database error: No alias or index file found for protein database [/media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr] in search path [/var/lib/galaxy-server/database/job_working_directory] As suggested previously by Peter Cock, I can list all the files with: ls /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr* which gives me the list: /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.phd /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.phi /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.phr /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.pin /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.pnd /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.pni /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.pog /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.ppd /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.ppi /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.psd /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.psi /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.00.psq ... /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.phd /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.phi /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.phr /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.pin /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.pnd /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.pni /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.pog /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.ppd /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.ppi /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.psd /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.psi /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.10.psq /media/sf_mikeds_bioinf/GenbankDB_Files/nr/nr.pal The alias file, nr.pal is the last one in the list above, so is present. I changed the default directory for the blast database: BLASTDB=/media/sf_mikeds_bioinf/GenbankDB_Files/nr - this made no difference to the error. I am running biolinux 7 (latest) in ubuntu within VirtualBox, on a macbook pro. The external drive is a shared folder, and I can see all the files within galaxy (e.g. I can import data data files). Any help would be appreciated. Regards, Mike DS

2 1

Re: [galaxy-dev] gff installation failed with easy_install
by Brad Chapman 29 Apr '13

29 Apr '13

Mic; > I have tried to install gff with easy_install, but I got the following > error: > $ easy_install --prefix=/home/mic/apps/pymodules -UZ > https://github.com/chapmanb/bcbb/tree/master/gff > Downloading https://github.com/chapmanb/bcbb/tree/master/gff > error: Unexpected HTML page found at > https://github.com/chapmanb/bcbb/tree/master/gff > > How is it possible to install gff? I don't know of a way to install directly from git with subdirectories like that. You'd need to clone, then install with easy_install or pip: $ git clone git://github.com/chapmanb/bcbb.git $ easy_install bcbb/gff $ pip install bcbb/gff Apologies about the convoluted setup. Depending on what you're doing, you might want to have a look at gffutils: https://github.com/daler/gffutils We're working on rolling the functionality from the gff library into this so there'll be one place to work from for GFF in python. Hope this helps, Brad

2 1

Command Line Tool for Creating New Users
by Neil.Burdett＠csiro.au 29 Apr '13

29 Apr '13

Hi, I'm trying to use the tool, however, when I run it I get: python create_galaxy_users.py Traceback (most recent call last): File "create_galaxy_users.py", line 2, in <module> from scripts.db_shell import * ImportError: No module named scripts.db_shell I've set my PYTHONPATH to be ~/galaxy-dist/lib. What else do I need to do? I'm running the script from ~/galaxy-dist/ directory, but I get the same error if I copy it into the scripts directory. I've checked and I can see that the scripts.db_shell.py script exists If I comment out the line then the next error I get is: python create_galaxy_users.py Traceback (most recent call last): File "create_galaxy_users.py", line 7, in <module> from galaxy.security import GalaxyRBACAgent File "/home/galaxy/galaxy-dist/lib/galaxy/security/__init__.py", line 9, in <module> from galaxy.model.orm import * File "/home/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 15, in <module> from galaxy.security import RBACAgent, get_permitted_actions ImportError: cannot import name RBACAgent Any help much appreciated Neil ---------------------------------------------------------------------- Message: 1 Date: Sat, 27 Apr 2013 14:07:49 -0500 From: John Chilton <chilton(a)msi.umn.edu> To: Adam Brenner <aebrenne(a)uci.edu> Cc: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> Subject: Re: [galaxy-dev] Command Line Tool for Creating New Users Message-ID: <CANwbokctvXO-DFmamr77FU6BbDHGR1STG7mpH_LP+fE2nMcnag(a)mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 I put together a script to do this. It is described in the galaxy-dev thread titled: "[galaxy-dev] user creation using API". It doesn't actually use the API, it uses the db_shell.py script. Hope this helps. -John On Thu, Apr 25, 2013 at 7:11 PM, Adam Brenner <aebrenne(a)uci.edu> wrote: > Howdy All, > > I was wondering if anyone has created a command line tool that will > add users to the Galaxy user database? > > I did look into switching to external user authentication via nginx, > however, this will result in us using the campus wide kerberos system > and allowing ~30,000 people access to Galaxy... and thus offering us > less fine grain control to Galaxy. > > Anything anyone has to share would be much appreciated, -Adam > > -- > Adam Brenner > Computer Science, Undergraduate Student Donald Bren School of > Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebrenne(a)uci.edu > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this and other > Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ------------------------------ _______________________________________________ galaxy-dev mailing list galaxy-dev(a)lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ End of galaxy-dev Digest, Vol 82, Issue 28 ******************************************

2 4

Galaxy Workflow Input Problem
by Amanda Dupuy 29 Apr '13

29 Apr '13

Dear Galaxy Team, I'm using the psu public galaxy software and cannot get my workflows to use the input files I'm selecting. Everytime I try to run the workflow, I enter all of the input files that I want to use, but after I hit run, the operations are performed only using my most recent Galaxy file as an input, instead of the ones that I selected. I realize that sometimes you have to wait for the page to refresh before executing if a new file is applied, but it doesn't seem that the page is trying to refresh. Do you have any tips for me? Thanks for your help, Amanda~ -- Amanda Dupuy PhD Student in Genetics and Genomics Department of Molecular and Cell Biology Center for Applied Genetics and Technology 354 Mansfield Road U-2131 Room 346 Storrs, CT 06269 Lab: 860-486-1896

2 1

Galaxy hg repository problem "abort: path ... traverses symbolic link"
by Peter Cock 29 Apr '13

29 Apr '13

Hi all, I've got an hg puzzle, can anyone on the Galaxy team explain the cause of this and what action I can take to avoid it? Starting with a clean checkout, $ hg clone https://bitbucket.org/galaxy/galaxy-central warning: bitbucket.org certificate with fingerprint 24:9c:45:8b:9c:aa:ba:55:4e:01:6d:58:ff:e4:28:7d:2a:14:ae:3b not verified (check hostfingerprints or web.cacerts config setting) destination directory: galaxy-central requesting all changes adding changesets adding manifests adding file changes added 8886 changesets with 33822 changes to 6914 files (+1 heads) updating to branch default 4019 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cd galaxy-central $ hg branch default $ hg branches default 8885:9852dd712f5c stable 8880:8da37d3985ab $ hg checkout stable abort: path 'static/june_2007_style/Makefile' traverses symbolic link 'static/june_2007_style' I was actually trying to checkout one of my own branches, having setup the .hg/hgrc to include a link to my own fork on bitbucket. Thanks! Peter

4 10

Blast Errors
by Ganote, Carrie L 29 Apr '13

29 Apr '13

Hi list, I have a newly installed Galaxy instance (security_2013.04.08) and I am testing out NCBI Blast+ on it. I'm having trouble getting it to run. I'm using the following tools from the Galaxy Main Toolshed: blast_datatypes b3a3ba0c1d47 ncbi_blast_plus 1f546099212f Looks like blast is being called fine, but I get this: Fatal error: Matched on Error: Error: NCBI C++ Exception: "/N/soft/rhel6/ncbi-blast+/ncbi-blast-2.2.28+-src/c++/src/corelib/ncbiobj.cpp", line 925: Critical: ncbi::CObject::ThrowNullPointerException() - Attempt to access NULL pointer. WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'asn1': 'module' object has no attribute 'GenericAsn1' WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'asn1-binary': 'module' object has no attribute 'GenericAsn1Binary' Anyone seen this? Thanks! Carrie Ganote

2 5

Galaxy, CloudMan & nginx 1.4.0
by John Chilton 28 Apr '13

28 Apr '13

I wanted to update CloudMan to use nginx 1.4.0 for web socket proxy support, but I hit a problem. nginx-upload-module does not work with nginx 1.4.0 and the author has no intention of continuing work on the module. https://github.com/vkholodkov/nginx-upload-module/issues/41 There is a patch that based on the comments in the above thread doesn't work: http://paste.davromaniak.eu/index.php?show=110 And there are some related projects but I don't understand enough about how Galaxy is interacting with nginx upload to know if they are a promising replacement. https://github.com/pgaertig/nginx-big-upload https://github.com/agentzh/lua-resty-upload It also kind of seems like nginx might provide some support for handling uploads directly without that module, here is the example from that thread: location /upload { limit_except POST { deny all; } client_body_temp_path /tmp/; client_body_in_file_only on; client_body_buffer_size 128K; client_max_body_size 50M; proxy_pass_request_headers on; proxy_set_body $request_body_file; proxy_pass http://upstream; # will receive file_name only proxy_redirect off; } Can the galaxy solution be adapted for this client_body_xxx style? If not, has anyone looked at something like nginx-big-upload and gotten it to work? I suppose this is not likely since nginx 1.4 just came out, but it is worth asking. If someone does figure it out someday, let me know and I can test and update the CloudBioLinux/CloudMan stuff for whatever you figure out. Thanks, -John

1 0