July 2013 - galaxy-dev - lists.galaxyproject.org

tool input is folder rather than file
by Pankaj Narang 24 Jul '13

24 Jul '13

Dear Developers, I want to design a tool whose input is folder containing files and sub-folders. I was wandering which value should I assign to "TYPE" attribute of PARAM tag in INPUTS. -- Best regards, Pankaj Narang

2 1

Re: [galaxy-dev] Bowtie2 error: Creates a 62 kb file.
by Maddhi, Srinivas 24 Jul '13

24 Jul '13

[Copying the list]. -----Original Message----- From: <Maddhi>, Srinivas Maddhi <srinivas-maddhi(a)uiowa.edu> Date: Wednesday, July 24, 2013 10:01 AM To: "Perez, Ricardo" <ricky_pz2(a)neo.tamu.edu> Subject: Re: [galaxy-dev] Bowtie2 error: Creates a 62 kb file. >Hi Ricardo, > >Ensuring that the appropriate reference indices are present, properly >configured and selected when running bowtie2 might help address this >problem. > >-----Original Message----- >From: <Perez>, Ricardo <ricky_pz2(a)neo.tamu.edu> >Date: Wednesday, July 24, 2013 1:35 AM >To: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> >Subject: [galaxy-dev] Bowtie2 error: Creates a 62 kb file. > >>Dear all, >> >>Today we ran some problems when running bowtie2. When we run bowtie2, >>the run manages to complete. However, bowtie2 creates a 62kb file. >>The error log that we get is the following: >> >>galaxy.jobs DEBUG 2013-07-23 17:09:42,378 (1483) Persisting job >>destination (destination id: local:///) >>galaxy.jobs.handler INFO 2013-07-23 17:09:42,392 (1483) Job dispatched >>galaxy.tools DEBUG 2013-07-23 17:09:42,556 Building dependency shell >>command for dependency 'bowtie2' >>galaxy.tools WARNING 2013-07-23 17:09:42,556 Failed to resolve dependency >>on 'bowtie2', ignoring >>galaxy.tools DEBUG 2013-07-23 17:09:42,556 Building dependency shell >>command for dependency 'samtools' >>galaxy.tools WARNING 2013-07-23 17:09:42,556 Failed to resolve dependency >>on 'samtools', ignoring >>galaxy.jobs.runners.local DEBUG 2013-07-23 17:09:42,626 (1483) executing: >>bowtie2 --version &> >>/usr/local/galaxy/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_1483; >>python /usr/local/galaxy/galaxy-dist/tools/sr_mapping/bowtie2_wrapper.py >> --num-threads="4" >>--output=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1673.da >>t >> >>--indexes-path="/data/db/fungidb/Ncrassa_OR74A/bowtie2/Ncrassa_wt_Chr01.C >>h >>r07.+ChrM.+rDNA_v12" --single-paired=paired >>--input1=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1668.da >>t >> >>--input2=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1669.da >>t >> -I 0 -X 250 --settings=preSet >>galaxy.jobs DEBUG 2013-07-23 17:09:42,696 (1483) Persisting job >>destination (destination id: local:///) >>galaxy.jobs.runners.local DEBUG 2013-07-23 17:09:42,766 execution >>finished: bowtie2 --version &> >>/usr/local/galaxy/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_1483; >>python /usr/local/galaxy/galaxy-dist/tools/sr_mapping/bowtie2_wrapper.py >> --num-threads="4" >>--output=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1673.da >>t >> >>--indexes-path="/data/db/fungidb/Ncrassa_OR74A/bowtie2/Ncrassa_wt_Chr01.C >>h >>r07.+ChrM.+rDNA_v12" --single-paired=paired >>--input1=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1668.da >>t >> >>--input2=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1669.da >>t >> -I 0 -X 250 --settings=preSet >>galaxy.jobs.runners DEBUG 2013-07-23 17:09:42,850 executing external >>set_meta script for job 1483: >>/usr/local/galaxy/galaxy-dist/set_metadata.sh ./database/files >>/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483 . >>/usr/local/galaxy/galaxy-dist/universe_wsgi.ini >>/usr/local/galaxy/galaxy-dist/database/tmp/tmpUiqpPD >>/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/gal >>a >>xy.json >>/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/met >>a >>data_in_HistoryDatasetAssociation_1767_wqhEmr,/usr/local/galaxy/galaxy-di >>s >>t/database/job_working_directory/001/1483/metadata_kwds_HistoryDatasetAss >>o >>ciation_1767_qVZjpG,/usr/local/galaxy/galaxy-dist/database/job_working_di >>r >>ectory/001/1483/metadata_out_HistoryDatasetAssociation_1767_vIfpRO,/usr/l >>o >>cal/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_r >>e >>sults_HistoryDatasetAssociation_1767_UNl6OR,,/usr/local/galaxy/galaxy-dis >>t >>/database/job_working_directory/001/1483/metadata_over! >> ride_HistoryDatasetAssociation_1767_MZilrL >>165.91.11.135 - - [23/Jul/2013:17:09:43 -0500] "GET >>/api/histories/ca8efa3c47500414 HTTP/1.1" 200 - >>"http://galaxy.tamu.edu/history" "Mozilla/5.0 (X11; Linux x86_64) >>AppleWebKit/537.22 (KHTML, like Gecko) Ubuntu Chromium/25.0.1364.160 >>Chrome/25.0.1364.160 Safari/537.22" >>165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET /history HTTP/1.1" >>200 - "http://galaxy.tamu.edu/tool_runner/index" "Mozilla/5.0 (Macintosh; >>Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) >>Chrome/28.0.1500.71 Safari/537.36" >>165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET >>/api/histories/1ab12f1e464820dd HTTP/1.1" 200 - >>"http://galaxy.tamu.edu/history" "Mozilla/5.0 (Macintosh; Intel Mac OS X >>10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 >>Safari/537.36" >>165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET >>/history/get_display_application_links HTTP/1.1" 200 - >>"http://galaxy.tamu.edu/history" "Mozilla/5.0 (Macintosh; Intel Mac OS X >>10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 >>Safari/537.36" >>galaxy.jobs.runners DEBUG 2013-07-23 17:09:44,457 execution of external >>set_meta for job 1483 finished >>galaxy.datatypes.metadata DEBUG 2013-07-23 17:09:44,545 loading metadata >>from file for: HistoryDatasetAssociation 1767 >> >>We currently have the most recent version of bowtie2 and samtools. >>Anyone know how to fix this? >> >>Thank you for your time, >>--Ricardo Perez >> >>___________________________________________________________ >>Please keep all replies on the list by using "reply all" >>in your mail client. To manage your subscriptions to this >>and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >>To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >

2 1

uninstalling migrated tool
by Shantanu Pavgi (Campus) 24 Jul '13

24 Jul '13

It seems like if we uninstall a migrated tool then it's entry in the migrated_tools_conf.xml file doesn't get deleted. Does it need to be removed manually? -- Thanks, Shantanu

3 2

All processes are freezed
by Ali Sharifi Zarchi 24 Jul '13

24 Jul '13

Hi everybody Since yesterday all processes I have run on Galaxy main are waiting. The system displayed a message to contact a Galaxy admin, but I could find no other way rather than this mailing list. Is this freeze due to a public maintenance, or it's just my problem? Best wishes, Ali

2 1

job_conf.xml workers
by Ido Tamir 24 Jul '13

24 Jul '13

Hi, what is the relationship between workers defined in universe and job_conf.xml? e.g.: [server:handler0] use = egg:Paste#http port = 9092 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 <plugins workers="4">  <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner"/> <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner" workers="10"/> </plugins> a) do I have to specify the sum of the workers in job_conf.xml in universe? e.g. 4 local, 10 drmaa => 14 threadpool_workers per handler? Or is universe overridden by job_conf.xml? b) How many concurrent jobs can I then expect to have concurrently? 14 per handler? thank you very much, ido

2 2

BWA Illumina Mapping / BWA Reference Genome
by Moritz Juchler 24 Jul '13

24 Jul '13

Hello Ladies and Gentlemen, I am Moritz Juchler from University Heidelberg. For my Bachelor thesis I have to choose a bioinformatic pipeline management tool to find SNP's in genomes from hcc patients. My decision was made in favor of galaxy. I have a 64-bit openSuse 11.3 server. I have installed Galaxy locally, since we have a) very large files (>30GB per patient) and b) the data is protection sensitive. I kept close to http://wiki.galaxyproject.org/Admin/Get%20Galaxy Now I would like to run this bpipe pipeline: http://pastebin.com/sZd5vfdL And the first step is to align my genome to a *hg19 reference genome* which I have locally under /genedata/human_genome_GRCh37/. trr@portalmoritz:~> ls -l /genedata/human_genome_GRCh37/ > total 8486312 > -rw-r--r-- 1 trr root 3199905909 2013-06-25 16:44 hg19.fa > -rw-r--r-- 1 trr root 8591 2013-07-01 16:06 hg19.fa.amb > -rw-r--r-- 1 trr root 4040 2013-07-01 16:06 hg19.fa.ann > -rw-r--r-- 1 trr root 3137161344 2013-07-01 16:05 hg19.fa.bwt > -rw-r--r-- 1 trr root 784290318 2013-07-01 16:06 hg19.fa.pac > -rw-r--r-- 1 trr root 1568580688 2013-07-01 16:31 hg19.fa.sa *bwa is installed and gives me:* > trr@portalmoritz:~> bwa > Program: bwa (alignment via Burrows-Wheeler transformation) > Version: 0.7.5a-r405 > Contact: Heng Li <lh3(a)sanger.ac.uk> Then I tried to follow this guide: http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup to get the reference files and http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies. This is my *$PATH* > trr@portalmoritz:~> echo $PATH > > /home/trr/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/home/trr/bpipe-0.9.8/bin:/home/trr/bwa-0.7.5a:/home/trr/samtools-0.1.19 *In the universe_wsgi.ini I changed:* > tool_dependency_dir = /home/trr/galaxy-dist/tool_dependency_dir > debug = False > use_interactive = True > library_import_dir = /genedata/ > allow_library_path_paste = True > admin_users = ... This is my *tool_dependency_dir:* > trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa> ls -l > total 4 > drwxr-xr-x 3 trr users 4096 2013-07-16 14:28 0.7.4 > lrwxrwxrwx 1 trr users 6 2013-07-16 14:17 default -> 0.7.4/ This is the* version folder of bwa:* > trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4> ls -l > total 8 > drwxr-xr-x 2 trr users 4096 2013-07-16 14:18 bin > -rw-r--r-- 1 trr users 47 2013-07-16 14:22 env.sh This is the *content of env.sh:* > trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4> cat env.sh > PATH="/home/trr/bwa-0.7.5a/:$PATH" > export PATH And this is the *content of the bin folder:* > trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4/bin> ls -l > total 3896 > -rw-r--r-- 1 trr users 6098 2013-07-16 14:18 bamlite.c > -rw-r--r-- 1 trr users 3124 2013-07-16 14:18 bamlite.h > -rw-r--r-- 1 trr users 24816 2013-07-16 14:18 bamlite.o > -rw-r--r-- 1 trr users 11508 2013-07-16 14:18 bntseq.c > -rw-r--r-- 1 trr users 2557 2013-07-16 14:18 bntseq.h > -rw-r--r-- 1 trr users 37440 2013-07-16 14:18 bntseq.o > -rwxr-xr-x 1 trr users 998217 2013-07-16 14:18 bwa > -rw-r--r-- 1 trr users 24225 2013-07-16 14:18 bwa.1 > -rw-r--r-- 1 trr users 9416 2013-07-16 14:18 bwa.c > -rw-r--r-- 1 trr users 1381 2013-07-16 14:18 bwa.h .... I got the xmls and .py from https://bitbucket.org/galaxy/galaxy-dist/src/da9d740fce31/tools/sr_mapping and i didnt change them at all and put them into ~/galaxy-dist/*tools/sr_mapping * (since they were missing in this folder) > bwa_color_wrapper.xml > bwa_wrapper.py > bwa_wrapper.xml I added *bwa_index_color.loc and bwa_index.loc* to ~galaxy-dist/*tool-data* (they were missing as well, there* were no* bwa_index_color.loc.sample or bwa_index.lox.sample files!!!) I only have this single line in both bwa_index_color.loc and bwa_index.loc > trr@portalmoritz:~/galaxy-dist/tool-data> cat bwa_index_color.loc > #This is a sample file distributed with Galaxy that enables tools > # > #<unique_build_id> <dbkey> <display_name> <file_path> > hg19 hg19 hg19 /genedata/human_genome_GRCh37/hg19.fa (Spaces are actually tabs!) After all that, I neither have the "Map with BWA for Illumina<https://main.g2.bx.psu.edu/tool_runner?tool_id=toolshed.g2.bx.psu.edu/repos…>" in my local Galaxy version, nor do I find the reference genome. If i missed on any required, please tell me, I will answer you as soon as possible. Sincerly Yours Moritz Juchler

3 7

Cloudman - is it possible to still use depreciated AMIs ?
by Ravpreet Setia 24 Jul '13

24 Jul '13

I tried launching an instance specifying "ami-da58aab3" as the AMI but I am unable to SSH or access Cloudman using the instance's URL.

2 3

Bug report
by Jianxin Wang 24 Jul '13

24 Jul '13

Hi Galaxy dev team, I'm trying to the you galaxy server to do NGS peak calling: Tools -> NGS: Peak Calling -> MACS<https://main.g2.bx.psu.edu/tool_runner?tool_id=peakcalling_macs> Model-based Analysis of ChIP-Seq. However, the page won't allow me enter the input files: the drop-down selection box won't show anything when clicked. Same problem for the ChIP-Seq Control File selection box. Please take a look and email me back if problem is fixed. Jianxin Wang, PhD Computational Biologist jianxin.wang(a)oicr.on.ca Ontario Institute for Cancer Research MaRS Centre, South Tower 101 College Street, Suite 800 Toronto, Ontario, Canada M5G 0A3 Tel: 416-673-8569 www.oicr.on.ca<http://www.oicr.on.ca> This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.

2 1

Test ToolShed repository_dependencies.xml processing
by Peter Cock 24 Jul '13

24 Jul '13

Hi Gred, I'm trying to update this repository (initially just to switch my README file to reStructuredText as the current plain text is not being rendered nicely): http://testtoolshed.g2.bx.psu.edu/view/peterjc/get_orfs_or_cdss Uploading my new tar-ball failed: Internal Server Error Galaxy was unable to sucessfully complete your request URL: http://testtoolshed.g2.bx.psu.edu/upload/upload?repository_id=8067bb858e913… Module galaxy.web.framework.middleware.error:149 in __call__ >> app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ >> environ, self.app) Module paste.wsgilib:543 in intercept_output >> app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request >> body = method( trans, **kwargs ) Module galaxy.web.framework:98 in decorator >> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.tool_shed.controllers.upload:120 in upload >> self.upload_tar( trans, repository, tar, uploaded_file, upload_point, remove_repo_files_not_in_tar, commit_message, new_repo_alert ) Module galaxy.webapps.tool_shed.controllers.upload:341 in upload_tar >> altered, root_elem = commit_util.handle_repository_dependencies_definition( trans, uploaded_file_name ) Module tool_shed.util.commit_util:243 in handle_repository_dependencies_definition >> raise Exception( exception_message ) Exception: The repository_dependencies.xml file contains an invalid <repository> tag. The tag is missing the required name attribute. The tag is missing the required owner attribute. The XML does have a name tag, it deliberately omits the ToolShed and revision as I want these fields to be auto-filled: $ more repository_dependencies.xml <?xml version="1.0"?>  <repositories description="This requires Biopython as a dependency.">  <repository owner="biopython" name="package_biopython_1_61" /> </repositories> Peter

2 2

Disable cheetah filtering
by Joachim Jacob | VIB | 24 Jul '13

24 Jul '13

Hi all, A Cheetah question: when I enter ";" as a value in a text box of a tool (stored in $exp), Cheetah prints this as "X", and passed this string to the tool. How to disable this behaviour in Cheetah? I have tried to play with #filter directives, but to no avail. Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team

3 8