- galaxy-dev - lists.galaxyproject.org

select both chromosome size and mappable genome size
by Ido Tamir 17 Apr '14

17 Apr '14

Hi, my tool needs both a file with chromosome lengths and the total mappable genome size. I would like to hardcode the mappable genome size per build somewhere. However the user should have to use only one drop down to select both. How can I achieve this? e.g. currently: <param name="FAI" type="select" label="Select chromsome sizes" > <options from_file="chrsize.loc"> <column name="name" index="0"/> <column name="value" index="1"/> </options> </param> with chrsize.loc: mm9 /groups/csf-ngs/wfsys/galaxy/tool-data/shared/ucsc/chrom/mm9.len mm10 /groups/csf-ngs/wfsys/galaxy/tool-data/shared/ucsc/chrom/mm10.len hg19 /groups/csf-ngs/wfsys/galaxy/tool-data/shared/ucsc/chrom/hg19.len dm3 /groups/csf-ngs/wfsys/galaxy/tool-data/shared/ucsc/chrom/dm3.len Currently I multiply the genome size from the fai file by 0.8 to get the mappable size within my tool, but I would like to use more exact numbers per build. thank you very much, ido

1 0

select both chromosome size and mappable genome size
by Service,solexa 17 Apr '14

17 Apr '14

Hi, my tool needs both a file with chromosome lengths and the total mappable genome size. I would like to hardcode the mappable genome size per build somewhere. However the user should have to use only one drop down to select both. How can I achieve this? e.g. currently: <param name="FAI" type="select" label="Select chromsome sizes" > <options from_file="chrsize.loc"> <column name="name" index="0"/> <column name="value" index="1"/> </options> </param> with chrsize.loc: mm9 /groups/csf-ngs/wfsys/galaxy/tool-data/shared/ucsc/chrom/mm9.len mm10 /groups/csf-ngs/wfsys/galaxy/tool-data/shared/ucsc/chrom/mm10.len hg19 /groups/csf-ngs/wfsys/galaxy/tool-data/shared/ucsc/chrom/hg19.len dm3 /groups/csf-ngs/wfsys/galaxy/tool-data/shared/ucsc/chrom/dm3.len Currently I multiply the genome size from the fai file by 0.8 to get the mappable size within my tool, but I would like to use more exact numbers per build. thank you very much, ido

1 0

Trackster shared Visualization in an iframe
by "<Raphael Müller>" 17 Apr '14

17 Apr '14

Hello, I am developing a web application, where I want to show a Trackster visualization in an iframe from our local Galaxy Server. The problem is that I am not able to specify over a locus via the URL when I am not the owner of the visualization. (When we click "saved Visualization" we can specify the region via the link to the visualization, we can call e.g. ".../visualization/trackster?id=1cd8e2f6b131e891/#chr19:1234-5678/".) Is there any possibility to show a shared visualization in an iframe at a specific locus? Greetings Raphael Müller

2 1

Adding a new menu option in the header/masthead
by Neil.Burdett＠csiro.au 17 Apr '14

17 Apr '14

On the banner/masthead there is a menu with things like Home, Shared Data, Admin, help etc ... How do you add a new option here. For example if I wanted something called "About" I could change ~/templates/webapps/galaxy/base_panels.mako and copy and paste something similar to the "Admin" tab i.e. ## About tab. ${tab( "about", "About", h.url_for( controller='/about', action='index'))} Then in ~/templates/webapps/galaxy create a new directory called "about" and copy in the admin/*.mako files (just to see if I am on the right track). However this doesn't work. I get "Not Found page. The resource could not be found. This link may not be followed from within Galaxy" Does anyone know how to add a new option in the header/masthead Thanks Neil

1 0

Re: [galaxy-dev] Enabling an HTML report to prime a tool form (Dooley, Damion)
by Dooley, Damion 16 Apr '14

16 Apr '14

For those who need it in the future, found the solution, so now the html report/form can trigger the given tool to run. Turns out the tool_state value wasn't the problem. It was the <input type="hidden" name="input" value="%(dataset_selection_id)s" /> value that was wrong. Solution: the xml tool template that triggers rendering of the html form needs to pass the right dataset id to the form (which then passes it to the target tool) so one needs to have $selection_file.id passed: <command interpreter="python"> blast_reporting.py $blastxml_file $tabular_file $html_file $out_format $selection_file $selection_file.id $html_template ... </command>

1 0

Re: [galaxy-dev] snpEff and FASTQ Summary Statistics errors
by Wang, Xiaofei 16 Apr '14

16 Apr '14

Dear Bjoern, As your suggestion, I tried to install the snpEff annotation. But, I think I am still be stuck there. What I did is: Admin > Data > Manage local data (beta), then the message is shown as below. (For the BWA and Fetching genome, I installed them from tool_sheds by myself. For the SnpEff staff, it has been there when I checked the data manager.) Next, I clicked on SnpEff Databases > Execute. But, when I checked the snpeff annotations in View managed data by Tool Data Table part, it is still empty. Could you give me more details about how to install the annotation with the snpeff data manager under the Admin panel? I really appreciate for your help, and thank you so much for your patience! Best, Xiaofei Data Manager Choose your data managing option from below. * Access data managers - get data, build indexes, etc * SnpEff Databases<http://127.0.0.1:8080/tool_runner?tool_id=toolshed.g2.bx.psu.edu%2Frepos%2F…> - Read the list of available snpEff databases * SnpEff Download<http://127.0.0.1:8080/tool_runner?tool_id=toolshed.g2.bx.psu.edu%2Frepos%2F…> - Download a new database * BWA index<http://127.0.0.1:8080/tool_runner?tool_id=toolshed.g2.bx.psu.edu%2Frepos%2F…> - builder * BWA Color index<http://127.0.0.1:8080/tool_runner?tool_id=toolshed.g2.bx.psu.edu%2Frepos%2F…> - builder * Reference Genome<http://127.0.0.1:8080/tool_runner?tool_id=toolshed.g2.bx.psu.edu%2Frepos%2F…> - fetching * View managed data by manager * SnpEff Databases<http://127.0.0.1:8080/data_manager/manage_data_manager?id=toolshed.g2.bx.ps…> - Read the list of available snpEff databases * SnpEff Download<http://127.0.0.1:8080/data_manager/manage_data_manager?id=toolshed.g2.bx.ps…> - Download a new database * BWA index<http://127.0.0.1:8080/data_manager/manage_data_manager?id=toolshed.g2.bx.ps…> - builder * BWA Color index<http://127.0.0.1:8080/data_manager/manage_data_manager?id=toolshed.g2.bx.ps…> - builder * Reference Genome<http://127.0.0.1:8080/data_manager/manage_data_manager?id=toolshed.g2.bx.ps…> - fetching * View managed data by Tool Data Table * snpeff_databases<http://127.0.0.1:8080/data_manager/manage_data_table?table_name=snpeff_data…> * snpeff_genomedb<http://127.0.0.1:8080/data_manager/manage_data_table?table_name=snpeff_geno…> * snpeff_regulationdb<http://127.0.0.1:8080/data_manager/manage_data_table?table_name=snpeff_regu…> * snpeff_annotations<http://127.0.0.1:8080/data_manager/manage_data_table?table_name=snpeff_anno…> * bwa_indexes<http://127.0.0.1:8080/data_manager/manage_data_table?table_name=bwa_indexes> * bwa_indexes_color<http://127.0.0.1:8080/data_manager/manage_data_table?table_name=bwa_indexes…> * all_fasta<http://127.0.0.1:8080/data_manager/manage_data_table?table_name=all_fasta> * * * * ________________________________________ * From: Björn Grüning [bjoern.gruening(a)gmail.com] * Sent: Wednesday, April 02, 2014 12:55 PM * To: Wang, Xiaofei; Björn Grüning; galaxy-dev(a)lists.bx.psu.edu * Subject: Re: [galaxy-dev] snpEff and FASTQ Summary Statistics errors * * Hi, * * that means that the snpeff annotation is missing. You can install it * with the snpeff datamanager under your Admin panel. Have a look at the * wiki page about data-manager and the snpeff wrapper documentation. * You do not need to edit xml files. If so that is an bug in the wrapper * and we should fix it :) * * Cheers, * Bjoern * * Am 02.04.2014 19:34, schrieb Wang, Xiaofei: * >>> When I used SnpEff in local galaxy instance, I got an error: * >>> Exception in thread "main" java.lang.UnsupportedClassVersionError: ca/mcgill/mcb/pcingola/snpSift/SnpSift : Unsupported major.minor version 51.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.j * > * > * > * > * >> Which Java version do you use. Please make sure you are running Java 1.7. * > * > * > * > * > Yes, it is figured, when I changed the java version to 1.7. Thank you so much! * > * > * > * > * > But, I got another error for snpEff. * > * > * > * > * > java.lang.RuntimeException: Property: 'dm5.30.genome' not found * > at ca.mcgill.mcb.pcingola.interval.Genome.<init>(Genome.java:92) * > at ca.mcgill.mcb.pcingola.snpEffect.Config.readGenomeConfig(Config.java:481) * > at ca.mcgill.mcb.pcingola.snpEffect.Config.readConfig(Config.java:444) * > at ca.mcgill.mcb.pcingola.snpEffect.Config.init(Config.java:345) * > at ca.mcgill.mcb.pcingola.snpEffect.Config.<init>(Config.java:96) * > at ca.mcgill.mcb.pcingola.snpEffect.commandLine.SnpEff.readConfig(SnpEff.java:245) * > at ca.mcgill.mcb.pcingola.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:981) * > at ca.mcgill.mcb.pcingola.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:963) * > at ca.mcgill.mcb.pcingola.snpEffect.commandLine.SnpEff.run(SnpEff.java:352) * > at ca.mcgill.mcb.pcingola.snpEffect.commandLine.SnpEff.run(SnpEff.java:320) * > at ca.mcgill.mcb.pcingola.snpEffect.commandLine.SnpEff.main(SnpEff.java:83) * > * > * > * > * > * > * > * > * > What does this mean? I found the similar error on this thead http://dev.list.galaxyproject.org/snpEff-on-Galaxy-td4659332.html, but there is no reply. * > * > * > * > * > Does this mean I need to add genome to Galaxy? Following this webpage: http://snpeff.sourceforge.net/SnpEff_manual.html? If yes, could you give me an example how to edit the snpEffect.xml file in dm (Drosophila) case as below? (In fact, on the drap-down menu of Genome in snpEff, there is different options for genome. Why the error shows '*.genome' not found? ) * > * > * > * > * > <param name="genomeVersion" type="select" label="Genome"> * > <option value="hg37">Human (hg37)<option> * > <option value="mm37.61">Mouse (mm37.61)<option> * > <param> * > * > * > * > ________________________________________ * > From: Björn Grüning [bjoern.gruening(a)gmail.com] * > Sent: Wednesday, April 02, 2014 10:35 AM * > To: Wang, Xiaofei; galaxy-dev(a)lists.bx.psu.edu * > Subject: Re: [galaxy-dev] snpEff and FASTQ Summary Statistics errors * > * > Hi Xiaofei, * > * >> Dear there, * >> * >> When I used SnpEff in local galaxy instance, I got an error: * >> Exception in thread "main" java.lang.UnsupportedClassVersionError: ca/mcgill/mcb/pcingola/snpSift/SnpSift : Unsupported major.minor version 51.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.j * > * > Which Java version do you use. Please make sure you are running Java 1.7. * > * >> Also, when I used the FASTQ Summary Statistics, I got an error like this: * >> Traceback (most recent call last): File "/Users/...../softwares/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastq_stats/9b7b4e0ca9db/fastq_stats/fastq_stats.py", line 48, in if __name__ == "__main__": main() File "/Users/... * > * > Can you give us the complete error message. The important part is missing. * > * > Ciao, * > Bjoern * > * >> I tried to google and solve them by myself. But I really do not know how to do that. Could anybody give me some clue? Thanks a lot for your help! * >> * >> * >> * >> * >> ___________________________________________________________ * >> Please keep all replies on the list by using "reply all" * >> in your mail client. To manage your subscriptions to this * >> and other Galaxy lists, please use the interface at: * >> http://lists.bx.psu.edu/ * >> * >> To search Galaxy mailing lists use the unified search at: * >> http://galaxyproject.org/search/mailinglists/ * >> * > * > * *

2 8

read_bits64 error when loading large genomes/data into trackster
by Raj Ayyampalayam 16 Apr '14

16 Apr '14

Hello, I am trying to visualize a large genome (Large number of scaffolds) and a large (bam file) in trackster on our local galaxy instance (running release_2014.02.10). When ever I try to do the above I see the following error in the logs: File "bbi_file.pyx", line 215, in bx.bbi.bbi_file.BBIFile.query (lib/bx/bbi/bbi_file.c:5596) File "bbi_file.pyx", line 222, in bx.bbi.bbi_file.BBIFile.query (lib/bx/bbi/bbi_file.c:5210) File "bbi_file.pyx", line 183, in bx.bbi.bbi_file.BBIFile.summarize (lib/bx/bbi/bbi_file.c:4475) File "bbi_file.pyx", line 248, in bx.bbi.bbi_file.BBIFile._get_chrom_id_and_size (lib/bx/bbi/bbi_file.c:5656) File "bpt_file.pyx", line 76, in bx.bbi.bpt_file.BPTFile.find (lib/bx/bbi/bpt_file.c:1388) File "bpt_file.pyx", line 55, in bx.bbi.bpt_file.BPTFile.r_find (lib/bx/bbi/bpt_file.c:1154) AttributeError: 'BinaryFileReader' object has no attribute 'read_bits64' Trackster works OK when I load smaller data sets. It seems that there was a fix for this in bx-python code, as per mail from Jeremy (http://dev.list.galaxyproject.org/trackster-error-for-viewing-rat-data-rn5-…) How do I get the fixed code into my galaxy instance? Thanks, -Raj

2 5

Wrapping tools with optional data files
by Ganote, Carrie L 16 Apr '14

16 Apr '14

Hi List, I am wrapping a Galaxy tool for the GFFread utility from Cufflinks. It has a few optional files as inputs, but I can't get this to translate to the tool wrapper. I started with a tool that has an advanced options menu. When this is selected as "advanced", it brings up a bunch of parameters, including the optional fasta file. If I select something for the optional field, it works fine; if I don't select anything (it says "Selection is Optional"), I get this error and the tool doesn't run: Error executing tool: 'param name' If I use the basic options, it works fine. This seems to have nothing to do with the <command> tag, which can never mention the param name. I looked for an example of a successful optional file tool and found the GATK depth of coverage tool. It doesn't have that conditional tag around its optional file, though - and when I add the advanced options conditional, it breaks it. Here's the prototype: <tool id="gffread" name="GFFread" version="0.0.1"> <command> gffread $input_file -o $output_file #if $adv_opts.adv_opts_selector == "advanced": #if str( $adv_opts.input_map ) != "None": -g $adv_opts.input_map #end if #end if </command> <inputs> <param name="input_file" type="data" format="gff,gtf,gff3" label="GFF file"/> <param name="works" type="data" format="fasta" label="This one works" optional="True" /> <conditional name="adv_opts"> <param name="adv_opts_selector" type="select" label="Advanced Options"> <option value="basic" selected="True">Hide Advanced Options</option> <option value="advanced">Show Advanced Options</option> </param> <when value="advanced" > <param name="input_map" type="data" format="fasta" label="Fasta map file" optional="True" /> </when> </conditional> </inputs> ... Is there another way around this? Thanks, Carrie Ganote

1 0

Unclear API and Twill test failure for undefined datatype
by Peter Cock 16 Apr '14

16 Apr '14

Hi all, I am using TravisCI for testing my Galaxy Tools developed on GitHub, see: http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy… Currently this uses a hard coded datatypes_conf.xml file consisting of the local datatypes in use (e.g. BLAST databases) plus a cut down list those core dataypes also being used (e.g. FASTA). The following test run failed because one of the newly added tests used "sam" format, which was not defined: https://travis-ci.org/peterjc/pico_galaxy/builds/23041174 I've added the SAM definition now, the reason for this email is to suggest clearer error messages from the test framework for undefined dataypes. Currently Twill (default) said: ====================================================================== ERROR: test_tool_000001 (functional.test_toolbox.TestForTool_samtools_depad) BAM mapping statistics ( samtools_depad ) > Test-2 ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py", line 106, in test_tool self.do_it( td ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py", line 29, in do_it stage_data_in_history( galaxy_interactor, testdef.test_data(), test_history, shed_tool_id ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py", line 30, in stage_data_in_history upload_waits.append( galaxy_interactor.stage_data_async( test_data, history, shed_tool_id ) ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py", line 336, in stage_data_async wait=(not async) ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/twilltestcase.py", line 209, in upload_file self.refresh_form( "file_type", ftype ) # Refresh, to support composite files File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/twilltestcase.py", line 1369, in refresh_form tc.fv( f.name, control.name, value ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/twill-0.9-py2.7.egg/twill/commands.py", line 450, in formvalue set_form_control_value(control, value) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/twill-0.9-py2.7.egg/twill/utils.py", line 190, in set_form_control_value raise ClientForm.ItemNotFoundError('cannot find value/label "%s" in list control' % (val,)) ItemNotFoundError: cannot find value/label "sam" in list control Similarly unclear, the API test framework said: galaxy.web.framework: ERROR: Uncaught exception in exposed API method: Traceback (most recent call last): File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/web/framework/__init__.py", line 199, in decorator rval = func( self, trans, *args, **kwargs) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/webapps/galaxy/api/tools.py", line 130, in create template, vars = tool.handle_input( trans, incoming, history=target_history, process_state=process_state, source="json" ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py", line 1921, in handle_input errors, params = self.__check_param_values( trans, incoming, state, old_errors, process_state, history=history, source=source ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py", line 2029, in __check_param_values errors = self.populate_state( trans, inputs, state.inputs, incoming, history, source=source ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py", line 2136, in populate_state current_case = input.get_current_case( value, trans ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/parameters/grouping.py", line 462, in get_current_case raise Exception( "No case matched value:", self.name, str_value ) Exception: ('No case matched value:', 'files_metadata', u'sam') requests.packages.urllib3.connectionpool: DEBUG: "POST /api/tools HTTP/1.1" 500 None In both cases, I would prefer something like a ValueError explicitly saying the problem is an undefined dataype (in this case, "sam" was not defined). This will be most useful when dealing with 3rd party tools using 3rd part datatype definitions. Thanks, Peter

2 1

Problem upgrading bx-python to fix trackster error.
by William Holtz 16 Apr '14

16 Apr '14

I have a set of galaxy generated bam and bigwig files containing over 600 contigs. When I attempt to view any of these files with trackster (on a local server running current galaxy-dist) I get the following in my log file: galaxy.webapps.galaxy.api.datasets ERROR 2014-04-09 17:41:16,063 Error in dataset API at listing contents: 'BinaryFileReader' object has no attribute 'read_bits64': 'BinaryFileReader' object has no attribute 'read_bits64' Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 44, in show is_true( kwd.get( 'retry', False ) ) ) File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 106, in _converted_datasets_state if not data_provider.has_data( chrom ): File "/home/galaxy/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py", line 1078, in has_data all_dat = bbi.query( chrom, 0, 2147483647, 1 ) or \ File "bbi_file.pyx", line 215, in bx.bbi.bbi_file.BBIFile.query (lib/bx/bbi/bbi_file.c:5596) File "bbi_file.pyx", line 222, in bx.bbi.bbi_file.BBIFile.query (lib/bx/bbi/bbi_file.c:5210) File "bbi_file.pyx", line 183, in bx.bbi.bbi_file.BBIFile.summarize (lib/bx/bbi/bbi_file.c:4475) File "bbi_file.pyx", line 248, in bx.bbi.bbi_file.BBIFile._get_chrom_id_and_size (lib/bx/bbi/bbi_file.c:5656) File "bpt_file.pyx", line 76, in bx.bbi.bpt_file.BPTFile.find (lib/bx/bbi/bpt_file.c:1388) File "bpt_file.pyx", line 55, in bx.bbi.bpt_file.BPTFile.r_find (lib/bx/bbi/bpt_file.c:1154) AttributeError: 'BinaryFileReader' object has no attribute 'read_bits64' Two people have previously reported similar errors to this list. I don't know how to reply to messages that predate me joining the list, so here are links to the threads: http://dev.list.galaxyproject.org/read-bits64-error-when-loading-large-geno… http://dev.list.galaxyproject.org/trackster-error-for-viewing-rat-data-rn5-… Following the suggestion of Jeremy Goecks in one of those old threads, I checked my bx-python version. The directory name bx-python is installed in indicates that I have version 0.7.1. Version 0.7.1 is from 2011 and pre-dates the the bug Jeremy mentioned (https://bitbucket.org/james_taylor/bx-python/commits/24893a84ce13a18815e300…) Additionally, I ran: strings ./galaxy-dist/eggs/bx_python-0.7.1_7b95ff194725-py2.7-linux-x86_64-ucs4.egg/bx/bbi/bpt_file.so | grep read_bits64 and found a match, thus confirming the installed version pre-dates the bug fix, as all references to read_bits64 were removed in that commit. Additionally I cloned and compiled the current version of bx-python and found that the above command did not find any matches. The offending installation of bx-python is part of the base installation of galaxy and not an item I added from the toolshed. I am unsure of how to go about replacing this egg with the newer version of bx-python. If someone can update the bx-python egg on http://eggs.g2.bx.psu.edu/ or give me instructions on how to replace my copy of the egg, I would greatly appreciate it! Additionally, I found that all three of the versions of bx-python in the galaxy-main toolshed still contain this bug. Of the three versions of bx-python int the galaxy-test toolshed, only 3:5457e32b427c (2013-12-14) contains the bug fix. Thank you in advance for your assistance! -Will

2 1