November 2011 - galaxy-dev - lists.galaxyproject.org

extract_genomic_dna.py checks alignseq.loc?
by Holger Klein 30 Nov '11

30 Nov '11

Hi all, I recently upgraded our local instance of galaxy to the latest revision of galaxy-dist. Now I tried to use "Extract Genomic DNA" with a 3 column bed file, which only contains chr, start and end, with database/build set to hg19. First of all, running the module results in the following error: --->%--- Traceback (most recent call last): File "/local/data/home/galaxy/galaxy-dist-2011-11-23/tools/extract/extract_genomic_dna.py", line 283, in <module> if __name__ == "__main__": __main__() File "/local/data/home/galaxy/galaxy-dist-2011-11-23/tools/extract/extract_genomic_dna.py", line 107, in __main__ seq_path = check_seq_file( dbkey, GALAXY_DATA_INDEX_DIR ) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/tools/extract/extract_genomic_dna.py", line 40, in check_seq_file for line in open( seq_file ): NameError: global name 'seq_file' is not defined ---%<--- I went to the respective file and found that the line which defines seq_file is commented out: (l.38 in def check_seq_file()) ## seq_file = "%s/alignseq.loc" % GALAXY_DATA_INDEX_DIR This seems to be a bug in the current version of the file. Removing the comment, the script tries to check for sequence entries in alignseq.loc, which I left empty before, since I didn't need aligned sequences in galaxy until now. Of course this results in another error: 'No sequences are available for 'hg19', request them by reporting this error.' I just wanted to raise the question if this dependency is right, wouldn't one rather like to check for the respective build in faseq.loc (unfortunately the file format is different, it doesn't contain the seq in the first column). Is there a fix for this somewhere already, or did I misunderstand how this is supposed to work? Cheers, Holger -- Dr. Holger Klein Core Facility Bioinformatics Institute of Molecular Biology gGmbH (IMB) http://www.imb-mainz.de/ Tel: +49(6131) 39 21511

2 1

Re: [galaxy-dev] jquery script error
by Iry Witham 30 Nov '11

30 Nov '11

I have have discovered that this error is not unique to the number of files I am linking. It has been reported from several of my users that there is a similar issue with Internet Explorer 8. Step 1: Import shared data into history: One user tried to import data from a shared library into her history. Internet Explorer puts up an ugly message “Stop running this script?” Complaining that continuing to run the script will cause her machine to stop functioning. Sometimes the display seemed to get truncated, only displaying a subset of the available datasets. The import seemed to work OK, though. Step 2: Debug the issue using IE built-in script debugger. I was able to replicate the issue and ran some minor debugging from within IE. The results of this are as follows: Error Message: Windows Internet Explorer Stop running this script? A script on this page is causing Internet Explorer to run slowly. If it continues to run, your computer might become unresponsive. Debugging results: Line: 1 Error: jQuery, MooTools or Prototype needs to be loaded before jStorage! (function(f){if(!f||!(f.toJSON||Object.toJSON||window.JSON)){throw new Error("jQuery, MooTools or Prototype needs to be loaded before jStorage!")}var g={},d={jStorage:"{}"},h=null,j=0,l=f.toJSON||Object.toJSON||(window.JSON&&(JSON.encode||JSON.stringify)),e=f.evalJSON||(window.JSON&&(JSON.decode||JSON.parse))||function(m){return String(m).evalJSON()},i=false;_XMLService={isXML:function(n){var m=(n?n.ownerDocument||n:0).documentElement;return m?m.nodeName!=="HTML":false},encode:function(n){if(!this.isXML(n)){return false}try{return new XMLSerializer().serializeToString(n)}catch(m){try{return n.xml}catch(o){}}return false},decode:function(n){var m=("DOMParser" in window&&(new DOMParser()).parseFromString)||(window.ActiveXObject&&function(p){var q=new ActiveXObject("Microsoft.XMLDOM");q.async="false";q.loadXML(p);return q}),o;if(!m){return false}o=m.call("DOMParser" in window&&(new DOMParser())||window,n,"text/xml");return this.isXML(o)?o:false}};function k(){if("localStorage" in window){try{if(window.localStorage){d=window.localStorage;i="localStorage"}}catch(p){}}else{if("globalStorage" in window){try{if(window.globalStorage){d=window.globalStorage[window.location.hostname];i="globalStorage"}}catch(o){}}else{h=document.createElement("link");if(h.addBehavior){h.style.behavior="url(#default#userData)";document.getElementsByTagName("head")[0].appendChild(h);h.load("jStorage");var n="{}";try{n=h.getAttribute("jStorage")}catch(m){}d.jStorage=n;i="userDataBehavior"}else{h=null;return}}}b()}function b(){if(d.jStorage){try{g=e(String(d.jStorage))}catch(m){d.jStorage="{}"}}else{d.jStorage="{}"}j=d.jStorage?String(d.jStorage).length:0}function c(){try{d.jStorage=l(g);if(h){h.setAttribute("jStorage",d.jStorage);h.save("jStorage")}j=d.jStorage?String(d.jStorage).length:0}catch(m){}}function a(m){if(!m||(typeof m!="string"&&typeof m!="number")){throw new TypeError("Key name must be string or numeric")}return true}f.jStorage={version:"0.1.5.0",set:function(m,n){a(m);if(_XMLService.isXML(n)){n={_is_xml:true,xml:_XMLService.encode(n)}}g[m]=n;c();return n},get:function(m,n){a(m);if(m in g){if(typeof g[m]=="object"&&g[m]._is_xml&&g[m]._is_xml){return _XMLService.decode(g[m].xml)}else{return g[m]}}return typeof(n)=="undefined"?null:n},deleteKey:function(m){a(m);if(m in g){delete g[m];c();return true}return false},flush:function(){g={};c();try{window.localStorage.clear()}catch(m){}return true},storageObj:function(){function m(){}m.prototype=g;return new m()},index:function(){var m=[],n;for(n in g){if(g.hasOwnProperty(n)){m.push(n)}}return m},storageSize:function(){return j},currentBackend:function(){return i},storageAvailable:function(){return !!i},reInit:function(){var m,o;if(h&&h.addBehavior){m=document.createElement("link");h.parentNode.replaceChild(m,h);h=m;h.style.behavior="url(#default#userData)";document.getElementsByTagName("head")[0].appendChild(h);h.load("jStorage");o="{}";try{o=h.getAttribute("jStorage")}catch(n){}d.jStorage=o;i="userDataBehavior"}b()}};k()})(window.jQuery||window.$); jQuery, MooTools or Prototype needs to be loaded before jStorage! I cannot replicate this with other browsers. I have tried it with Google Crome and Firefox 3.6.20. Thanks, Iry Witham

2 1

Re: [galaxy-dev] Proposed patch to blastxml_to_tabular.py
by Peter Cock 30 Nov '11

30 Nov '11

On Wed, Nov 30, 2011 at 9:49 AM, Peter van Heusden wrote: > Yes, this went away with newer versions of BLAST (and BLAST+). > The 2.2.15 version is from 2006. OK, that would explain why I didn't find the issue myself during the development. > I've added test data and tests to the tool's XML file. Is this on a bitbucket fork? > Do you know how to run a single tool's test in Galaxy? > > Peter Yes, run_functional_tests.sh supports this. You'll need the test ID, which you can work out from the wrapper XML, or $ ./run_functional_tests.sh -list ... Then, in this case, $ ./run_functional_tests.sh -id blastxml_to_tabular ... Peter

2 1

Proposed patch to blastxml_to_tabular.py
by Peter van Heusden 30 Nov '11

30 Nov '11

Hi there. When trying to convert some BLAST XML to tabular format, from an old version of BLAST (2.2.15), I had to make some changes to blastxml_to_tabular.py to get correct results. A patch is enclosed in this message (and attached). I don't think they should negatively affect anyone, but the semantics are as follows: 1) If you find an Iteration with no Iteration_query-ID tag, return skip it, it has no hits. 1) If the Iteration_query-ID tag matches regexp lcl\|\d+_\d+ (e.g. lcl|1_0) the real accession is in the first word of Iteration_query-def. 2) If the Hit_id matches regexp gnl\|BL_ORD_ID\|d+ (e.g. gnl|BL_ORD_ID|278) and the Hit_accession tag contains the text of that \d+ (e.g. 278) then the real accession is in the first word of Hit_def. I'm basing these semantic on the BLAST reports I've got here, so if anyone can see an error in them please speak up. Else, here is the patch: --- a/tools/ncbi_blast_plus/blastxml_to_tabular.py Tue Nov 29 17:35:14 2011 -0500 +++ b/tools/ncbi_blast_plus/blastxml_to_tabular.py Wed Nov 30 09:31:11 2011 +0200 @@ -103,7 +103,7 @@ stop_err( "Invalid data format." ) -re_default_query_id = re.compile("^Query_\d+$") +re_default_query_id = re.compile("^(lcl\|\d+_\d+|Query_\d+)$") assert re_default_query_id.match("Query_101") assert not re_default_query_id.match("Query_101a") assert not re_default_query_id.match("MyQuery_101") @@ -112,6 +112,7 @@ assert not re_default_subject_id.match("Subject_") assert not re_default_subject_id.match("Subject_12a") assert not re_default_subject_id.match("TheSubject_1") +re_default_subject_id2 = re.compile("^gnl\|BL_ORD_ID\|(\d+)$") outfile = open(out_file, 'w') @@ -133,6 +134,9 @@ # <Iteration_query-len>516</Iteration_query-len> # <Iteration_hits>... qseqid = elem.findtext("Iteration_query-ID") + if qseqid == None: + # no query ID - this happens when there are no hits, so skip this Iteration + continue if re_default_query_id.match(qseqid): #Place holder ID, take the first word of the query definition qseqid = elem.findtext("Iteration_query-def").split(None,1)[0] @@ -152,8 +156,10 @@ #apparently depending on the parse_deflines switch sseqid = hit.findtext("Hit_id").split(None,1)[0] hit_def = sseqid + " " + hit.findtext("Hit_def") - if re_default_subject_id.match(sseqid) \ - and sseqid == hit.findtext("Hit_accession"): + match2 = re_default_subject_id2.match(sseqid) + if (re_default_subject_id.match(sseqid) \ + and sseqid == hit.findtext("Hit_accession")) \ + or (match2 and match2.group(1) == hit.findtext("Hit_accession")): #Place holder ID, take the first word of the subject definition hit_def = hit.findtext("Hit_def") sseqid = hit_def.split(None,1)[0]

2 1

FASTA manipulation!
by David PANG 29 Nov '11

29 Nov '11

Dear All, I am new to Galaxy! When I tried to manipulate mt fasta sequences from RNA-Seq, I just add >1, >2...........>1000000 etc! Right now, I am trying to add libraryA to all of the sequences like >libraryA_1, >libraryA_2.........>libraryA_1000000. I can not figure out how to do it on Galaxy. Could somebody help to this end. Thanks David -- David PANG, Ph.D.

2 1

custom indexes
by Candace Seeve 29 Nov '11

29 Nov '11

I'd like to map RNA-Seq data from two different bacterial species to their genomes. How do I go about getting a custom index added to the galaxy server? One of the genomes is not yet published so is it possible to keep the indexes private? Thanks, Candace

2 1

Integration with LIMS
by Sonderegger Bernhard 29 Nov '11

29 Nov '11

Hello, I am exploring the possibility of using a local galaxy installation for light bioinformatics on sequence data in the scope of an existing LIMS system. Ideally I would like to be able to do the following 1. Push datasets from the LIMS to galaxy (directly into the user's account or into a fresh temporary session) 2. Allow the user to perform tasks within galaxy, backtracking and retrying as necessary. 3. Push the results along with the galaxy history (ideally converted to a workflow) back to the LIMS 4. (Allow workflows stored in the LIMS to be pushed to Galaxy along with further datasets or even completely automatic launch of these workflows from within the LIMS) Is this feasible? How much wrok would be required? Since I am new to Galaxy (at least from the development side), I would very much appreciate comments advice and pointers to documentation or existing projects of a similar nature. Thanks in advance, Bernhard

2 1

trannsfer files from remote system to galaxy
by shashi shekhar 29 Nov '11

29 Nov '11

Hi, In my local instance of Galaxy ,I want to add one option in which i can get files from remote system to galaxy in data library . except url is there any option to get remote files from galaxy ? Regards

2 1

Shogun Toolbox?
by Kipper Fletez-Brant 29 Nov '11

29 Nov '11

Dear List, I am new to developing for Galaxy and haven't found a good answer to this question yet. Does Galaxy Test Toolshed use or recognize Shogun Toolbox (http://www.shogun-toolbox.org/)? I'm considering writing a module for Galaxy, but the underlying Python code relies on Shogun, and I haven't seen a suppported 3rd party software list, so I can't tell if Galaxy would be able to run the module without my rewriting it not to depend on Shogun. Thank you, Kipper Fletez-Brant

2 4

galaxy core services vs wrapper.py duplication & caching of .fai files
by Robert Curtis Hendrickson 29 Nov '11

29 Nov '11

Folks, I was writing a samtools/mpileup wrapper for our local use. When I delved into how the existing samtools/sam_pileup.py adaptor worked, if found that it has a local copy of the routine to look up the samtools .fai file in sam_fa_indices.loc for "installed" genomes. I then noticed that this routine is duplicated in many different adaptors: [curtish@cheaha galaxy]$ find . -name "*.py" | xargs grep sam_fa_indices.loc ./tools/samtools/sam_pileup.py: seqFile = '%s/sam_fa_indices.loc' % GALAXY_DATA_INDEX_DIR ./tools/samtools/sam_mpileup_view.py: seqFile = '%s/sam_fa_indices.loc' % GALAXY_DATA_INDEX_DIR ./tools/samtools/sam_to_bam.py: cached_seqs_pointer_file = '%s/sam_fa_indices.loc' % options.index_dir ./tools/ngs_rna/cufflinks_wrapper_with_gtf.py: cached_seqs_pointer_file = os.path.join( options.index_dir, 'sam_fa_indices.loc' ) ./tools/ngs_rna/cuffdiff_wrapper.py: cached_seqs_pointer_file = os.path.join( options.index_dir, 'sam_fa_indices.loc' ) ./tools/ngs_rna/cufflinks_wrapper.py: cached_seqs_pointer_file = os.path.join( options.index_dir, 'sam_fa_indices.loc' ) ./tools/ngs_rna/cuffcompare_wrapper.py: cached_seqs_pointer_file = os.path.join( options.index_dir, 'sam_fa_indices.loc' ) ./tools/ngs_rna/cufflinks_wrapper_without_gtf.py: cached_seqs_pointer_file = os.path.join( options.index_dir, 'sam_fa_indices.loc' ) Is there any place in galaxy-core where such a core service lives and could be used by all these adaptors, rather than replicating the code everywhere? As a related question, for fasta genomes from the current history, these wrappers compute the .fai file on the fly, in TMP, then throw it away, every time. Has there been any discussion about storing such derived indices in the dataset's metadata (like the .bai file on a .bam data set), so it gets computed once, then re-used?

2 1