January 2009 - galaxy-dev - lists.galaxyproject.org

[hg] galaxy 1719: Fix an infinite loop in eggs.require()
by Nate Coraor 23 Jan '09

23 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/4fa4d0ddbf1f changeset: 1719:4fa4d0ddbf1f user: Nate Coraor <nate(a)bx.psu.edu> date: Fri Jan 23 12:55:35 2009 -0500 description: Fix an infinite loop in eggs.require() 1 file(s) affected in this change: lib/galaxy/eggs/__init__.py diffs (31 lines): diff -r 520c6e39b3d0 -r 4fa4d0ddbf1f lib/galaxy/eggs/__init__.py --- a/lib/galaxy/eggs/__init__.py Thu Jan 22 12:46:37 2009 -0500 +++ b/lib/galaxy/eggs/__init__.py Fri Jan 23 12:55:35 2009 -0500 @@ -530,7 +530,7 @@ return except pkg_resources.VersionConflict, e: # there's a conflicting egg on the pythonpath, remove it - dist = pkg_resources.get_distribution( name ) + dist = e.args[0] working_set = pkg_resources.working_set # use the canonical path for comparisons location = os.path.realpath( dist.location ) @@ -543,18 +543,6 @@ del working_set.by_key[dist.key] working_set.entry_keys[entry] = [] sys.path.remove(entry) - # get - egg.find() - if not egg.have: - if not egg.fetch(): - raise EggNotFetchable( egg.name ) - pkg_resources.working_set.require( "%s==%s" % ( name, egg.get_vertag() ) ) - if dist.project_name in sys.modules: - try: - mod = sys.modules[dist.project_name] - reload( mod ) - except: - raise NewEgg( "Galaxy downloaded a new egg (%s) but was unable to reload the module it contained. Please try starting Galaxy again." % egg.name ) except pkg_resources.DistributionNotFound, e: # the initial require itself is the first dep, but it can have # multiple deps, which will be fetched by the require below.

1 0

[hg] galaxy 1718: Changed grouping output to tabular
by Nate Coraor 23 Jan '09

23 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/520c6e39b3d0 changeset: 1718:520c6e39b3d0 user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Thu Jan 22 12:46:37 2009 -0500 description: Changed grouping output to tabular 1 file(s) affected in this change: tools/stats/grouping.xml diffs (12 lines): diff -r 4004ab767cf6 -r 520c6e39b3d0 tools/stats/grouping.xml --- a/tools/stats/grouping.xml Thu Jan 22 12:29:44 2009 -0500 +++ b/tools/stats/grouping.xml Thu Jan 22 12:46:37 2009 -0500 @@ -33,7 +33,7 @@ </repeat> </inputs> <outputs> - <data format="input" name="out_file1" metadata_source="input1" /> + <data format="tabular" name="out_file1" /> </outputs> <requirements> <requirement type="python-module">rpy</requirement>

1 0

[hg] galaxy 1717: Update welcome page with Anton's copy from sec...
by Nate Coraor 22 Jan '09

22 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/4004ab767cf6 changeset: 1717:4004ab767cf6 user: Nate Coraor <nate(a)bx.psu.edu> date: Thu Jan 22 12:29:44 2009 -0500 description: Update welcome page with Anton's copy from security 1 file(s) affected in this change: static/welcome.html diffs (116 lines): diff -r 19d18731ce19 -r 4004ab767cf6 static/welcome.html --- a/static/welcome.html Thu Jan 22 11:22:39 2009 -0500 +++ b/static/welcome.html Thu Jan 22 12:29:44 2009 -0500 @@ -6,48 +6,80 @@ <meta name="generator" content="Docutils 0.3.9: http://docutils.sourceforge.net/" /> <title></title> <link rel="stylesheet" href="style/base.css" type="text/css" /> + <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> + <meta name="generator" content="Docutils 0.3.9: http://docutils.sourceforge.net/" /> + <title></title> + <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> + <meta name="generator" content="Docutils 0.6: http://docutils.sourceforge.net/"/> + <script src="http://www.apple.com/library/quicktime/scripts/ac_quicktime.js" language="JavaScript" type="text/javascript"></script> + <script src="http://www.apple.com/library/quicktime/scripts/qtp_library.js" language="JavaScript" type="text/javascript"></script> + <link href="http://www.apple.com/library/quicktime/stylesheets/qtp_library.css" rel="StyleSheet" type="text/css" /> + <link rel="stylesheet" href="style/base.css" type="text/css" /> + </head> + </head> <body> <div class="document"> -<div class="warningmessage"> - <strong>Galaxy and 2008 Meeting Season</strong> - <hr> - <ul> - <li><a target="_blank" href="http://www.beyondgenome.com/08Personalized_Medicine_Day1.asp">Beyond Genome</a> | San Francisco | June 9 - 11</li> - <li>ISMB 2008 | Toronto | July 19 - 23</li> - <li>Genome Informatics | Hinxton, UK | September 10 - 14</li> - <li><a target="_blank" href="http://www.ashg.org/2008meeting/pages/workshops.shtml#4">ASHG</a> | Philadelphia | November 11 - 15</li> - </ul> - <hr> - Download <strong>new</strong> Galaxy brochure <a href="/static/images/brochure.pdf">here</a>. +<div class="infomessage"> + <strong>We are hiring!</strong> + <hr> + Thanks to your support and an unprecedented level of usage, we are looking for an experienced software developer to join our team. For more information about this position, please, click <a target="_blank" href="https://www.bx.psu.edu/cgi-bin/trac.cgi/galaxy/wiki/PythonDeveloper">here</a>. </div> <hr> -<div class="section" align="center"> - <strong>Unsequenced Genomes of the World</strong> | June 2008 - <br> - <br> - <img src="images/welcomePhoto.jpg" border="0"> - <br> - Przewalski's Horse (<i>Equus przewalskii</i>) | Escondido, California - <br> - <br> -</div> +<div class="welcomeBlue" id="screencasts" align="center"> +<strong>Introducing Galactic Quickies</strong> + <hr> + Galactic quickies are <i>super-short</i> screencasts that are <i>always</i> under 5 minutes. We thought it may be a good way to spread the word about Galaxy's functionality while keeping the "annoyance factor" to the minimum. The quickies will be updated weekly. + <hr> + <table width="100%" border="0" height="100%"> + <tr> + <td valign="middle" align="center"> + + <br> + <script type="text/javascript"> + + </script> + <noscript> + <object width="640" height="480" classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B" codebase="http://www.apple.com/qtactivex/qtplugin.cab"> + <param name="src" value="http://screencast.g2.bx.psu.edu/galaxy/quickie1_TabSeq/quickie1_TabSeq.jpg" /> + <param name="href" value="http://screencast.g2.bx.psu.edu/galaxy/quickie1_TabSeq/quickie1_TabSeq.mov" /> + <param name="target" value="myself" /> + <param name="controller" value="false" /> + <param name="autoplay" value="false" /> + <param name="scale" value="aspect" /> + <embed width="640" height="480" type="video/quicktime" pluginspage="http://www.apple.com/quicktime/download/" + src="http://screencast.g2.bx.psu.edu/galaxy/quickie1_TabSeq/quickie1_TabSeq.jpg" + href="http://screencast.g2.bx.psu.edu/galaxy/quickie1_TabSeq/quickie1_TabSeq.mov" + target="myself" + controller="false" + autoplay="false" + scale="aspect"> + </embed> + </object> + </noscript> + + </td> + </tr> + </table> + <br> + For a high resolution version <a target="_blank" class="reference" href="http://screencast.g2.bx.psu.edu/galaxy/quickie1_TabSeq/">click here</a>. + <br> + <hr> + </div> <hr> -<div class="welcomeBlue"> - <strong>Galaxy is for Biologists</strong> - <br> - Use this site to access popular sources of data like the UCSC Table Browser. Run analyses right on the spot using a variety of integrated tools. Your results are always available and can be easily shared with others. Just <a target="_blank" href="http://g2.trac.bx.psu.edu/wiki/ScreenCasts">watch</a> how. -</div> -<br> -<div class="welcomeRed"> - <strong>Galaxy is for Developers</strong> - <br> - Galaxy is an easy-to-use, open-source, scalable framework for tool and data integration. Stop wasting time writing interfaces and get your tools used by biologists! Galaxy includes everything you need to get started, so <a target="_blank" href="http://g2.trac.bx.psu.edu/wiki/HowToInstall">download</a> and start <a target="_blank" href="http://g2.trac.bx.psu.edu/wiki/HowToInstall">integrating</a>! -</div> + <p><a target="_blank" class="reference" href="http://g2.trac.bx.psu.edu/wiki/GalaxyTeam">Galaxy team</a> is a part of <a target="_blank" class="reference" href="http://www.bx.psu.edu">BX</a> at <a target="_blank" class="reference" href="http://www.psu.edu">Penn State</a>.</p> <hr class="docutils" /> This project is supported in part by <a target="_blank" class="reference" href="http://www.nsf.gov">NSF</a> and <a target="_blank" class="reference" href="http://www.huck.psu.edu">the Huck Institutes of the Life Sciences</a>. <p><small>Galaxy build: <b>$Rev$</b></small></p>

1 0

[hg] galaxy 1716: Fixes in the upload tool for removing temporar...
by Nate Coraor 22 Jan '09

22 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/19d18731ce19 changeset: 1716:19d18731ce19 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Thu Jan 22 11:22:39 2009 -0500 description: Fixes in the upload tool for removing temporary files. 1 file(s) affected in this change: lib/galaxy/tools/actions/upload.py diffs (81 lines): diff -r 5c293dbf0726 -r 19d18731ce19 lib/galaxy/tools/actions/upload.py --- a/lib/galaxy/tools/actions/upload.py Thu Jan 22 08:50:52 2009 -0500 +++ b/lib/galaxy/tools/actions/upload.py Thu Jan 22 11:22:39 2009 -0500 @@ -13,7 +13,11 @@ def __init__( self ): self.empty = False self.line_count = None - + def remove_tempfile( self, filename ): + try: + os.unlink( filename ) + except: + log.exception( 'failure removing temporary file: %s' % filename ) def execute( self, tool, trans, incoming={}, set_output_hid = True ): data_file = incoming['file_data'] file_type = incoming['file_type'] @@ -48,6 +52,7 @@ data_list.append( self.add_file( trans, data_file.local_filename, file_name, file_type, dbkey, space_to_tab=space_to_tab ) ) except Exception, e: log.exception( 'exception in add_file using datafile.local_filename %s: %s' % ( data_file.local_filename, str( e ) ) ) + self.remove_tempfile( data_file.local_filename ) return self.upload_empty( trans, job, "Error:", str( e ) ) elif 'filename' in dir( data_file ): file_name = data_file.filename @@ -57,16 +62,13 @@ temp_name = sniff.stream_to_file( data_file.file, prefix='upload' ) except Exception, e: log.exception( 'exception in sniff.stream_to_file using file %s: %s' % ( data_file.filename, str( e ) ) ) - try: - # Attempt to remove temporary file - os.unlink( temp_name ) - except: - log.exception( 'failure removing temporary file: %s' % temp_name ) + self.remove_tempfile( temp_name ) return self.upload_empty( trans, job, "Error:", str( e ) ) try: data_list.append( self.add_file( trans, temp_name, file_name, file_type, dbkey, space_to_tab=space_to_tab ) ) except Exception, e: log.exception( 'exception in add_file using file temp_name %s: %s' % ( str( temp_name ), str( e ) ) ) + self.remove_tempfile( temp_name ) return self.upload_empty( trans, job, "Error:", str( e ) ) if url_paste not in [ None, "" ]: if url_paste.lower().find( 'http://' ) >= 0 or url_paste.lower().find( 'ftp://' ) >= 0: @@ -78,16 +80,13 @@ temp_name = sniff.stream_to_file( urllib.urlopen( line ), prefix='url_paste' ) except Exception, e: log.exception( 'exception in sniff.stream_to_file using url_paste %s: %s' % ( url_paste, str( e ) ) ) - try: - # Attempt to remove temporary file - os.unlink( temp_name ) - except: - log.exception( 'failure removing temporary file: %s' % temp_name ) + self.remove_tempfile( temp_name ) return self.upload_empty( trans, job, "Error:", str( e ) ) try: data_list.append( self.add_file( trans, temp_name, line, file_type, dbkey, info="uploaded url", space_to_tab=space_to_tab ) ) except Exception, e: log.exception( 'exception in add_file using url_paste temp_name %s: %s' % ( str( temp_name ), str( e ) ) ) + self.remove_tempfile( temp_name ) return self.upload_empty( trans, job, "Error:", str( e ) ) else: is_valid = False @@ -101,16 +100,13 @@ temp_name = sniff.stream_to_file( StringIO.StringIO( url_paste ), prefix='strio_url_paste' ) except Exception, e: log.exception( 'exception in sniff.stream_to_file using StringIO.StringIO( url_paste ) %s: %s' % ( url_paste, str( e ) ) ) - try: - # Attempt to remove temporary file - os.unlink( temp_name ) - except: - log.exception( 'failure removing temporary file: %s' % temp_name ) + self.remove_tempfile( temp_name ) return self.upload_empty( trans, job, "Error:", str( e ) ) try: data_list.append( self.add_file( trans, temp_name, 'Pasted Entry', file_type, dbkey, info="pasted entry", space_to_tab=space_to_tab ) ) except Exception, e: log.excception( 'exception in add_file using StringIO.StringIO( url_paste ) temp_name %s: %s' % ( str( temp_name ), str( e ) ) ) + self.remove_tempfile( temp_name ) return self.upload_empty( trans, job, "Error:", str( e ) ) else: return self.upload_empty( trans, job, "No data error:", "you pasted no data." )

1 0

[hg] galaxy 1713: remove the usage of hast tables in the fasta-t...
by Greg Von Kuster 22 Jan '09

22 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/6d849785ff86 changeset: 1713:6d849785ff86 user: wychung date: Mon Jan 19 15:47:02 2009 -0500 description: remove the usage of hast tables in the fasta-tools 8 file(s) affected in this change: test-data/fasta_to_tabular_out1.tabular test-data/fasta_to_tabular_out3.tabular test-data/fasta_tool_compute_length_1.out test-data/fasta_tool_compute_length_3.out test-data/fasta_tool_filter_length_1.out tools/fasta_tools/fasta_compute_length.py tools/fasta_tools/fasta_filter_by_length.py tools/fasta_tools/fasta_to_tabular.py diffs (305 lines): diff -r e76a153769d4 -r 6d849785ff86 test-data/fasta_to_tabular_out1.tabular --- a/test-data/fasta_to_tabular_out1.tabular Mon Jan 19 11:47:29 2009 -0500 +++ b/test-data/fasta_to_tabular_out1.tabular Mon Jan 19 15:47:02 2009 -0500 @@ -14,5 +14,5 @@ EYKX4VC01BB4QL length=57 xy=0431_0363 region=1 run=R_2007_11_07_16_15_57_ GGGGAGGAGCTAATAATATGCTCTTGGGGAGGAGCTAATTATATGCTCTTGGGGAGG EYKX4VC01BJ37M length=64 xy=0522_0192 region=1 run=R_2007_11_07_16_15_57_ TCGAGTATGTATCAAGGACTACATACAAATTTGCCAAAAGAGATTATGCACTATCCCGACTTCC EYKX4VC01BV9R8 length=54 xy=0660_2038 region=1 run=R_2007_11_07_16_15_57_ AAAACTCGGAGAAACTATTCAGCAGCACTGCGTTTCGCTGAATTTTAGACCGTT +EYKX4VC01CEPP8 length=60 xy=0870_2350 region=1 run=R_2007_11_07_16_15_57_ CTGGGTGGGTGCACTACAGGAACGTCATTTGTTCAATCCTCACGTTGTTGTTAGTGTCAG EYKX4VC01BTLME length=78 xy=0630_0292 region=1 run=R_2007_11_07_16_15_57_ TTATCCACACGCTGTCCGGATCCAGCGCCAGGCGCCGACGCTGGACTTCCGCCGCCTGCGCCCAGTTGCCCTGACTTC -EYKX4VC01CEPP8 length=60 xy=0870_2350 region=1 run=R_2007_11_07_16_15_57_ CTGGGTGGGTGCACTACAGGAACGTCATTTGTTCAATCCTCACGTTGTTGTTAGTGTCAG diff -r e76a153769d4 -r 6d849785ff86 test-data/fasta_to_tabular_out3.tabular --- a/test-data/fasta_to_tabular_out3.tabular Mon Jan 19 11:47:29 2009 -0500 +++ b/test-data/fasta_to_tabular_out3.tabular Mon Jan 19 15:47:02 2009 -0500 @@ -14,5 +14,5 @@ EYKX4VC01BB4QL GGGGAGGAGCTAATAATATGCTCTTGGGGAGGAGCTAATTATATGCTCTTGGGGAGG EYKX4VC01BJ37M TCGAGTATGTATCAAGGACTACATACAAATTTGCCAAAAGAGATTATGCACTATCCCGACTTCC EYKX4VC01BV9R8 AAAACTCGGAGAAACTATTCAGCAGCACTGCGTTTCGCTGAATTTTAGACCGTT +EYKX4VC01CEPP8 CTGGGTGGGTGCACTACAGGAACGTCATTTGTTCAATCCTCACGTTGTTGTTAGTGTCAG EYKX4VC01BTLME TTATCCACACGCTGTCCGGATCCAGCGCCAGGCGCCGACGCTGGACTTCCGCCGCCTGCGCCCAGTTGCCCTGACTTC -EYKX4VC01CEPP8 CTGGGTGGGTGCACTACAGGAACGTCATTTGTTCAATCCTCACGTTGTTGTTAGTGTCAG diff -r e76a153769d4 -r 6d849785ff86 test-data/fasta_tool_compute_length_1.out --- a/test-data/fasta_tool_compute_length_1.out Mon Jan 19 11:47:29 2009 -0500 +++ b/test-data/fasta_tool_compute_length_1.out Mon Jan 19 15:47:02 2009 -0500 @@ -14,5 +14,5 @@ EYKX4VC01BB4QL length=57 xy=0431_0363 region=1 run=R_2007_11_07_16_15_57_ 57 EYKX4VC01BJ37M length=64 xy=0522_0192 region=1 run=R_2007_11_07_16_15_57_ 64 EYKX4VC01BV9R8 length=54 xy=0660_2038 region=1 run=R_2007_11_07_16_15_57_ 54 +EYKX4VC01CEPP8 length=60 xy=0870_2350 region=1 run=R_2007_11_07_16_15_57_ 60 EYKX4VC01BTLME length=78 xy=0630_0292 region=1 run=R_2007_11_07_16_15_57_ 78 -EYKX4VC01CEPP8 length=60 xy=0870_2350 region=1 run=R_2007_11_07_16_15_57_ 60 diff -r e76a153769d4 -r 6d849785ff86 test-data/fasta_tool_compute_length_3.out --- a/test-data/fasta_tool_compute_length_3.out Mon Jan 19 11:47:29 2009 -0500 +++ b/test-data/fasta_tool_compute_length_3.out Mon Jan 19 15:47:02 2009 -0500 @@ -14,5 +14,5 @@ EYKX4VC01BB4QL 57 EYKX4VC01BJ37M 64 EYKX4VC01BV9R8 54 +EYKX4VC01CEPP8 60 EYKX4VC01BTLME 78 -EYKX4VC01CEPP8 60 diff -r e76a153769d4 -r 6d849785ff86 test-data/fasta_tool_filter_length_1.out --- a/test-data/fasta_tool_filter_length_1.out Mon Jan 19 11:47:29 2009 -0500 +++ b/test-data/fasta_tool_filter_length_1.out Mon Jan 19 15:47:02 2009 -0500 @@ -53,9 +53,9 @@ >EYKX4VC01BV9R8 length=54 xy=0660_2038 region=1 run=R_2007_11_07_16_15_57_ AAAACTCGGAGAAACTATTCAGCAGCACTGCGTTTCGCTGAATTTTAGAC CGTT +>EYKX4VC01CEPP8 length=60 xy=0870_2350 region=1 run=R_2007_11_07_16_15_57_ +CTGGGTGGGTGCACTACAGGAACGTCATTTGTTCAATCCTCACGTTGTTG +TTAGTGTCAG >EYKX4VC01BTLME length=78 xy=0630_0292 region=1 run=R_2007_11_07_16_15_57_ TTATCCACACGCTGTCCGGATCCAGCGCCAGGCGCCGACGCTGGACTTCC GCCGCCTGCGCCCAGTTGCCCTGACTTC ->EYKX4VC01CEPP8 length=60 xy=0870_2350 region=1 run=R_2007_11_07_16_15_57_ -CTGGGTGGGTGCACTACAGGAACGTCATTTGTTCAATCCTCACGTTGTTG -TTAGTGTCAG diff -r e76a153769d4 -r 6d849785ff86 tools/fasta_tools/fasta_compute_length.py --- a/tools/fasta_tools/fasta_compute_length.py Mon Jan 19 11:47:29 2009 -0500 +++ b/tools/fasta_tools/fasta_compute_length.py Mon Jan 19 15:47:02 2009 -0500 @@ -1,8 +1,8 @@ #! /usr/bin/python """ -Input: fasta, minimal length, maximal length -Output: fasta -Return sequences whose lengths are within the range. +Input: fasta, int +Output: tabular +Return titles with lengths of corresponding seq """ import sys, os @@ -10,41 +10,37 @@ assert sys.version_info[:2] >= ( 2, 4 ) def __main__(): - input_filename = sys.argv[1] - output_filename = sys.argv[2] + + infile = sys.argv[1] + outfile = sys.argv[2] keep_first = int( sys.argv[3] ) - tmp_title = tmp_seq = '' - tmp_seq_count = 0 - seq_hash = {} + + fasta_title = fasta_seq = '' + # number of char to keep in the title if keep_first == 0: keep_first = None else: keep_first += 1 - for i, line in enumerate( file( input_filename ) ): + out = open(outfile, 'w') + + for i, line in enumerate( file( infile ) ): line = line.rstrip( '\r\n' ) if not line or line.startswith( '#' ): continue if line[0] == '>': - if len( tmp_seq ) > 0: - tmp_seq_count += 1 - seq_hash[ ( tmp_seq_count, tmp_title ) ] = tmp_seq - tmp_title = line - tmp_seq = '' + if len( fasta_seq ) > 0 : + out.write( "%s\t%d\n" % ( fasta_title[ 1:keep_first ], len( fasta_seq ) ) ) + fasta_title = line + fasta_seq = '' else: - tmp_seq = "%s%s" % ( tmp_seq, line ) - if line.split() and line.split()[0].isdigit(): - tmp_seq = "%s " % tmp_seq - if len( tmp_seq ) > 0: - seq_hash[ ( tmp_seq_count, tmp_title ) ] = tmp_seq - - title_keys = seq_hash.keys() - title_keys.sort() - output_handle = open( output_filename, 'w' ) - for i, fasta_title in title_keys: - tmp_seq = seq_hash[ ( i, fasta_title ) ] - output_handle.write( "%s\t%d\n" % ( fasta_title[ 1:keep_first ], len( tmp_seq ) ) ) - output_handle.close() + fasta_seq = "%s%s" % ( fasta_seq, line ) + + # check the last sequence + if len( fasta_seq ) > 0: + out.write( "%s\t%d\n" % ( fasta_title[ 1:keep_first ], len( fasta_seq ) ) ) + + out.close() if __name__ == "__main__" : __main__() \ No newline at end of file diff -r e76a153769d4 -r 6d849785ff86 tools/fasta_tools/fasta_filter_by_length.py --- a/tools/fasta_tools/fasta_filter_by_length.py Mon Jan 19 11:47:29 2009 -0500 +++ b/tools/fasta_tools/fasta_filter_by_length.py Mon Jan 19 15:47:02 2009 -0500 @@ -15,7 +15,7 @@ def __main__(): - input_filename = sys.argv[1] + infile = sys.argv[1] try: min_length = int( sys.argv[2] ) except: @@ -24,49 +24,62 @@ max_length = int( sys.argv[3] ) except: stop_err( "Maximum length of the return sequence requires a numerical value." ) - output_filename = sys.argv[4] - tmp_title = tmp_seq = '' - tmp_seq_count = 0 - seq_hash = {} + outfile = sys.argv[4] + fasta_title = fasta_seq = '' + at_least_one = 0 + + out = open( outfile, 'w' ) - for i, line in enumerate( file( input_filename ) ): + for i, line in enumerate( file( infile ) ): line = line.rstrip( '\r\n' ) if not line or line.startswith( '#' ): continue + if line[0] == '>': - if len( tmp_seq ) > 0: - tmp_seq_count += 1 - seq_hash[ ( tmp_seq_count, tmp_title ) ] = tmp_seq - tmp_title = line - tmp_seq = '' + if len( fasta_seq ) > 0: + + if max_length <= 0: + compare_max_length = len( fasta_seq ) + 1 + else: + compare_max_length = max_length + + l = len( fasta_seq ) + + if l >= min_length and l <= compare_max_length: + at_least_one += 1 + out.write( "%s\n" % fasta_title ) + c = 0 + s = fasta_seq + while c < l: + b = min( c + 50, l ) + out.write( "%s\n" % s[ c:b ] ) + c = b + + fasta_title = line + fasta_seq = '' else: - tmp_seq = "%s%s" % ( tmp_seq, line ) - if line.split()[0].isdigit(): - tmp_seq = "%s " % tmp_seq - if len( tmp_seq ) > 0: - seq_hash[ ( tmp_seq_count, tmp_title ) ] = tmp_seq + fasta_seq = "%s%s" % ( fasta_seq, line ) - title_keys = seq_hash.keys() - title_keys.sort() - output_handle = open( output_filename, 'w' ) - at_least_one = 0 - for i, fasta_title in title_keys: - tmp_seq = seq_hash[ ( i, fasta_title ) ] + if len( fasta_seq ) > 0: + if max_length <= 0: - compare_max_length = len( tmp_seq ) + 1 + compare_max_length = len( fasta_seq ) + 1 else: compare_max_length = max_length - l = len( tmp_seq ) + + l = len( fasta_seq ) + if l >= min_length and l <= compare_max_length: at_least_one += 1 - output_handle.write( "%s\n" % fasta_title ) + out.write( "%s\n" % fasta_title ) c = 0 - s = tmp_seq + s = fasta_seq while c < l: b = min( c + 50, l ) - output_handle.write( "%s\n" % s[ c:b ] ) + out.write( "%s\n" % s[ c:b ] ) c = b - output_handle.close() + + out.close() if at_least_one == 0: print "There is no sequence that falls within your range." diff -r e76a153769d4 -r 6d849785ff86 tools/fasta_tools/fasta_to_tabular.py --- a/tools/fasta_tools/fasta_to_tabular.py Mon Jan 19 11:47:29 2009 -0500 +++ b/tools/fasta_tools/fasta_to_tabular.py Mon Jan 19 15:47:02 2009 -0500 @@ -1,9 +1,9 @@ #! /usr/bin/python # This code exists in 2 places: ~/datatypes/converters and ~/tools/fasta_tools """ -Input: fasta, minimal length, maximal length -Output: fasta -Return sequences whose lengths are within the range. +Input: fasta, int +Output: tabular +format convert: fasta to tabular """ import sys, os @@ -14,38 +14,31 @@ infile = sys.argv[1] outfile = sys.argv[2] keep_first = int( sys.argv[3] ) - title = '' - sequence = '' - sequence_count = 0 + fasta_title = fasta_seq = '' if keep_first == 0: keep_first = None else: keep_first += 1 + out = open( outfile, 'w' ) + for i, line in enumerate( open( infile ) ): line = line.rstrip( '\r\n' ) if not line or line.startswith( '#' ): continue if line.startswith( '>' ): - if sequence: - sequence_count += 1 - seq_hash[( sequence_count, title )] = sequence - title = line - sequence = '' + if fasta_seq: + out.write( "%s\t%s\n" %( fasta_title[ 1:keep_first ], fasta_seq ) ) + fasta_title = line + fasta_seq = '' else: - sequence = "%s%s" % ( sequence, line ) - if line.split() and line.split()[0].isdigit(): - sequence += ' ' - if sequence: - seq_hash[( sequence_count, title )] = sequence - # return only those lengths are in the range - title_keys = seq_hash.keys() - title_keys.sort() - out = open( outfile, 'w' ) - for i, fasta_title in title_keys: - sequence = seq_hash[( i, fasta_title )] - out.write( "%s\t%s\n" %( fasta_title[ 1:keep_first ], sequence ) ) + if line: + fasta_seq = "%s%s" % ( fasta_seq, line ) + + if fasta_seq: + out.write( "%s\t%s\n" %( fasta_title[ 1:keep_first ], fasta_seq ) ) + out.close() if __name__ == "__main__" : __main__() \ No newline at end of file

1 0

[hg] galaxy 1714: Fixes for Biomart and UCSC to better ensure th...
by Greg Von Kuster 22 Jan '09

22 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/dbb3210e4549 changeset: 1714:dbb3210e4549 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Jan 21 17:03:39 2009 -0500 description: Fixes for Biomart and UCSC to better ensure the data type is correctly set. A data type of "tabular" is now the default, and data will be sniffed if this is the data type. 6 file(s) affected in this change: lib/galaxy/tools/__init__.py tools/data_source/biomart.xml tools/data_source/biomart_test.xml tools/data_source/ucsc_tablebrowser.xml tools/data_source/ucsc_tablebrowser_archaea.xml tools/data_source/ucsc_tablebrowser_test.xml diffs (182 lines): diff -r 6d849785ff86 -r dbb3210e4549 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Mon Jan 19 15:47:02 2009 -0500 +++ b/lib/galaxy/tools/__init__.py Wed Jan 21 17:03:39 2009 -0500 @@ -1240,8 +1240,6 @@ def exec_before_job( self, app, inp_data, out_data, param_dict={} ): if self.tool_type == 'data_source': - # List for converting UCSC to Galaxy exts, if not in following dictionary, use provided datatype - data_type_to_ext = { 'wigdata':'wig', 'tab':'interval', 'hyperlinks':'html', 'sequence':'fasta' } dbkey = param_dict.get( 'dbkey' ) organism = param_dict.get( 'organism' ) table = param_dict.get( 'table' ) @@ -1263,12 +1261,9 @@ data.name = '%s on %s' % ( data.name, gb_landmark_region ) data.info = info data.dbkey = dbkey - try: - data_type = data_type_to_ext[ data_type ] - except: - pass - if data_type not in app.datatypes_registry.datatypes_by_extension: - data_type = 'interval' + if data_type not in app.datatypes_registry.datatypes_by_extension: + # Setting data_type to tabular will force the data to be sniffed in exec_after_process() + data_type = 'tabular' data = app.datatypes_registry.change_datatype( data, data_type ) # Store external data source's request parameters temporarily in output file. # In case the config setting for "outputs_to_working_directory" is True, we must write to @@ -1284,8 +1279,6 @@ return out_data def exec_after_process( self, app, inp_data, out_data, param_dict ): - # TODO: for data_source tools at least, this code can probably be handled more optimally by adding a new - # tag set in the tool config. if self.tool_type == 'data_source': name, data = out_data.items()[0] data.set_size() @@ -1294,9 +1287,10 @@ data.info = param_dict.get( 'info', data.name ) data.dbkey = param_dict.get( 'dbkey', data.dbkey ) data.extension = param_dict.get( 'data_type', data.extension ) - if data.extension == 'txt': + if data.extension in [ 'txt', 'tabular' ]: data_type = sniff.guess_ext( data.file_name, sniff_order=app.datatypes_registry.sniff_order ) - data = app.datatypes_registry.change_datatype( data, data_type ) + if data.extension != data_type: + data = app.datatypes_registry.change_datatype( data, data_type ) elif not isinstance( data.datatype, datatypes.interval.Bed ) and isinstance( data.datatype, datatypes.interval.Interval ): data.set_meta() if data.missing_meta(): diff -r 6d849785ff86 -r dbb3210e4549 tools/data_source/biomart.xml --- a/tools/data_source/biomart.xml Mon Jan 19 15:47:02 2009 -0500 +++ b/tools/data_source/biomart.xml Wed Jan 21 17:03:39 2009 -0500 @@ -21,17 +21,21 @@ <param_from_source name="GALAXY_URL" missing="0" /> </add_to_url> </request_param> + <request_param galaxy_name="data_type" remote_name="exportView_outputformat" missing="tabular" > + <data_type_translation> + <format galaxy_format="tabular" remote_format="TSV" /> + </data_type_translation> + </request_param> <request_param galaxy_name="dbkey" remote_name="dbkey" missing="?" /> <request_param galaxy_name="organism" remote_name="organism" missing="" /> <request_param galaxy_name="table" remote_name="table" missing="" /> <request_param galaxy_name="description" remote_name="description" missing="" /> <request_param galaxy_name="name" remote_name="name" missing="Biomart query" /> <request_param galaxy_name="info" remote_name="info" missing="" /> - <request_param galaxy_name="data_type" remote_name="type" missing="txt" /> </request_param_translation> <uihints minwidth="800"/> <outputs> - <data name="output" format="txt" /> + <data name="output" format="tabular" /> </outputs> <options sanitize="False" refresh="True"/> </tool> diff -r 6d849785ff86 -r dbb3210e4549 tools/data_source/biomart_test.xml --- a/tools/data_source/biomart_test.xml Mon Jan 19 15:47:02 2009 -0500 +++ b/tools/data_source/biomart_test.xml Wed Jan 21 17:03:39 2009 -0500 @@ -21,17 +21,21 @@ <param_from_source name="GALAXY_URL" missing="0" /> </add_to_url> </request_param> + <request_param galaxy_name="data_type" remote_name="exportView_outputformat" missing="tabular" > + <data_type_translation> + <format galaxy_format="tabular" remote_format="TSV" /> + </data_type_translation> + </request_param> <request_param galaxy_name="dbkey" remote_name="dbkey" missing="?" /> <request_param galaxy_name="organism" remote_name="organism" missing="" /> <request_param galaxy_name="table" remote_name="table" missing="" /> <request_param galaxy_name="description" remote_name="description" missing="" /> <request_param galaxy_name="name" remote_name="name" missing="Biomart test query" /> <request_param galaxy_name="info" remote_name="info" missing="" /> - <request_param galaxy_name="data_type" remote_name="type" missing="txt" /> </request_param_translation> <uihints minwidth="800"/> <outputs> - <data name="output" format="txt" /> + <data name="output" format="tabular" /> </outputs> <options sanitize="False" refresh="True"/> </tool> diff -r 6d849785ff86 -r dbb3210e4549 tools/data_source/ucsc_tablebrowser.xml --- a/tools/data_source/ucsc_tablebrowser.xml Mon Jan 19 15:47:02 2009 -0500 +++ b/tools/data_source/ucsc_tablebrowser.xml Wed Jan 21 17:03:39 2009 -0500 @@ -21,16 +21,20 @@ <request_param galaxy_name="organism" remote_name="org" missing="unknown species" /> <request_param galaxy_name="table" remote_name="hgta_track" missing="unknown table" /> <request_param galaxy_name="description" remote_name="hgta_regionType" missing="no description" /> - <request_param galaxy_name="data_type" remote_name="hgta_outputType" missing="bed" > + <request_param galaxy_name="data_type" remote_name="hgta_outputType" missing="tabular" > <data_type_translation> <format galaxy_format="tabular" remote_format="primaryTable" /> <format galaxy_format="tabular" remote_format="selectedFields" /> + <format galaxy_format="wig" remote_format="wigdata" /> + <format galaxy_format="interval" remote_format="tab" /> + <format galaxy_format="html" remote_format="hyperlinks" /> + <format galaxy_format="fasta" remote_format="sequence" /> </data_type_translation> </request_param> </request_param_translation> <uihints minwidth="800"/> <outputs> - <data name="output" format="bed" /> + <data name="output" format="tabular" /> </outputs> <options sanitize="False" refresh="True"/> </tool> diff -r 6d849785ff86 -r dbb3210e4549 tools/data_source/ucsc_tablebrowser_archaea.xml --- a/tools/data_source/ucsc_tablebrowser_archaea.xml Mon Jan 19 15:47:02 2009 -0500 +++ b/tools/data_source/ucsc_tablebrowser_archaea.xml Wed Jan 21 17:03:39 2009 -0500 @@ -21,16 +21,20 @@ <request_param galaxy_name="organism" remote_name="org" missing="unknown species" /> <request_param galaxy_name="table" remote_name="hgta_track" missing="" /> <request_param galaxy_name="description" remote_name="hgta_regionType" missing="" /> - <request_param galaxy_name="data_type" remote_name="hgta_outputType" missing="bed" > + <request_param galaxy_name="data_type" remote_name="hgta_outputType" missing="tabular" > <data_type_translation> <format galaxy_format="tabular" remote_format="primaryTable" /> <format galaxy_format="tabular" remote_format="selectedFields" /> + <format galaxy_format="wig" remote_format="wigdata" /> + <format galaxy_format="interval" remote_format="tab" /> + <format galaxy_format="html" remote_format="hyperlinks" /> + <format galaxy_format="fasta" remote_format="sequence" /> </data_type_translation> </request_param> </request_param_translation> <uihints minwidth="800"/> <outputs> - <data name="output" format="bed" /> + <data name="output" format="tabular" /> </outputs> <options sanitize="False" refresh="True"/> </tool> diff -r 6d849785ff86 -r dbb3210e4549 tools/data_source/ucsc_tablebrowser_test.xml --- a/tools/data_source/ucsc_tablebrowser_test.xml Mon Jan 19 15:47:02 2009 -0500 +++ b/tools/data_source/ucsc_tablebrowser_test.xml Wed Jan 21 17:03:39 2009 -0500 @@ -21,16 +21,20 @@ <request_param galaxy_name="organism" remote_name="org" missing="unknown species" /> <request_param galaxy_name="table" remote_name="hgta_track" missing="" /> <request_param galaxy_name="description" remote_name="hgta_regionType" missing="" /> - <request_param galaxy_name="data_type" remote_name="hgta_outputType" missing="bed" > + <request_param galaxy_name="data_type" remote_name="hgta_outputType" missing="tabular" > <data_type_translation> <format galaxy_format="tabular" remote_format="primaryTable" /> <format galaxy_format="tabular" remote_format="selectedFields" /> + <format galaxy_format="wig" remote_format="wigdata" /> + <format galaxy_format="interval" remote_format="tab" /> + <format galaxy_format="html" remote_format="hyperlinks" /> + <format galaxy_format="fasta" remote_format="sequence" /> </data_type_translation> </request_param> </request_param_translation> <uihints minwidth="800"/> <outputs> - <data name="output" format="bed" /> + <data name="output" format="tabular" /> </outputs> <options sanitize="False" refresh="True"/> </tool>

1 0

[hg] galaxy 1715: Fixes for formatting end of line chars in html...
by Greg Von Kuster 22 Jan '09

22 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/5c293dbf0726 changeset: 1715:5c293dbf0726 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Thu Jan 22 08:50:52 2009 -0500 description: Fixes for formatting end of line chars in html when displaying dataset.info. Also change the biomart tool version. 3 file(s) affected in this change: lib/galaxy/datatypes/data.py tools/data_source/biomart.xml tools/data_source/biomart_test.xml diffs (62 lines): diff -r dbb3210e4549 -r 5c293dbf0726 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py Wed Jan 21 17:03:39 2009 -0500 +++ b/lib/galaxy/datatypes/data.py Thu Jan 22 08:50:52 2009 -0500 @@ -128,16 +128,22 @@ out = "Can't create peek %s" % str( exc ) return out def display_name(self, dataset): - """Returns formated html of dataset name""" + """Returns formatted html of dataset name""" try: return escape(dataset.name) except: return "name unavailable" def display_info(self, dataset): - """Returns formated html of dataset info""" + """Returns formatted html of dataset info""" try: # Change new line chars to html - return escape( dataset.info ).replace( "\r", "\n" ).replace( "\n", "<br>" ) + if dataset.info.find( '\r\n' ) >= 0: + dataset.info = dataset.info.replace( '\r\n', '<br/>' ) + if dataset.info.find( '\r' ) >= 0: + dataset.info = dataset.info.replace( '\r', '<br/>' ) + if dataset.info.find( '\n' ) >= 0: + dataset.info = dataset.info.replace( '\n', '<br/>' ) + return escape( dataset.info ) except: return "info unavailable" def validate(self, dataset): @@ -154,7 +160,7 @@ Adds a display app to the datatype. app_id is a unique id label is the primary display label, ie display at 'UCSC' - file_function is a string containing the name of the function that returns a properly formated display + file_function is a string containing the name of the function that returns a properly formatted display links_function is a string containing the name of the function that returns a list of (link_name,link) """ self.supported_display_apps = self.supported_display_apps.copy() diff -r dbb3210e4549 -r 5c293dbf0726 tools/data_source/biomart.xml --- a/tools/data_source/biomart.xml Wed Jan 21 17:03:39 2009 -0500 +++ b/tools/data_source/biomart.xml Thu Jan 22 08:50:52 2009 -0500 @@ -7,7 +7,7 @@ TODO: Hack to get biomart to work - the 'add_to_URL' param can be eliminated when the Biomart team encodes URL prior to sending, meanwhile everything including and beyond the first '&' is truncated from URL. They said they'll let us know when this is fixed at their end. --> -<tool name="BioMart" id="biomart" tool_type="data_source" URL_method="get"> +<tool name="BioMart" id="biomart" tool_type="data_source" URL_method="get" version="1.0.1"> <description>Central server</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://www.biomart.org/biomart/martview" check_values="false" method="get" target="_top"> diff -r dbb3210e4549 -r 5c293dbf0726 tools/data_source/biomart_test.xml --- a/tools/data_source/biomart_test.xml Wed Jan 21 17:03:39 2009 -0500 +++ b/tools/data_source/biomart_test.xml Thu Jan 22 08:50:52 2009 -0500 @@ -7,7 +7,7 @@ TODO: Hack to get biomart to work - the 'add_to_URL' param can be eliminated when the Biomart team encodes URL prior to sending, meanwhile everything including and beyond the first '&' is truncated from URL. They said they'll let us know when this is fixed at their end. --> -<tool name="BioMart" id="biomart_test" tool_type="data_source" URL_method="get"> +<tool name="BioMart" id="biomart_test" tool_type="data_source" URL_method="get" version="1.0.1"> <description>Test server</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://test.biomart.org/biomart/martview" check_values="false" method="get" target="_top">

1 0

[hg] galaxy 1710: Eliminate the use of the "order=" for the snif...
by Nate Coraor 19 Jan '09

19 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/c565de071f7f changeset: 1710:c565de071f7f user: Greg Von Kuster <greg(a)bx.psu.edu> date: Thu Jan 15 14:23:53 2009 -0500 description: Eliminate the use of the "order=" for the sniffers in datatypes.conf - since we've moved from using an ini file to an xml file, ordering is implicit. 2 file(s) affected in this change: datatypes_conf.xml.sample lib/galaxy/datatypes/registry.py diffs (74 lines): diff -r 90d04fcbcdf4 -r c565de071f7f datatypes_conf.xml.sample --- a/datatypes_conf.xml.sample Thu Jan 15 13:10:04 2009 -0500 +++ b/datatypes_conf.xml.sample Thu Jan 15 14:23:53 2009 -0500 @@ -146,20 +146,22 @@  - <sniffer order="005" type="galaxy.datatypes.xml:BlastXml"/> - <sniffer order="010" type="galaxy.datatypes.sequence:Maf"/> - <sniffer order="015" type="galaxy.datatypes.sequence:Lav"/> - <sniffer order="020" type="galaxy.datatypes.sequence:Fasta"/> - <sniffer order="025" type="galaxy.datatypes.sequence:FastqSolexa"/> - <sniffer order="030" type="galaxy.datatypes.interval:Wiggle"/> - <sniffer order="035" type="galaxy.datatypes.images:Html"/> - <sniffer order="040" type="galaxy.datatypes.sequence:Axt"/> - <sniffer order="045" type="galaxy.datatypes.interval:Bed"/> - <sniffer order="050" type="galaxy.datatypes.interval:CustomTrack"/> - <sniffer order="055" type="galaxy.datatypes.interval:Gff"/> - <sniffer order="060" type="galaxy.datatypes.interval:Gff3"/> - <sniffer order="065" type="galaxy.datatypes.interval:Interval"/> + <sniffer type="galaxy.datatypes.xml:BlastXml"/> + <sniffer type="galaxy.datatypes.sequence:Maf"/> + <sniffer type="galaxy.datatypes.sequence:Lav"/> + <sniffer type="galaxy.datatypes.sequence:Fasta"/> + <sniffer type="galaxy.datatypes.sequence:FastqSolexa"/> + <sniffer type="galaxy.datatypes.interval:Wiggle"/> + <sniffer type="galaxy.datatypes.images:Html"/> + <sniffer type="galaxy.datatypes.sequence:Axt"/> + <sniffer type="galaxy.datatypes.interval:Bed"/> + <sniffer type="galaxy.datatypes.interval:CustomTrack"/> + <sniffer type="galaxy.datatypes.interval:Gff"/> + <sniffer type="galaxy.datatypes.interval:Gff3"/> + <sniffer type="galaxy.datatypes.interval:Interval"/> </sniffers> </datatypes> diff -r 90d04fcbcdf4 -r c565de071f7f lib/galaxy/datatypes/registry.py --- a/lib/galaxy/datatypes/registry.py Thu Jan 15 13:10:04 2009 -0500 +++ b/lib/galaxy/datatypes/registry.py Thu Jan 15 14:23:53 2009 -0500 @@ -59,18 +59,15 @@ self.converters.append( ( converter_config, extension, target_datatype ) ) except Exception, e: self.log.warning( 'Error loading datatype "%s", problem: %s' % ( extension, str( e ) ) ) - # Load datatype sniffers from config + # Load datatype sniffers from the config sniff_order = [] sniffers = root.find( 'sniffers' ) for elem in sniffers.findall( 'sniffer' ): - order = elem.get( 'order', None ) type = elem.get( 'type', None ) - if order and type: - sniff_order.append( ( order, type ) ) - sniff_order.sort() - for ele in sniff_order: + if type: + sniff_order.append( type ) + for type in sniff_order: try: - type = ele[1] fields = type.split( ":" ) datatype_module = fields[0] datatype_class = fields[1] @@ -147,6 +144,7 @@ sequence.Maf(), sequence.Lav(), sequence.Fasta(), + sequence.FastqSolexa(), interval.Wiggle(), images.Html(), sequence.Axt(),

1 0

[hg] galaxy 1712: Flush newly created users directly in __get_or...
by Nate Coraor 19 Jan '09

19 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/e76a153769d4 changeset: 1712:e76a153769d4 user: Nate Coraor <nate(a)bx.psu.edu> date: Mon Jan 19 11:47:29 2009 -0500 description: Flush newly created users directly in __get_or_create_remote_user, solves the new user creation when using remote_user bug found by Ross Lazarus. Also, external users get a random password now (makes the secret key safer if a bad guy gets the database). 1 file(s) affected in this change: lib/galaxy/web/framework/__init__.py diffs (41 lines): diff -r 2a36ccdb2a38 -r e76a153769d4 lib/galaxy/web/framework/__init__.py --- a/lib/galaxy/web/framework/__init__.py Fri Jan 16 13:30:01 2009 -0500 +++ b/lib/galaxy/web/framework/__init__.py Mon Jan 19 11:47:29 2009 -0500 @@ -4,7 +4,7 @@ import pkg_resources -import os, sys, time +import os, sys, time, random, string pkg_resources.require( "Cheetah" ) from Cheetah.Template import Template import base @@ -229,6 +229,9 @@ user_for_new_session = self.__get_or_create_remote_user( remote_user_email ) log.warning( "User logged in as '%s' externally, but has a cookie as '%s' invalidating session", remote_user_email, prev_galaxy_session.user.email ) + else: + # No session exists, get/create user for new session + user_for_new_session = self.__get_or_create_remote_user( remote_user_email ) else: if galaxy_session is not None and galaxy_session.user and galaxy_session.user.external: # Remote user support is not enabled, but there is an existing @@ -282,15 +285,15 @@ def __get_or_create_remote_user( self, remote_user_email ): """ Return the user in $HTTP_REMOTE_USER and create if necessary - - Caller is responsible for flushing the returned user. """ # remote_user middleware ensures HTTP_REMOTE_USER exists user = self.app.model.User.filter_by( email=remote_user_email ).first() if user is None: + random.seed() user = self.app.model.User( email=remote_user_email ) - user.set_password_cleartext( 'external' ) + user.set_password_cleartext( ''.join( random.sample( string.letters + string.digits, 12 ) ) ) user.external = True + user.flush() #self.log_event( "Automatically created account '%s'", user.email ) return user def __update_session_cookie( self, name='galaxysession' ):

1 0

[hg] galaxy 1711: Removed an option from grouping tool, which is...
by Nate Coraor 19 Jan '09

19 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/2a36ccdb2a38 changeset: 1711:2a36ccdb2a38 user: guru date: Fri Jan 16 13:30:01 2009 -0500 description: Removed an option from grouping tool, which is now part of the LCA tool under Taxonomy. 2 file(s) affected in this change: tools/stats/grouping.py tools/stats/grouping.xml diffs (354 lines): diff -r c565de071f7f -r 2a36ccdb2a38 tools/stats/grouping.py --- a/tools/stats/grouping.py Thu Jan 15 14:23:53 2009 -0500 +++ b/tools/stats/grouping.py Fri Jan 16 13:30:01 2009 -0500 @@ -12,15 +12,13 @@ def main(): inputfile = sys.argv[2] - in_columns = int( sys.argv[5] ) - show_remaining_cols = sys.argv[4] ops = [] cols = [] rounds = [] elems = [] - for var in sys.argv[6:]: + for var in sys.argv[4:]: ops.append(var.split()[0]) cols.append(var.split()[1]) rounds.append(var.split()[2]) @@ -81,19 +79,9 @@ if error_code != 0: stop_err( "Sorting input dataset resulted in error: %s: %s" %( error_code, stdout )) - - if show_remaining_cols == 'yes': - show_cols_list = [1]*in_columns - show_cols_list[group_col] = 0 - for c in cols: - c = int(c)-1 - show_cols_list[c] = 0 - #at the end of this, only the indices of the remaining columns will be set to 1 - remaining_cols = [j for j,k in enumerate(show_cols_list) if k==1] #this is the list of remaining column indices - + prev_item = "" prev_vals = [] - remaining_vals = [] skipped_lines = 0 first_invalid_line = 0 invalid_line = '' @@ -128,30 +116,22 @@ invalid_column = col+1 if valid: prev_vals[i].append(fields[col].strip()) - #Store values from all the remaning columns - if show_remaining_cols == 'yes': - for j, index in enumerate(remaining_cols): - remaining_vals[j].append(fields[index].strip()) else: """ When a new value is encountered, write the previous value and the corresponding aggregate values into the output file. This works due to the sort on group_col we've applied to the data above. """ - out_list = ['']*in_columns - out_list[group_col] = str(prev_item) - + out_str = prev_item + for i, op in enumerate( ops ): rfunc = "r." + op if op not in ['c','length','unique','random']: for j, elem in enumerate( prev_vals[i] ): prev_vals[i][j] = float( elem ) + rout = "%g" %( eval( rfunc )( prev_vals[i] )) if rounds[i] == 'yes': - rout = "%f" %( eval( rfunc )( prev_vals[i] )) rout = int(round(float(rout))) - else: - rout = "%g" %( eval( rfunc )( prev_vals[i] )) - else: if op != 'random': rout = eval( rfunc )( prev_vals[i] ) @@ -162,21 +142,9 @@ if op == 'unique': rfunc = "r.length" rout = eval( rfunc )( rout ) - - out_list[int(cols[i])-1] = str(rout) - - if show_remaining_cols == 'yes': - for index,el in enumerate(remaining_cols): - if index == 0: - try: - random_index = random.randint(0,len(remaining_vals[index])-1) - except: - random_index = 0 - #pick a random value from each of the remaning columns - rand_out = remaining_vals[index][random_index] - out_list[el] = str(rand_out) - - print >>fout, '\t'.join([elem for elem in out_list if elem != '']) + out_str += "\t" + str(rout) + + print >>fout, out_str prev_item = item prev_vals = [] @@ -185,14 +153,6 @@ val_list = [] val_list.append(fields[col].strip()) prev_vals.append(val_list) - - if show_remaining_cols == 'yes': - remaining_vals = [] - for index in remaining_cols: - remaining_val_list = [] - remaining_val_list.append(fields[index].strip()) - remaining_vals.append(remaining_val_list) - else: # This only occurs once, right at the start of the iteration. prev_item = item @@ -201,15 +161,8 @@ val_list = [] val_list.append(fields[col].strip()) prev_vals.append(val_list) - - if show_remaining_cols == 'yes': - remaining_vals = [] - for index in remaining_cols: - remaining_val_list = [] - remaining_val_list.append(fields[index].strip()) - remaining_vals.append(remaining_val_list) - except Exception: + except Exception, exc: skipped_lines += 1 if not first_invalid_line: first_invalid_line = ii+1 @@ -219,8 +172,7 @@ first_invalid_line = ii+1 # Handle the last grouped value - out_list = ['']*in_columns - out_list[group_col] = str(prev_item) + out_str = prev_item for i, op in enumerate(ops): rfunc = "r." + op @@ -228,11 +180,9 @@ if op not in ['c','length','unique','random']: for j, elem in enumerate( prev_vals[i] ): prev_vals[i][j] = float( elem ) + rout = '%g' %( eval( rfunc )( prev_vals[i] )) if rounds[i] == 'yes': - rout = '%f' %( eval( rfunc )( prev_vals[i] )) rout = int(round(float(rout))) - else: - rout = '%g' %( eval( rfunc )( prev_vals[i] )) else: if op != 'random': rout = eval( rfunc )( prev_vals[i] ) @@ -243,22 +193,13 @@ if op == 'unique': rfunc = "r.length" rout = eval( rfunc )( rout ) - out_list[int(cols[i])-1] = str(rout) + out_str += "\t" + str( rout ) except: skipped_lines += 1 if not first_invalid_line: first_invalid_line = ii+1 - if show_remaining_cols == 'yes': - for index,el in enumerate(remaining_cols): - if index == 0: - try: - random_index = random.randint(0,len(remaining_vals[index])-1) - except: - random_index = 0 - rand_out = remaining_vals[index][random_index] - out_list[el] = str(rand_out) - print >>fout, '\t'.join([elem for elem in out_list if elem != '']) + print >>fout, out_str # Generate a useful info message. msg = "--Group by c%d: " %(group_col+1) diff -r c565de071f7f -r 2a36ccdb2a38 tools/stats/grouping.xml --- a/tools/stats/grouping.xml Thu Jan 15 14:23:53 2009 -0500 +++ b/tools/stats/grouping.xml Fri Jan 16 13:30:01 2009 -0500 @@ -1,49 +1,43 @@ -<tool id="Grouping1" name="Group" version="1.5.0"> - <description>data by a column and perform aggregate operation on other columns.</description> - <command interpreter="python"> - grouping.py - $out_file1 - $input1 +<tool id="Grouping1" name="Group" version="1.4.0"> + <description>data by a column and perform aggregate operation on other columns.</description> + <command interpreter="python"> + grouping.py + $out_file1 + $input1 $groupcol - $othercols - ${input1.metadata.columns} - #for $op in $operations - '${op.optype} - ${op.opcol} - ${op.opround}' - #end for - </command> - <inputs> - <param format="tabular" name="input1" type="data" label="Select data" help="Query missing? See TIP below."/> - <param name="groupcol" label="Group by column" type="data_column" data_ref="input1" /> - <repeat name="operations" title="Operation"> - <param name="optype" type="select" label="Type"> - <option value="mean">Mean</option> - <option value="max">Maximum</option> - <option value="min">Minimum</option> - <option value="sum">Sum</option> - <option value="length">Count</option> - <option value="unique">Count Distinct</option> + #for $op in $operations + '${op.optype} + ${op.opcol} + ${op.opround}' + #end for + </command> + <inputs> + <param format="tabular" name="input1" type="data" label="Select data" help="Query missing? See TIP below."/> + <param name="groupcol" label="Group by column" type="data_column" data_ref="input1" /> + <repeat name="operations" title="Operation"> + <param name="optype" type="select" label="Type"> + <option value="mean">Mean</option> + <option value="max">Maximum</option> + <option value="min">Minimum</option> + <option value="sum">Sum</option> + <option value="length">Count</option> + <option value="unique">Count Distinct</option> <option value="c">Concatenate</option> - <option value="random">Randomly pick</option> - </param> - <param name="opcol" label="On column" type="data_column" data_ref="input1" /> - <param name="opround" type="select" label="Round result to nearest integer?"> - <option value="no">NO</option> - <option value="yes">YES</option> - </param> - </repeat> - <param name="othercols" type="select" label="Randomly pick an entry from each of the remaining columns (besides the columns chosen for group and aggregate operations above) ?"> + <option value="random">Randomly pick</option> + </param> + <param name="opcol" label="On column" type="data_column" data_ref="input1" /> + <param name="opround" type="select" label="Round result to nearest integer?"> <option value="no">NO</option> <option value="yes">YES</option> - </param> - </inputs> - <outputs> - <data format="input" name="out_file1" metadata_source="input1" /> - </outputs> - <requirements> - <requirement type="python-module">rpy</requirement> - </requirements> + </param> + </repeat> + </inputs> + <outputs> + <data format="input" name="out_file1" metadata_source="input1" /> + </outputs> + <requirements> + <requirement type="python-module">rpy</requirement> + </requirements> <tests>  <test> @@ -52,9 +46,9 @@ <param name="optype" value="mean"/> <param name="opcol" value="2"/> <param name="opround" value="no"/> - <param name="othercols" value="no"/> <output name="out_file1" file="groupby_out1.dat"/> </test> +  <test> <param name="input1" value="1.tabular"/> @@ -62,39 +56,38 @@ <param name="optype" value="mean"/> <param name="opcol" value="2"/> <param name="opround" value="no"/> - <param name="othercols" value="no"/> <output name="out_file1" file="groupby_out2.dat"/> - </test> - </tests> - <help> - -.. class:: infomark - -**TIP:** If your data is not TAB delimited, use *Text Manipulation->Convert* - ------ - -**Syntax** - -This tool allows you to group the input dataset by a particular column and perform aggregate functions like Mean, Sum, Max, Min and Concatenate on other columns. - -- All invalid, blank and comment lines are skipped when performing the aggregate functions. The number of skipped lines is displayed in the resulting history item. - ------ - -**Example** - -- For the following input:: - - chr22 1000 NM_17 - chr22 2000 NM_18 - chr10 2200 NM_10 - chr10 1200 NM_11 - chr22 1600 NM_19 - -- running this tool with **Group by column 1**, Operations **Mean on column 2** and **Concatenate on column 3** will return:: - - chr10 1700.00 ['NM_11', 'NM_10'] - chr22 1533.33 ['NM_17', 'NM_19', 'NM_18'] - </help> -</tool> + </test> + </tests> + <help> + +.. class:: infomark + +**TIP:** If your data is not TAB delimited, use *Text Manipulation->Convert* + +----- + +**Syntax** + +This tool allows you to group the input dataset by a particular column and perform aggregate functions like Mean, Sum, Max, Min and Concatenate on other columns. + +- All invalid, blank and comment lines are skipped when performing the aggregate functions. The number of skipped lines is displayed in the resulting history item. + +----- + +**Example** + +- For the following input:: + + chr22 1000 NM_17 + chr22 2000 NM_18 + chr10 2200 NM_10 + chr10 1200 NM_11 + chr22 1600 NM_19 + +- running this tool with **Group by column 1**, Operations **Mean on column 2** and **Concatenate on column 3** will return:: + + chr10 1700.00 ['NM_11', 'NM_10'] + chr22 1533.33 ['NM_17', 'NM_19', 'NM_18'] + </help> +</tool>

1 0