April 2009 - galaxy-dev - lists.galaxyproject.org

How to make Galaxy link to a local installed UCSC database?
by xue dong 11 Jun '09

11 Jun '09

Hallo every one, I am new to Galaxy, I have a idea which maybe not new to people here. I work on arabidopsis. maybe you know that UCSC do not offer the genome assembly for plant. so i will install UCSC locally and built the genome assembly for arabidopsis. this work is half done now. I also installed Galaxy, if i understand correctly, Galaxy has the link to UCSC so the user can get data from UCSC. since what i interested is arabidopsis information, I hope Galaxy can retrive data form the locally installed UCSC in future. I do not know if it is possible to make this idea come ture.maybe some of you have done it. if yes, I am gratefully if you can tell me the how to get it. many thanks, Xue

2 1

Round-Robin Scheduling
by Assaf Gordon 11 Jun '09

11 Jun '09

Hello, I have another question regarding the local job scheduler: Is it possible to limit the number of jobs *per user* ? That is - any given user can have up to X number of jobs running concurrently, regardless of the value of local_job_queue_workers ? Imagine the following situation: local_job_queue_worker = 5 job_scheduler_policy = galaxy.jobs.schedulingpolicy.roundrobin:UserRoundRobin Which means that at any given moment, galaxy can run only five jobs. Now, Galaxy is completely Idle, no jobs are running. One users starts 7 very long running jobs (each jobs will take about two hours). If I understand correctly - since no jobs are running, 5 of the user's job will be started immediately, even with the round-robin policy, right ? And this means that for the next two hours, every other user which starts a job - his/her job will be either new or limbo-running, but none will actually be started, right ? I think I'm experiencing this situation on my galaxy server. Users are complaining their jobs have been running 'forever' or not even starting for a long long time. Close examination shows that there are running 5 jobs (all from the same user) which have been running for three hours, and they are kind of hogging all the worker threads. To make a long story short - I would like to make sure a single user can't hog Galaxy. Is it possible with the local job runner, and if not - is it possible with the SGE job runner ? Thanks for reading so far, Gordon.

2 1

[hg] galaxy 2390: Reintroduce the ability to copy a history item...
by Greg Von Kuster 01 May '09

01 May '09

details: http://www.bx.psu.edu/hg/galaxy/rev/ee9a29c9efc1 changeset: 2390:ee9a29c9efc1 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Thu Apr 30 16:12:59 2009 -0400 description: Reintroduce the ability to copy a history item between histories (and within the same). 3 file(s) affected in this change: lib/galaxy/web/controllers/dataset.py templates/dataset/copy_view.mako templates/dataset/edit_attributes.mako diffs (53 lines): diff -r 54be726b7711 -r ee9a29c9efc1 lib/galaxy/web/controllers/dataset.py --- a/lib/galaxy/web/controllers/dataset.py Thu Apr 30 15:37:37 2009 -0400 +++ b/lib/galaxy/web/controllers/dataset.py Thu Apr 30 16:12:59 2009 -0400 @@ -157,6 +157,7 @@ user = trans.get_user() history = trans.get_history() create_new_history = False + refresh_frames = [] if source_dataset_ids: if not isinstance( source_dataset_ids, list ): source_dataset_ids = source_dataset_ids.split( "," ) @@ -203,6 +204,8 @@ else: for hist in target_histories: hist.add_dataset( data.copy( copy_children = True ) ) + if history in target_histories: + refresh_frames = ['history'] trans.app.model.flush() done_msg = "%i datasets copied to %i histories." % ( len( source_dataset_ids ) - invalid_datasets, len( target_histories ) ) history.refresh() @@ -220,4 +223,5 @@ target_histories = target_histories, new_history_name = new_history_name, done_msg = done_msg, - error_msg = error_msg ) + error_msg = error_msg, + refresh_frames = refresh_frames ) diff -r 54be726b7711 -r ee9a29c9efc1 templates/dataset/copy_view.mako --- a/templates/dataset/copy_view.mako Thu Apr 30 15:37:37 2009 -0400 +++ b/templates/dataset/copy_view.mako Thu Apr 30 16:12:59 2009 -0400 @@ -1,5 +1,8 @@ <%inherit file="/base.mako"/> +<%namespace file="/message.mako" import="javascripts" /> <%def name="title()">Copy History Items</%def> + +${javascripts()} %if error_msg: <p> diff -r 54be726b7711 -r ee9a29c9efc1 templates/dataset/edit_attributes.mako --- a/templates/dataset/edit_attributes.mako Thu Apr 30 15:37:37 2009 -0400 +++ b/templates/dataset/edit_attributes.mako Thu Apr 30 16:12:59 2009 -0400 @@ -155,3 +155,10 @@ </div> </div> %endif +<p/> +<div class="toolForm"> +<div class="toolFormTitle">Copy History Item</div> +<div class="toolFormBody"> + Click <a href="${h.url_for( controller='dataset', action='copy_datasets', source_dataset_ids=data.id, target_history_ids=data.history_id )}" target="galaxy_main">here</a> to make a copy of this history item. +</div> +</div>

1 0

[hg] galaxy 2389: Fix for HID incrementing by two when importing...
by Greg Von Kuster 01 May '09

01 May '09

details: http://www.bx.psu.edu/hg/galaxy/rev/54be726b7711 changeset: 2389:54be726b7711 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Thu Apr 30 15:37:37 2009 -0400 description: Fix for HID incrementing by two when importing from a library. 2 file(s) affected in this change: lib/galaxy/model/__init__.py lib/galaxy/web/controllers/library.py diffs (44 lines): diff -r ff2fca65799e -r 54be726b7711 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py Thu Apr 30 11:07:34 2009 -0400 +++ b/lib/galaxy/model/__init__.py Thu Apr 30 15:37:37 2009 -0400 @@ -777,8 +777,7 @@ self.copied_from_library_dataset_dataset_association = copied_from_library_dataset_dataset_association self.library_dataset = library_dataset self.user = user - def to_history_dataset_association( self, target_history, parent_id=None ): - hid = target_history._next_hid() + def to_history_dataset_association( self, target_history, parent_id = None, add_to_history = False ): hda = HistoryDatasetAssociation( name=self.name, info=self.info, blurb=self.blurb, @@ -790,12 +789,13 @@ deleted=self.deleted, parent_id=parent_id, copied_from_library_dataset_dataset_association=self, - history=target_history, - hid=hid ) + history=target_history ) hda.flush() hda.metadata = self.metadata #need to set after flushed, as MetadataFiles require dataset.id + if add_to_history and target_history: + target_history.add_dataset( hda ) for child in self.children: - child_copy = child.to_history_dataset_association( target_history=target_history, parent_id=hda.id ) + child_copy = child.to_history_dataset_association( target_history = target_history, parent_id = hda.id, add_to_history = False ) if not self.datatype.copy_safe_peek: hda.set_peek() #in some instances peek relies on dataset_id, i.e. gmaj.zip for viewing MAFs hda.flush() diff -r ff2fca65799e -r 54be726b7711 lib/galaxy/web/controllers/library.py --- a/lib/galaxy/web/controllers/library.py Thu Apr 30 11:07:34 2009 -0400 +++ b/lib/galaxy/web/controllers/library.py Thu Apr 30 15:37:37 2009 -0400 @@ -175,9 +175,7 @@ history = trans.get_history() for ldda_id in ldda_ids: ldda = trans.app.model.LibraryDatasetDatasetAssociation.get( ldda_id ) - hda = ldda.to_history_dataset_association( target_history=history ) - history.add_dataset( hda ) - hda.flush() + hda = ldda.to_history_dataset_association( target_history=history, add_to_history = True ) history.flush() msg = "%i dataset(s) have been imported into your history" % len( ldda_ids ) return trans.response.send_redirect( web.url_for( controller='library',

1 0

[hg] galaxy 2387: Added test datasets for qualityscore unit tests.
by Greg Von Kuster 01 May '09

01 May '09

details: http://www.bx.psu.edu/hg/galaxy/rev/609ee7376d0d changeset: 2387:609ee7376d0d user: guru date: Thu Apr 30 10:54:43 2009 -0400 description: Added test datasets for qualityscore unit tests. 2 file(s) affected in this change: lib/galaxy/datatypes/test/sequence.qual454 lib/galaxy/datatypes/test/sequence.qualsolid diffs (38 lines): diff -r 799a333342c4 -r 609ee7376d0d lib/galaxy/datatypes/test/sequence.qual454 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/galaxy/datatypes/test/sequence.qual454 Thu Apr 30 10:54:43 2009 -0400 @@ -0,0 +1,16 @@ +>EYKX4VC04IWAEA length=68 xy=3531_0528 region=4 run=R_2007_11_07_16_15_57_ +22 13 9 6 4 3 2 2 1 1 1 1 24 44 33 23 16 11 7 2 28 33 23 18 28 27 27 28 20 21 42 35 21 6 24 25 31 21 28 27 41 34 15 28 28 27 28 28 33 24 27 28 28 24 27 36 27 28 28 28 +28 28 36 30 8 34 25 18 +>EYKX4VC04JKOGH length=48 xy=3808_3903 region=4 run=R_2007_11_07_16_15_57_ +28 28 27 28 38 31 10 28 28 27 27 34 25 28 24 26 27 28 27 37 29 34 25 31 21 28 21 36 28 31 20 24 27 37 28 28 34 27 3 34 25 24 28 28 26 28 35 28 +>EYKX4VC04JIUVK length=84 xy=3788_0830 region=4 run=R_2007_11_07_16_15_57_ +29 20 14 11 8 6 3 1 25 27 25 28 28 27 28 27 28 28 36 28 27 28 36 29 7 28 28 28 27 27 27 35 26 35 26 27 36 28 28 28 38 32 11 28 36 28 27 26 35 25 28 38 31 11 27 28 37 28 27 27 +28 36 29 8 33 24 41 34 19 3 26 28 28 28 35 26 36 29 8 38 32 11 28 28 +>EYKX4VC04JWDRY length=78 xy=3942_1068 region=4 run=R_2007_11_07_16_15_57_ +36 24 14 5 27 20 28 27 28 32 22 28 27 43 36 23 11 27 28 28 28 32 23 36 27 28 28 26 38 32 11 34 25 27 43 36 23 11 38 31 11 37 28 28 28 27 28 30 20 28 32 22 28 36 27 37 30 9 27 28 +28 27 28 42 35 20 5 28 28 28 35 26 27 27 26 39 32 12 +>EYKX4VC04JWMUW length=55 xy=3945_0550 region=4 run=R_2007_11_07_16_15_57_ +36 24 14 4 28 17 34 25 35 25 31 20 28 28 36 27 28 28 24 27 28 28 37 28 27 27 35 25 31 21 27 39 32 12 28 36 28 28 26 27 28 27 26 28 42 35 20 6 28 27 28 28 28 28 28 +>EYKX4VC04JH4RG length=85 xy=3779_3850 region=4 run=R_2007_11_07_16_15_57_ +37 28 35 26 38 31 10 27 37 28 28 38 31 10 27 35 25 25 28 28 28 28 28 28 28 28 27 28 33 23 28 32 22 35 25 31 20 34 25 31 21 26 28 27 26 26 15 36 29 7 27 27 24 36 27 28 37 28 36 28 +27 28 28 28 37 28 28 40 34 14 37 28 28 26 28 36 28 26 28 37 28 28 28 28 27 \ No newline at end of file diff -r 799a333342c4 -r 609ee7376d0d lib/galaxy/datatypes/test/sequence.qualsolid --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/galaxy/datatypes/test/sequence.qualsolid Thu Apr 30 10:54:43 2009 -0400 @@ -0,0 +1,12 @@ +>946_21_302_F3 +2 10 2 2 4 2 21 2 2 4 4 17 5 2 5 2 11 4 2 2 2 10 7 3 15 2 2 2 3 19 3 2 6 3 2 +>946_21_659_F3 +3 31 3 2 2 2 34 3 2 2 2 31 2 2 3 4 31 8 3 2 2 30 4 8 3 3 2 2 6 9 4 4 6 2 2 +>946_21_1071_F3 +5 5 2 2 2 8 5 3 2 3 7 7 2 3 4 6 5 2 2 2 5 5 2 2 2 3 8 2 3 3 3 8 2 3 2 +>946_21_1115_F3 +21 5 2 8 13 31 6 2 17 24 10 27 4 21 29 8 20 2 11 21 13 24 5 5 6 24 31 2 13 6 22 17 6 27 10 +>946_21_1218_F3 +11 21 2 13 13 16 27 16 19 27 22 28 14 26 24 23 29 10 15 13 6 4 7 16 26 22 11 6 16 22 21 6 4 7 21 +>946_21_1232_F3 +17 16 2 28 21 31 15 16 10 11 8 20 6 5 18 6 13 23 7 13 4 12 19 8 6 9 10 19 7 10 6 10 20 14 8 \ No newline at end of file

1 0

[hg] galaxy 2386: Change variable names in validation from min/m...
by Greg Von Kuster 01 May '09

01 May '09

details: http://www.bx.psu.edu/hg/galaxy/rev/799a333342c4 changeset: 2386:799a333342c4 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Thu Apr 30 10:24:41 2009 -0400 description: Change variable names in validation from min/max to range/length_min/max. 1 file(s) affected in this change: lib/galaxy/tools/parameters/validation.py diffs (40 lines): diff -r add0911007d9 -r 799a333342c4 lib/galaxy/tools/parameters/validation.py --- a/lib/galaxy/tools/parameters/validation.py Wed Apr 29 17:37:22 2009 -0400 +++ b/lib/galaxy/tools/parameters/validation.py Thu Apr 30 10:24:41 2009 -0400 @@ -102,10 +102,10 @@ @classmethod def from_element( cls, param, elem ): return cls( elem.get( 'message', None ), elem.get( 'min' ), elem.get( 'max' ) ) - def __init__( self, message, min, max ): - self.message = message or ( "Value must be between %f and %f" % ( min, max ) ) - self.min = float( min ) - self.max = float( max ) + def __init__( self, message, range_min, range_max ): + self.message = message or ( "Value must be between %f and %f" % ( range_min, range_max ) ) + self.min = float( range_min ) + self.max = float( range_max ) def validate( self, value, history=None ): if not( self.min <= float( value ) <= self.max ): raise ValueError( self.message ) @@ -134,14 +134,14 @@ @classmethod def from_element( cls, param, elem ): return cls( elem.get( 'message', None ), elem.get( 'min', None ), elem.get( 'max', None ) ) - def __init__( self, message, min, max ): + def __init__( self, message, length_min, length_max ): self.message = message - if min is not None: - min = int( min ) - if max is not None: - max = int( max ) - self.min = min - self.max = max + if length_min is not None: + length_min = int( length_min ) + if length_max is not None: + length_max = int( length_max ) + self.min = length_min + self.max = length_max def validate( self, value, history=None ): if self.min is not None and len( value ) < self.min: raise ValueError( self.message or ( "Must have length of at least %d" % self.min ) )

1 0

[hg] galaxy 2388: A tiny bug fix for linear regression tool.
by Greg Von Kuster 01 May '09

01 May '09

details: http://www.bx.psu.edu/hg/galaxy/rev/ff2fca65799e changeset: 2388:ff2fca65799e user: guru date: Thu Apr 30 11:07:34 2009 -0400 description: A tiny bug fix for linear regression tool. 1 file(s) affected in this change: tools/regVariation/linear_regression.py diffs (12 lines): diff -r 609ee7376d0d -r ff2fca65799e tools/regVariation/linear_regression.py --- a/tools/regVariation/linear_regression.py Thu Apr 30 10:54:43 2009 -0400 +++ b/tools/regVariation/linear_regression.py Thu Apr 30 11:07:34 2009 -0400 @@ -17,7 +17,7 @@ print "Predictor columns: %s; Response column: %d" %(x_cols,y_col+1) fout = open(outfile,'w') - +elems = [] for i, line in enumerate( file ( infile )): line = line.rstrip('\r\n') if len( line )>0 and not line.startswith( '#' ):

1 0

[hg] galaxy 2380: Replacing \"qual\" datatype with 3 new datatyp...
by Greg Von Kuster 01 May '09

01 May '09

details: http://www.bx.psu.edu/hg/galaxy/rev/22b08d47f7ba changeset: 2380:22b08d47f7ba user: guru date: Wed Apr 29 11:53:22 2009 -0400 description: Replacing \"qual\" datatype with 3 new datatypes: qualsolid, qual454 and qualsolexa. 10 file(s) affected in this change: datatypes_conf.xml.sample lib/galaxy/datatypes/converters/fastqsolexa_to_qual_converter.xml lib/galaxy/datatypes/qualityscore.py lib/galaxy/datatypes/registry.py lib/galaxy/model/migrate/versions/0006_change_qual_datatype.py tools/metag_tools/fastqsolexa_to_fasta_qual.xml tools/metag_tools/rmapq_wrapper.xml tools/metag_tools/short_reads_figure_high_quality_length.xml tools/metag_tools/short_reads_figure_score.xml tools/metag_tools/short_reads_trim_seq.xml diffs (409 lines): diff -r 5a4aac327bad -r 22b08d47f7ba datatypes_conf.xml.sample --- a/datatypes_conf.xml.sample Wed Apr 29 10:27:57 2009 -0400 +++ b/datatypes_conf.xml.sample Wed Apr 29 11:53:22 2009 -0400 @@ -19,7 +19,7 @@ </datatype> <datatype extension="fastqsolexa" type="galaxy.datatypes.sequence:FastqSolexa" display_in_upload="true"> <converter file="fastqsolexa_to_fasta_converter.xml" target_datatype="fasta"/> - <converter file="fastqsolexa_to_qual_converter.xml" target_datatype="qual"/> + <converter file="fastqsolexa_to_qual_converter.xml" target_datatype="qualsolexa"/> </datatype> <datatype extension="genetrack" type="galaxy.datatypes.tracks:GeneTrack"/> <datatype extension="gff" type="galaxy.datatypes.interval:Gff" display_in_upload="true"> @@ -42,7 +42,9 @@ </datatype> <datatype extension="pdf" type="galaxy.datatypes.images:Image" mimetype="application/pdf"/> <datatype extension="png" type="galaxy.datatypes.images:Image" mimetype="image/png"/> - <datatype extension="qual" type="galaxy.datatypes.qualityscore:QualityScore" display_in_upload="true"/> + <datatype extension="qualsolexa" type="galaxy.datatypes.qualityscore:QualityScoreSolexa" display_in_upload="true"/> + <datatype extension="qualsolid" type="galaxy.datatypes.qualityscore:QualityScoreSOLiD" display_in_upload="true"/> + <datatype extension="qual454" type="galaxy.datatypes.qualityscore:QualityScore454" display_in_upload="true"/> <datatype extension="scf" type="galaxy.datatypes.images:Scf" mimetype="application/octet-stream" display_in_upload="true"/> <datatype extension="taxonomy" type="galaxy.datatypes.tabular:Taxonomy" display_in_upload="true"/> <datatype extension="tabular" type="galaxy.datatypes.tabular:Tabular" display_in_upload="true"/> @@ -186,7 +188,9 @@ <sniffer type="galaxy.datatypes.sequence:Maf"/> <sniffer type="galaxy.datatypes.sequence:Lav"/> <sniffer type="galaxy.datatypes.sequence:csFasta"/> - <sniffer type="galaxy.datatypes.qualityscore:QualityScore"/> + <sniffer type="galaxy.datatypes.qualityscore:QualityScoreSolexa"/> + <sniffer type="galaxy.datatypes.qualityscore:QualityScoreSOLiD"/> + <sniffer type="galaxy.datatypes.qualityscore:QualityScore454"/> <sniffer type="galaxy.datatypes.sequence:Fasta"/> <sniffer type="galaxy.datatypes.sequence:FastqSolexa"/> <sniffer type="galaxy.datatypes.interval:Wiggle"/> diff -r 5a4aac327bad -r 22b08d47f7ba lib/galaxy/datatypes/converters/fastqsolexa_to_qual_converter.xml --- a/lib/galaxy/datatypes/converters/fastqsolexa_to_qual_converter.xml Wed Apr 29 10:27:57 2009 -0400 +++ b/lib/galaxy/datatypes/converters/fastqsolexa_to_qual_converter.xml Wed Apr 29 11:53:22 2009 -0400 @@ -4,7 +4,7 @@ <param format="fastqsolexa" name="input1" type="data" label="Choose Fastqsolexa file"/> </inputs> <outputs> - <data format="qual" name="output1" /> + <data format="qualsolexa" name="output1" /> </outputs> <help> </help> diff -r 5a4aac327bad -r 22b08d47f7ba lib/galaxy/datatypes/qualityscore.py --- a/lib/galaxy/datatypes/qualityscore.py Wed Apr 29 10:27:57 2009 -0400 +++ b/lib/galaxy/datatypes/qualityscore.py Wed Apr 29 11:53:22 2009 -0400 @@ -9,11 +9,11 @@ log = logging.getLogger(__name__) -class QualityScore ( data.Text ): +class QualityScoreSOLiD ( data.Text ): """ until we know more about quality score formats """ - file_ext = "qual" + file_ext = "qualsolid" def set_peek( self, dataset, line_count=None ): if not dataset.dataset.purged: @@ -21,7 +21,7 @@ if line_count is None: dataset.blurb = data.nice_size( dataset.get_size() ) else: - dataset.blurb = "%s lines, Quality score file" % util.commaify( str( line_count ) ) + dataset.blurb = "%s lines, SOLiD Quality score file" % util.commaify( str( line_count ) ) else: dataset.peek = 'file does not exist' dataset.blurb = 'file purged from disk' @@ -30,15 +30,80 @@ try: return dataset.peek except: - return "Quality score file (%s)" % ( data.nice_size( dataset.get_size() ) ) + return "SOLiD Quality score file (%s)" % ( data.nice_size( dataset.get_size() ) ) def sniff( self, filename ): """ >>> fname = get_test_fname( 'sequence.fasta' ) - >>> QualityScore().sniff( fname ) + >>> QualityScoreSOLiD().sniff( fname ) False - >>> fname = get_test_fname( 'sequence.qual' ) - >>> QualityScore().sniff( fname ) + >>> fname = get_test_fname( 'sequence.qualsolid' ) + >>> QualityScoreSOLiD().sniff( fname ) + True + """ + try: + fh = open( filename ) + readlen = None + goodblock = 0 + while True: + line = fh.readline() + if not line: + if goodblock > 0: + return True + else: + break #EOF + line = line.strip() + if line and not line.startswith( '#' ): #first non-empty non-comment line + if line.startswith( '>' ): + line = fh.readline().strip() + if line == '' or line.startswith( '>' ): + break + try: + [ int( x ) for x in line.split() ] + if not(readlen): + readlen = len(line.split()) + assert len(line.split()) == readlen #SOLiD reads should be of the same length + except: + break + goodblock += 1 + if goodblock > 10: + return True + else: + break #we found a non-empty line, but it's not a header + except: + pass + return False + +class QualityScore454 ( data.Text ): + """ + until we know more about quality score formats + """ + file_ext = "qual454" + + def set_peek( self, dataset, line_count=None ): + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + if line_count is None: + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.blurb = "%s lines, 454 Quality score file" % util.commaify( str( line_count ) ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' + + def display_peek(self, dataset): + try: + return dataset.peek + except: + return "454 Quality score file (%s)" % ( data.nice_size( dataset.get_size() ) ) + + def sniff( self, filename ): + """ + >>> fname = get_test_fname( 'sequence.fasta' ) + >>> QualityScore454().sniff( fname ) + False + >>> fname = get_test_fname( 'sequence.qual454' ) + >>> QualityScore454().sniff( fname ) True """ try: @@ -63,3 +128,59 @@ except: pass return False + +class QualityScoreSolexa ( data.Text ): + """ + until we know more about quality score formats + """ + file_ext = "qualsolexa" + + def set_peek( self, dataset, line_count=None ): + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + if line_count is None: + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.blurb = "%s lines, Solexa Quality score file" % util.commaify( str( line_count ) ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' + + def display_peek(self, dataset): + try: + return dataset.peek + except: + return "Solexa Quality score file (%s)" % ( data.nice_size( dataset.get_size() ) ) + + def sniff( self, filename ): + """ + >>> fname = get_test_fname( 'sequence.fasta' ) + >>> QualityScoreSolexa().sniff( fname ) + False + >>> fname = get_test_fname( 'sequence.qualsolexa' ) + >>> QualityScoreSolexa().sniff( fname ) + True + """ + try: + fh = open( filename ) + readlen = None + while True: + line = fh.readline() + if not line: + break #EOF + line = line.strip() + if line and not line.startswith( '#' ): + if len(line.split('\t')) > 1: + break + try: + [ int( x ) for x in line.split() ] + if not(readlen): + readlen = len(line.split()) + assert len(line.split()) == readlen #Solexa reads should be of the same length + except: + break + + except: + pass + return False + diff -r 5a4aac327bad -r 22b08d47f7ba lib/galaxy/datatypes/registry.py --- a/lib/galaxy/datatypes/registry.py Wed Apr 29 10:27:57 2009 -0400 +++ b/lib/galaxy/datatypes/registry.py Wed Apr 29 11:53:22 2009 -0400 @@ -116,7 +116,9 @@ 'laj' : images.Laj(), 'lav' : sequence.Lav(), 'maf' : sequence.Maf(), - 'qual' : qualityscore.QualityScore(), + 'qualsolid' : qualityscore.QualityScoreSOLiD(), + 'qualsolexa' : qualityscore.QualityScoreSolexa(), + 'qual454' : qualityscore.QualityScore454(), 'scf' : images.Scf(), 'tabular' : tabular.Tabular(), 'taxonomy' : tabular.Taxonomy(), @@ -140,7 +142,9 @@ 'laj' : 'text/plain', 'lav' : 'text/plain', 'maf' : 'text/plain', - 'qual' : 'text/plain', + 'qualsolid' : 'text/plain', + 'qualsolexa' : 'text/plain', + 'qual454' : 'text/plain', 'scf' : 'application/octet-stream', 'tabular' : 'text/plain', 'taxonomy' : 'text/plain', diff -r 5a4aac327bad -r 22b08d47f7ba lib/galaxy/model/migrate/versions/0006_change_qual_datatype.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/galaxy/model/migrate/versions/0006_change_qual_datatype.py Wed Apr 29 11:53:22 2009 -0400 @@ -0,0 +1,47 @@ +from sqlalchemy import * +from sqlalchemy.orm import * +from migrate import * +import sys, logging + +log = logging.getLogger( __name__ ) +log.setLevel(logging.DEBUG) +handler = logging.StreamHandler( sys.stdout ) +format = "%(name)s %(levelname)s %(asctime)s %(message)s" +formatter = logging.Formatter( format ) +handler.setFormatter( formatter ) +log.addHandler( handler ) + +metadata = MetaData( migrate_engine ) +db_session = scoped_session( sessionmaker( bind=migrate_engine, autoflush=False, transactional=False ) ) +HistoryDatasetAssociation_table = Table( "history_dataset_association", metadata, autoload=True ) + +def upgrade(): + # Load existing tables + metadata.reflect() + # Add 2 indexes to the galaxy_user table + i = Index( 'ix_hda_extension', HistoryDatasetAssociation_table.c.extension ) + try: + i.create() + except Exception, e: + log.debug( "Adding index 'ix_hda_extension' to history_dataset_association table failed: %s" % ( str( e ) ) ) + + # Set the default data in the galaxy_user table, but only for null values + cmd = "UPDATE history_dataset_association SET extension = 'qual454' WHERE extension = 'qual' and peek like \'>%%\'" + try: + db_session.execute( cmd ) + except Exception, e: + log.debug( "Resetting extension qual to qual454 in history_dataset_association failed: %s" % ( str( e ) ) ) + cmd = "UPDATE history_dataset_association SET extension = 'qualsolexa' WHERE extension = 'qual' and peek not like \'>%%\'" + try: + db_session.execute( cmd ) + except Exception, e: + log.debug( "Resetting extension qual to qualsolexa in history_dataset_association failed: %s" % ( str( e ) ) ) + # Add 1 index to the history_dataset_association table + try: + i.drop() + except Exception, e: + log.debug( "Dropping index 'ix_hda_extension' to history_dataset_association table failed: %s" % ( str( e ) ) ) + + +def downgrade(): + pass diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/fastqsolexa_to_fasta_qual.xml --- a/tools/metag_tools/fastqsolexa_to_fasta_qual.xml Wed Apr 29 10:27:57 2009 -0400 +++ b/tools/metag_tools/fastqsolexa_to_fasta_qual.xml Wed Apr 29 11:53:22 2009 -0400 @@ -6,7 +6,7 @@ </inputs> <outputs> <data name="output1" format="fasta"/> - <data name="output2" format="qual"/> + <data name="output2" format="qualsolexa"/> </outputs> <tests>  diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/rmapq_wrapper.xml --- a/tools/metag_tools/rmapq_wrapper.xml Wed Apr 29 10:27:57 2009 -0400 +++ b/tools/metag_tools/rmapq_wrapper.xml Wed Apr 29 11:53:22 2009 -0400 @@ -13,7 +13,7 @@ </options> </param> <param name="input_seq" type="data" format="fasta" label="Sequence file"/> - <param name="input_score" type="data" format="qual" label="Quality score file"/> + <param name="input_score" type="data" format="qualsolexa" label="Quality score file"/> <param name="high_score" type="float" size="15" value="40" label="Minimum score for high-quality base (-q)"/> <param name="high_len" type="integer" size="15" value="36" label="Minimal high-quality bases (-M)"/> <param name="align_len" type="integer" size="15" value="11" label="Minimal length of a hit (-h)" help="seed"/> @@ -46,7 +46,7 @@ <test> <param name="database" value="/depot/data2/galaxy/faseq/test" /> <param name="input_seq" value="rmapq_wrapper_test1.fasta" ftype="fasta"/> - <param name="input_score" value="rmapq_wrapper_test1.qual" ftype="qual" /> + <param name="input_score" value="rmapq_wrapper_test1.qual" ftype="qualsolexa" /> <param name="high_score" value="40" /> <param name="high_len" value="36" /> <param name="read_len" value="36" /> diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/short_reads_figure_high_quality_length.xml --- a/tools/metag_tools/short_reads_figure_high_quality_length.xml Wed Apr 29 10:27:57 2009 -0400 +++ b/tools/metag_tools/short_reads_figure_high_quality_length.xml Wed Apr 29 11:53:22 2009 -0400 @@ -5,7 +5,7 @@ <inputs> <page> - <param name="input1" type="data" format="qual,txtseq.zip" label="Quality score file" help="No dataset? Read tip below"/> + <param name="input1" type="data" format="qualsolexa,qual454,txtseq.zip" label="Quality score file" help="No dataset? Read tip below"/> <param name="input2" type="integer" size="5" value="20" label="Quality score threshold" /> </page> </inputs> @@ -17,12 +17,12 @@ </requirements> <tests> <test> - <param name="input1" value="solexa.qual" ftype="qual" /> + <param name="input1" value="solexa.qual" ftype="qualsolexa" /> <param name="input2" value="5" /> <output name="output1" file="solexa_high_quality_hist.pdf" ftype="pdf"/> </test> <test> - <param name="input1" value="454.qual" ftype="qual" /> + <param name="input1" value="454.qual" ftype="qual454" /> <param name="input2" value="5" /> <output name="output1" file="454_high_quality_hist.pdf" ftype="pdf"/> </test> diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/short_reads_figure_score.xml --- a/tools/metag_tools/short_reads_figure_score.xml Wed Apr 29 10:27:57 2009 -0400 +++ b/tools/metag_tools/short_reads_figure_score.xml Wed Apr 29 11:53:22 2009 -0400 @@ -5,7 +5,7 @@ <inputs> <page> - <param name="input1" type="data" format="qual, txtseq.zip" label="Quality score file" help="No dataset? Read tip below"/> + <param name="input1" type="data" format="qualsolexa, qual454, txtseq.zip" label="Quality score file" help="No dataset? Read tip below"/> </page> </inputs> @@ -17,11 +17,11 @@ </requirements> <tests> <test> - <param name="input1" value="solexa.qual" ftype="qual" /> + <param name="input1" value="solexa.qual" ftype="qualsolexa" /> <output name="output1" file="solexaScore.png" ftype="png" /> </test> <test> - <param name="input1" value="454.qual" ftype="qual" /> + <param name="input1" value="454.qual" ftype="qual454" /> <output name="output1" file="454Score.png" ftype="png" /> </test> </tests> diff -r 5a4aac327bad -r 22b08d47f7ba tools/metag_tools/short_reads_trim_seq.xml --- a/tools/metag_tools/short_reads_trim_seq.xml Wed Apr 29 10:27:57 2009 -0400 +++ b/tools/metag_tools/short_reads_trim_seq.xml Wed Apr 29 11:53:22 2009 -0400 @@ -7,7 +7,7 @@ <inputs> <page> <param name="input1" type="data" format="fasta,txtseq.zip" label="Reads" /> - <param name="input2" type="data" format="qual,txtseq.zip" label="Quality scores" /> + <param name="input2" type="data" format="qualsolexa,qual454,txtseq.zip" label="Quality scores" /> <param name="trim" type="integer" size="5" value="20" label="Minimal quality score" help="bases scoring below this value will trigger splitting"/> <param name="length" type="integer" size="5" value="100" label="Minimal length of contiguous segment" help="report all high quality segments above this length. Setting this option to '0' will cause the program to return a single longest run of high quality bases per read" /> <conditional name="sequencing_method_choice"> @@ -36,7 +36,7 @@ <test> <param name="sequencer" value="454" /> <param name="input1" value="454.fasta" ftype="fasta" /> - <param name="input2" value="454.qual" ftype="qual" /> + <param name="input2" value="454.qual" ftype="qual454" /> <param name="input3" value="no" /> <param name="trim" value="20" /> <param name="length" value="0" /> @@ -45,7 +45,7 @@ <test> <param name="sequencer" value="Solexa" /> <param name="input1" value="solexa.fasta" ftype="fasta" /> - <param name="input2" value="solexa.qual" ftype="qual" /> + <param name="input2" value="solexa.qual" ftype="qualsolexa" /> <param name="input3" value="0" /> <param name="trim" value="20" /> <param name="length" value="0" />

1 0

[hg] galaxy 2382: Fix for reporting job errors - somehow necessa...
by Greg Von Kuster 01 May '09

01 May '09

details: http://www.bx.psu.edu/hg/galaxy/rev/89541ac4983f changeset: 2382:89541ac4983f user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Apr 29 13:56:59 2009 -0400 description: Fix for reporting job errors - somehow necessary imported stuff got eliminated. 1 file(s) affected in this change: lib/galaxy/web/controllers/dataset.py diffs (24 lines): diff -r dae0313bf5bb -r 89541ac4983f lib/galaxy/web/controllers/dataset.py --- a/lib/galaxy/web/controllers/dataset.py Wed Apr 29 13:46:02 2009 -0400 +++ b/lib/galaxy/web/controllers/dataset.py Wed Apr 29 13:56:59 2009 -0400 @@ -1,7 +1,7 @@ -import logging, os, mimetypes, smtplib +import logging, os, sets, string, shutil, re, socket, mimetypes, smtplib from galaxy.web.base.controller import * -from galaxy import web, model +from galaxy import util, datatypes, jobs, web, model from cgi import escape, FieldStorage from email.MIMEText import MIMEText @@ -89,8 +89,8 @@ s.sendmail( frm, [ to ], msg.as_string() ) s.close() return trans.show_ok_message( "Your error report has been sent" ) - except: - return trans.show_error_message( "An error occurred sending the report by email" ) + except Exception, e: + return trans.show_error_message( "An error occurred sending the report by email: %s" % str( e ) ) @web.expose def default(self, trans, dataset_id=None, **kwd):

1 0

[hg] galaxy 2383: Accidently commited a testing change, removed
by Greg Von Kuster 01 May '09

01 May '09

details: http://www.bx.psu.edu/hg/galaxy/rev/656f1df80b46 changeset: 2383:656f1df80b46 user: James Taylor <james(a)jamestaylor.org> date: Wed Apr 29 14:47:18 2009 -0400 description: Accidently commited a testing change, removed 1 file(s) affected in this change: tools/maf/interval2maf.xml diffs (11 lines): diff -r 89541ac4983f -r 656f1df80b46 tools/maf/interval2maf.xml --- a/tools/maf/interval2maf.xml Wed Apr 29 13:56:59 2009 -0400 +++ b/tools/maf/interval2maf.xml Wed Apr 29 14:47:18 2009 -0400 @@ -6,7 +6,6 @@ #end if </command> <inputs> - <param type="text" name="TEST" label="Test" /> <param format="interval" name="input1" type="data" label="Choose intervals"> <validator type="unspecified_build" /> </param>

1 0