January 2009 - galaxy-dev - lists.galaxyproject.org

New report page - disk usage over time
by Assaf Gordon 18 Jan '09

18 Jan '09

Last report (for today), Shows how much space is consumed each month. At this going rate, my new 1.0TB disk wouldn't last more than six months... :-) Regards, - Gordon.

1 0

New report page - disk usage by file-size
by Assaf Gordon 17 Jan '09

17 Jan '09

Hello, I'd like to suggest another small feature: A report page that shows the largest datasets, with the relevant user information. With this report I can see which users use the most space. The current report (of files over 4GB) is not enough for me - so far I don't have files larger than 4GB, but have many files over 1GB (and less than 1TB of storage space...). As with my previous email - this feature works fine for small Galaxies, where one knows the users personally (and can set them straight, so to speak). With larger installation, this report might not be necessary. Regards, Gordon.

1 0

New report page - find Datasets by ID
by Assaf Gordon 17 Jan '09

17 Jan '09

Hello, I'd like to suggest a small feature: A new report page that helps finding datasets information based on ID. This report allows getting all the information about a dataset ( user email, history name, dataset name, file path on server) based on either the Dataset-ID or the HistoryDatasetAssociation-ID. I think this tool is useful for Galaxy servers with small number of users (as it is in my case). It might be also useful for 'larger' Galaxies. This tool serves two common scenarios: 1. When a tool crashes, or generates and error, or just outputs something weird - if the tool has a log, or if I look at the galaxy log - I can easily see the input dataset that was used with this tool (e.g. 'dataset_1192.dat') - Now I want to know what was this file, which user used it, and for what (As I said - in a small Galaxy I personally know all the users). So I just go to this report page, enter 1192 as the File-ID, and get all the needed information. 2. When a user complains about something not working properly, or when I want to change a dataset in-place without creating a new dataset (again, probably something that happens only with on a small Galaxy) - I just ask the user for the number of the dataset, and with this report I quickly find out which file on the local disk correspond to the user's dataset. Regards, - Gordon.

1 0

[hg] galaxy 1709: More changes to mutability tool.
by gua110＠scofield.bx.psu.edu 15 Jan '09

15 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/90d04fcbcdf4 changeset: 1709:90d04fcbcdf4 user: guru date: Thu Jan 15 13:10:04 2009 -0500 description: More changes to mutability tool. 2 file(s) affected in this change: tools/regVariation/microsats_mutability.py tools/regVariation/microsats_mutability.xml diffs (99 lines): diff -r 78c2f403cb43 -r 90d04fcbcdf4 tools/regVariation/microsats_mutability.py --- a/tools/regVariation/microsats_mutability.py Thu Jan 15 10:49:01 2009 -0500 +++ b/tools/regVariation/microsats_mutability.py Thu Jan 15 13:10:04 2009 -0500 @@ -128,11 +128,13 @@ uniq_s_elems_2 = get_binned_lists(uniq_s_elems_2,s_bin_size) for pitem1 in uniq_elems_1: - repeats1 = [] - repeats2 = [] + #repeats1 = [] + #repeats2 = [] thresholds = [] if s_group_cols[0] != -1: #Sub-group by feature is not None for sitem1 in uniq_s_elems_1: + repeats1 = [] + repeats2 = [] if type(sitem1) == type(''): sitem1 = sitem1.strip() for bline in blk_lines: @@ -223,11 +225,13 @@ count1[str(pitem1)]=sum(repeats2) for pitem2 in uniq_elems_2: - repeats1 = [] - repeats2 = [] + #repeats1 = [] + #repeats2 = [] thresholds = [] if s_group_cols[0] != -1: #Sub-group by feature is not None for sitem2 in uniq_s_elems_2: + repeats1 = [] + repeats2 = [] if type(sitem2)==type(''): sitem2 = sitem2.strip() for bline in blk_lines: @@ -349,11 +353,10 @@ count = count1[key] mut = "%.2e" %(mut/num_generations) if region == 'align': - print >>fout, str(blk) + '\t'+seq1 + '\t' + start1+ '\t'+end1+ '\t'+seq2 + '\t'+start2+ '\t'+end2+ '\t'+key.strip()+ '\t'+str(mut) + '\t'+ str(count) + print >>fout, str(blk) + '\t'+seq1 + '\t' + seq2 + '\t' +key.strip()+ '\t'+str(mut) + '\t'+ str(count) elif region == 'win': fout.write("%s\t%s\t%s\t%s\n" %(blk,key.strip(),mut,count)) fout.flush() - #print >>fout, blk + '\t'+key.strip()+ '\t'+str(mut)+ '\t'+ str(count) #catch any remaining repeats, for instance if the orthologous position contained different repeat units for remaining_key in mut2.keys(): @@ -361,7 +364,7 @@ mut = "%.2e" %(mut/num_generations) count = count2[remaining_key] if region == 'align': - print >>fout, str(blk) + '\t'+seq1 + '\t' + start1+ '\t'+end1+ '\t'+seq2 + '\t'+start2+ '\t'+end2+ '\t'+remaining_key.strip()+ '\t'+str(mut)+ '\t'+ str(count) + print >>fout, str(blk) + '\t'+seq1 + '\t'+seq2 + '\t'+remaining_key.strip()+ '\t'+str(mut)+ '\t'+ str(count) elif region == 'win': fout.write("%s\t%s\t%s\t%s\n" %(blk,remaining_key.strip(),mut,count)) fout.flush() @@ -420,17 +423,9 @@ fix_strand = True) msatTree = quicksect.IntervalTree() for item in msats: - #print >>sys.stderr, item if type( item ) is GenomicInterval: msatTree.insert( item, msats.linenum, item.fields ) - """ - result = [] - msatTree.traverse(lambda node: result.append( node )) - for n in result: - print >>sys.stderr,n.other - print >>sys.stderr,msatTree.chroms - #sys.exit() - """ + for iline in fint: try: iline = iline.rstrip('\r\n') @@ -460,7 +455,7 @@ print "Skipped %d intervals as invalid." %(skipped) elif region == 'align': if s_group_cols[0] != -1: - print >>fout, "#Window\tSpecies_1\tWindow_Start\tWindow_End\tSpecies_2\tGroupby_Feature\tSubGroupby_Feature\tMutability\tCount" + print >>fout, "#Window\tSpecies_1\tSpecies_2\tGroupby_Feature\tSubGroupby_Feature\tMutability\tCount" else: print >>fout, "#Window\tSpecies_1\tWindow_Start\tWindow_End\tSpecies_2\tGroupby_Feature\tMutability\tCount" prev_bnum = -1 diff -r 78c2f403cb43 -r 90d04fcbcdf4 tools/regVariation/microsats_mutability.xml --- a/tools/regVariation/microsats_mutability.xml Thu Jan 15 10:49:01 2009 -0500 +++ b/tools/regVariation/microsats_mutability.xml Thu Jan 15 13:10:04 2009 -0500 @@ -106,6 +106,10 @@ This tool computes microsatellite mutability for the orthologous microsatellites fetched from 'Extract Orthologous Microsatellites from pair-wise alignments' tool. +Mutability is computed according to the method described in the following paper: + +*Webster et al., Microsatellite evolution inferred from human-chimpanzee genomic sequence alignments, Proc Natl Acad Sci 2002 June 25; 99(13): 8748-8753* + ----- .. class:: warningmark

1 0

Adding the 'name' content of the selection parameter
by Assaf Gordon 15 Jan '09

15 Jan '09

Hello, Continuing the discussion from Dec. 19th, http://mail.bx.psu.edu/pipermail/galaxy-user/2008-December/000422.html The following hack adds an additional key/value which contains the 'name' of the selected option in a <select> input parameter. With this hack, the following tool configuration is possible (note the output/label option): ========================== <tool> .. <inputs> <input format="fasta" type="data" name="input"> <input type="select" name="database" label="Database"> <option value="/home/gordon/long/path/hg18.fa">Human (hg18)</option> <option value="/home/gordon/long/path/dm3.fa">Fly (dm3)</option> <option value="/home/gordon/long/path/mm9.fa">Mouse (mm9)</option> </input> </inputs> <outputs>  <data format="txt" name="output" label="Blat $input1 on $database" />  <data format="txt" name="output" label="Blat $input1 on $database_name" /> </outputs> ========================== Currently, using "$database" in the output/label part will put the *value* of the selected option (e.g. "/home/gordon/long/path/hg18.fa") in the dataset's name. This doesn't really help the user understand what the dataset contains. With this hack, for each <select> input, a new variable is created (with the "_name" suffix) which will contain the *name* of the selected option (e.g. "Human (hg18)"). For example, if the user selected the second option, Then "$database" will contain "/home/gordon/long/path/dm3.fa", and "$database_name" will contain "Fly (dm3)". I'm well aware this is an ugly hack. A better way would be to have each parameter (in the incoming dictionary) be an object (e.g. ToolParameter) and not a string. Then we could write something like "$database.name" or "$database.value". The hack contains two patches. The first add support for 'value_to_display_text' with dynamic options, in "Parameters/Basic.py": ============================================================== --- galaxy_prod/lib/galaxy/tools/parameters/basic.py +++ galaxy_devel/lib/galaxy/tools/parameters/basic.py @@ -576,7 +576,7 @@ elif len( value ) == 1: value = value[0] return value - def value_to_display_text( self, value, app ): + def value_to_display_text( self, value, app, other_values={} ): if isinstance( value, UnvalidatedValue ): suffix = "\n(value not yet validated)" value = value.value @@ -584,10 +584,11 @@ suffix = "" if not isinstance( value, list ): value = [ value ] - # FIXME: Currently only translating values back to labels if they - # are not dynamic if self.is_dynamic: - rval = map( str, value ) + rval = [ ] + for name in self.options.get_options ( app , other_values ) : + if ( name[1] in value ): + rval.append ( name[0] ) else: options = list( self.static_options ) rval = [] ============================================================== The second patch iterates the tool's parameters, and for each <select> type parameter, adds the additional key/value for the selected option: ============================================================== --- galaxy_prod/lib/galaxy/tools/actions/__init__.py +++ galaxy_devel/lib/galaxy/tools/actions/__init__.py @@ -96,6 +96,18 @@ on_text = "" # Add the dbkey to the incoming parameters incoming[ "dbkey" ] = input_dbkey + + ##28dec2008, gordon + ## For every 'Select' parameter, we get only the 'value' part of the selection from the HTTP request. + ## The following code gets the 'name' part for the selected option + selection_parameter_names = {} + for param_name, param_selected_value in incoming.iteritems(): + param_obj = tool.get_param ( param_name ) + if isinstance( param_obj, basic.SelectToolParameter): + param_selected_name = param_obj.value_to_display_text( param_selected_value, trans, incoming ) + selection_parameter_names [ param_name + "_name" ] = param_selected_name.replace("\n","") + incoming.update( selection_parameter_names ) + # Keep track of parent / child relationships, we'll create all the # datasets first, then create the associations parent_to_child_pairs = [] ============================================================== I tried to make the changes as non intrusive as possible. I hope the added 'other_value' parameter doesn't affect other places that call 'value_to_display_text' - but I'm not quite sure about it (I'm still very new to python). I've tested this hack with several tools, and it seems to work (or at least not crash horribly). Comments are welcomed, Gordon.

2 1

request: add newlines to dataset's info field
by Assaf Gordon 15 Jan '09

15 Jan '09

Hello, I'd like to request/suggest a tiny change in the dataset's info field. Whenever a tool outputs some lines into STDOUT, they are displayed in the dataset's info field. Currently, if the info field contains several lines, they are displayed as a single line (because HTML ignores new lines). The following patch causes the info field to be displayed correctly. ================================================================= --- galaxy_prod/lib/galaxy/datatypes/data.py +++ galaxy_devel/lib/galaxy/datatypes/data.py @@ -132,7 +132,7 @@ def display_info(self, dataset): """Returns formated html of dataset info""" try: - return escape(dataset.info) + return escape(dataset.info).replace("\n", "<br>") except: return "info unavailable" def validate(self, dataset): ================================================================== Thanks, gordon.

2 1

[hg] galaxy 1708: Change History.copy() back to use datasets rat...
by Greg Von Kuster 15 Jan '09

15 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/78c2f403cb43 changeset: 1708:78c2f403cb43 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Thu Jan 15 10:49:01 2009 -0500 description: Change History.copy() back to use datasets rather than active_datasets. Add checks in set_peek() to properly set peek on purged datasets. Fixes sharing a history that contains purged datasets. 9 file(s) affected in this change: lib/galaxy/datatypes/data.py lib/galaxy/datatypes/genetics.py lib/galaxy/datatypes/images.py lib/galaxy/datatypes/interval.py lib/galaxy/datatypes/qualityscore.py lib/galaxy/datatypes/sequence.py lib/galaxy/datatypes/xml.py lib/galaxy/model/__init__.py tools/data_source/echo.xml diffs (441 lines): diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/data.py Thu Jan 15 10:49:01 2009 -0500 @@ -103,8 +103,12 @@ return False def set_peek( self, dataset ): """Set the peek and blurb text""" - dataset.peek = '' - dataset.blurb = 'data' + if not dataset.dataset.purged: + dataset.peek = '' + dataset.blurb = 'data' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): """Create HTML table, used for displaying peek""" out = ['<table cellspacing="0" cellpadding="3">'] @@ -275,19 +279,27 @@ return 'text/plain' def set_peek( self, dataset, line_count=None ): - dataset.peek = get_file_peek( dataset.file_name ) - if line_count is None: - dataset.blurb = "%s lines" % util.commaify( str( get_line_count( dataset.file_name ) ) ) + if not dataset.dataset.purged: + # The file must exist on disk for the get_file_peek() method + dataset.peek = get_file_peek( dataset.file_name ) + if line_count is None: + dataset.blurb = "%s lines" % util.commaify( str( get_line_count( dataset.file_name ) ) ) + else: + dataset.blurb = "%s lines" % util.commaify( str( line_count ) ) else: - dataset.blurb = "%s lines" % util.commaify( str( line_count ) ) + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' class Binary( Data ): """Binary data""" - def set_peek( self, dataset ): """Set the peek and blurb text""" - dataset.peek = 'binary data' - dataset.blurb = 'data' + if not dataset.dataset.purged: + dataset.peek = 'binary data' + dataset.blurb = 'data' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def get_test_fname( fname ): """Returns test data filename""" diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/genetics.py --- a/lib/galaxy/datatypes/genetics.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/genetics.py Thu Jan 15 10:49:01 2009 -0500 @@ -43,11 +43,14 @@ def set_peek( self, dataset ): """Set the peek and blurb text""" - dataset.peek = data.get_file_peek( dataset.file_name ) - ## dataset.peek = self.make_html_table( dataset.peek ) - dataset.blurb = util.commaify( str( data.get_line_count( dataset.file_name ) ) ) + " rows" - #i don't think set_meta should not be called here, it should be called separately - self.set_meta( dataset ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = util.commaify( str( data.get_line_count( dataset.file_name ) ) ) + " rows" + #i don't think set_meta should not be called here, it should be called separately + self.set_meta( dataset ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def get_estimated_display_viewport( self, dataset ): """Return a chrom, start, stop tuple for viewing a file.""" @@ -130,8 +133,12 @@ file_ext="snpmatrix" def set_peek( self, dataset ): - dataset.peek = "Binary RGenetics file" - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = "Binary RGenetics file" + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ """ diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/images.py --- a/lib/galaxy/datatypes/images.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/images.py Thu Jan 15 10:49:01 2009 -0500 @@ -14,9 +14,13 @@ """Class describing an ab1 binary sequence file""" file_ext = "ab1" def set_peek( self, dataset ): - export_url = "/history_add_to?"+urlencode({'history_id':dataset.history_id,'ext':'ab1','name':'ab1 sequence','info':'Sequence file','dbkey':dataset.dbkey}) - dataset.peek = "Binary ab1 sequence file" - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + export_url = "/history_add_to?" + urlencode({'history_id':dataset.history_id,'ext':'ab1','name':'ab1 sequence','info':'Sequence file','dbkey':dataset.dbkey}) + dataset.peek = "Binary ab1 sequence file" + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -27,9 +31,13 @@ """Class describing an scf binary sequence file""" file_ext = "scf" def set_peek( self, dataset ): - export_url = "/history_add_to?"+urlencode({'history_id':dataset.history_id,'ext':'scf','name':'scf sequence','info':'Sequence file','dbkey':dataset.dbkey}) - dataset.peek = "Binary scf sequence file" - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + export_url = "/history_add_to?" + urlencode({'history_id':dataset.history_id,'ext':'scf','name':'scf sequence','info':'Sequence file','dbkey':dataset.dbkey}) + dataset.peek = "Binary scf sequence file" + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -40,10 +48,14 @@ """Class describing a zip archive of binary sequence files""" file_ext = "binseq.zip" def set_peek( self, dataset ): - zip_file = zipfile.ZipFile( dataset.file_name, "r" ) - num_files = len( zip_file.namelist() ) - dataset.peek = "Archive of %s binary sequence files" % ( str( num_files ) ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + zip_file = zipfile.ZipFile( dataset.file_name, "r" ) + num_files = len( zip_file.namelist() ) + dataset.peek = "Archive of %s binary sequence files" % ( str( num_files ) ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -57,10 +69,14 @@ """Class describing a zip archive of text sequence files""" file_ext = "txtseq.zip" def set_peek( self, dataset ): - zip_file = zipfile.ZipFile( dataset.file_name, "r" ) - num_files = len( zip_file.namelist() ) - dataset.peek = "Archive of %s text sequence files" % ( str( num_files ) ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + zip_file = zipfile.ZipFile( dataset.file_name, "r" ) + num_files = len( zip_file.namelist() ) + dataset.peek = "Archive of %s text sequence files" % ( str( num_files ) ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -73,8 +89,12 @@ class Image( data.Data ): """Class describing an image""" def set_peek( self, dataset ): - dataset.peek = 'Image in %s format' % dataset.extension - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = 'Image in %s format' % dataset.extension + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def create_applet_tag_peek( class_name, archive, params ): text = """ @@ -104,18 +124,22 @@ """Class describing a GMAJ Applet""" file_ext = "gmaj.zip" def set_peek( self, dataset ): - params = { - "bundle":"display?id=%s&tofile=yes&toext=.zip" % dataset.id, - "buttonlabel": "Launch GMAJ", - "nobutton": "false", - "urlpause" :"100", - "debug": "false", - "posturl": "history_add_to?%s" % urlencode( { 'history_id': dataset.history_id, 'ext': 'maf', 'name': 'GMAJ Output on data %s' % dataset.hid, 'info': 'Added by GMAJ', 'dbkey': dataset.dbkey } ) - } - class_name = "edu.psu.bx.gmaj.MajApplet.class" - archive = "/static/gmaj/gmaj.jar" - dataset.peek = create_applet_tag_peek( class_name, archive, params ) - dataset.blurb = 'GMAJ Multiple Alignment Viewer' + if not dataset.dataset.purged: + params = { + "bundle":"display?id=%s&tofile=yes&toext=.zip" % dataset.id, + "buttonlabel": "Launch GMAJ", + "nobutton": "false", + "urlpause" :"100", + "debug": "false", + "posturl": "history_add_to?%s" % urlencode( { 'history_id': dataset.history_id, 'ext': 'maf', 'name': 'GMAJ Output on data %s' % dataset.hid, 'info': 'Added by GMAJ', 'dbkey': dataset.dbkey } ) + } + class_name = "edu.psu.bx.gmaj.MajApplet.class" + archive = "/static/gmaj/gmaj.jar" + dataset.peek = create_applet_tag_peek( class_name, archive, params ) + dataset.blurb = 'GMAJ Multiple Alignment Viewer' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -147,8 +171,12 @@ """Class describing an html file""" file_ext = "html" def set_peek( self, dataset ): - dataset.peek = "HTML file" - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = "HTML file" + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def get_mime(self): """Returns the mime type of the datatype""" return 'text/html' @@ -176,17 +204,21 @@ """Class describing a LAJ Applet""" file_ext = "laj" def set_peek( self, dataset ): - params = { - "alignfile1": "display?id=%s" % dataset.id, - "buttonlabel": "Launch LAJ", - "title": "LAJ in Galaxy", - "posturl": "history_add_to?%s" % urlencode( { 'history_id': dataset.history_id, 'ext': 'lav', 'name': 'LAJ Output', 'info': 'Added by LAJ', 'dbkey': dataset.dbkey } ), - "noseq": "true" - } - class_name = "edu.psu.cse.bio.laj.LajApplet.class" - archive = "/static/laj/laj.jar" - dataset.peek = create_applet_tag_peek( class_name, archive, params ) - dataset.blurb = 'LAJ Multiple Alignment Viewer' + if not dataset.dataset.purged: + params = { + "alignfile1": "display?id=%s" % dataset.id, + "buttonlabel": "Launch LAJ", + "title": "LAJ in Galaxy", + "posturl": "history_add_to?%s" % urlencode( { 'history_id': dataset.history_id, 'ext': 'lav', 'name': 'LAJ Output', 'info': 'Added by LAJ', 'dbkey': dataset.dbkey } ), + "noseq": "true" + } + class_name = "edu.psu.cse.bio.laj.LajApplet.class" + archive = "/static/laj/laj.jar" + dataset.peek = create_applet_tag_peek( class_name, archive, params ) + dataset.blurb = 'LAJ Multiple Alignment Viewer' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/interval.py --- a/lib/galaxy/datatypes/interval.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/interval.py Thu Jan 15 10:49:01 2009 -0500 @@ -59,11 +59,15 @@ def set_peek( self, dataset, line_count=None ): """Set the peek and blurb text""" - dataset.peek = data.get_file_peek( dataset.file_name ) - if line_count is None: - dataset.blurb = "%s regions" % util.commaify( str( data.get_line_count( dataset.file_name ) ) ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + if line_count is None: + dataset.blurb = "%s regions" % util.commaify( str( data.get_line_count( dataset.file_name ) ) ) + else: + dataset.blurb = "%s regions" % util.commaify( str( line_count ) ) else: - dataset.blurb = "%s regions" % util.commaify( str( line_count ) ) + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def set_meta( self, dataset, overwrite = True, first_line_is_header = False, **kwd ): Tabular.set_meta( self, dataset, overwrite = overwrite, skip = 0 ) diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/qualityscore.py --- a/lib/galaxy/datatypes/qualityscore.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/qualityscore.py Thu Jan 15 10:49:01 2009 -0500 @@ -16,11 +16,15 @@ file_ext = "qual" def set_peek( self, dataset, line_count=None ): - dataset.peek = data.get_file_peek( dataset.file_name ) - if line_count is None: - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + if line_count is None: + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.blurb = "%s lines, Quality score file" % util.commaify( str( line_count ) ) else: - dataset.blurb = "%s lines, Quality score file" % util.commaify( str( line_count ) ) + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/sequence.py --- a/lib/galaxy/datatypes/sequence.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/sequence.py Thu Jan 15 10:49:01 2009 -0500 @@ -31,8 +31,12 @@ file_ext = "fasta" def set_peek( self, dataset ): - dataset.peek = data.get_file_peek( dataset.file_name ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ @@ -86,8 +90,12 @@ file_ext = "csfasta" def set_peek( self, dataset ): - dataset.peek = data.get_file_peek( dataset.file_name ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ @@ -108,8 +116,12 @@ file_ext = "fastqsolexa" def set_peek( self, dataset ): - dataset.peek = data.get_file_peek( dataset.file_name ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/xml.py --- a/lib/galaxy/datatypes/xml.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/xml.py Thu Jan 15 10:49:01 2009 -0500 @@ -12,8 +12,12 @@ file_ext = "blastxml" def set_peek( self, dataset ): """Set the peek and blurb text""" - dataset.peek = data.get_file_peek( dataset.file_name ) - dataset.blurb = 'NCBI Blast XML data' + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = 'NCBI Blast XML data' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ Determines whether the file is blastxml diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/model/__init__.py Thu Jan 15 10:49:01 2009 -0500 @@ -287,14 +287,26 @@ return self.datatype.find_conversion_destination( self, accepted_formats, datatypes_registry, **kwd ) def copy( self, copy_children = False, parent_id = None ): - des = HistoryDatasetAssociation( hid=self.hid, name=self.name, info=self.info, blurb=self.blurb, peek=self.peek, extension=self.extension, dbkey=self.dbkey, dataset = self.dataset, visible=self.visible, deleted=self.deleted, parent_id=parent_id, copied_from_history_dataset_association = self ) + des = HistoryDatasetAssociation( hid=self.hid, + name=self.name, + info=self.info, + blurb=self.blurb, + peek=self.peek, + extension=self.extension, + dbkey=self.dbkey, + dataset=self.dataset, + visible=self.visible, + deleted=self.deleted, + parent_id=parent_id, + copied_from_history_dataset_association=self ) des.flush() des.set_size() des.metadata = self.metadata #need to set after flushed, as MetadataFiles require dataset.id if copy_children: for child in self.children: child_copy = child.copy( copy_children = copy_children, parent_id = des.id ) - des.set_peek() #in some instances peek relies on dataset_id, i.e. gmaj.zip for viewing MAFs + # In some instances peek relies on dataset_id ( e.g., gmaj.zip for viewing MAFs ) + des.set_peek() des.flush() return des @@ -376,7 +388,7 @@ des.flush() des.name = self.name des.user_id = self.user_id - for data in self.active_datasets: + for data in self.datasets: new_data = data.copy( copy_children = True ) des.add_dataset( new_data ) new_data.flush() diff -r 4cf0aa26070f -r 78c2f403cb43 tools/data_source/echo.xml --- a/tools/data_source/echo.xml Thu Jan 15 10:42:22 2009 -0500 +++ b/tools/data_source/echo.xml Thu Jan 15 10:49:01 2009 -0500 @@ -6,14 +6,18 @@ echoes parameters </description> - <command interpreter="python">echo.py $input $output </command> + <command interpreter="python">echo.py $input $database $output </command> <inputs> <param format="tabular" name="input" type="data" label="Input stuff"/> + <param type="select" name="database" label="Database"> + <option value="alignseq.loc">Human (hg18)</option> + <option value="faseq.loc">Fly (dm3)</option> + </param> </inputs> <outputs> - <data format="input" name="output" /> + <data format="input" name="output" label="Blat on ${database.value_label}" /> </outputs> </tool>

1 0

[hg] galaxy 1707: Fixed a bug in microsatellite mutability tool.
by Greg Von Kuster 15 Jan '09

15 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/4cf0aa26070f changeset: 1707:4cf0aa26070f user: guru date: Thu Jan 15 10:42:22 2009 -0500 description: Fixed a bug in microsatellite mutability tool. 1 file(s) affected in this change: tools/regVariation/microsats_mutability.py diffs (23 lines): diff -r cfe79f4c6673 -r 4cf0aa26070f tools/regVariation/microsats_mutability.py --- a/tools/regVariation/microsats_mutability.py Wed Jan 14 17:20:22 2009 -0500 +++ b/tools/regVariation/microsats_mutability.py Thu Jan 15 10:42:22 2009 -0500 @@ -409,19 +409,9 @@ blk=0 win=0 linestr="" - ff=open("junkmabc","w") if region == 'win': - """ - sorted_infile = tempfile.NamedTemporaryFile() - if winspecies == 1: - cmdline = "sort -n -k 3"+" -o "+sorted_infile.name+" "+infile - elif winspecies == 2: - cmdline = "sort -n -k 10"+" -o "+sorted_infile.name+" "+infile - os.system(cmdline) - print >>ff, "Finished sorting" - """ msats = NiceReaderWrapper( fileinput.FileInput( infile ), chrom_col = speciesind, start_col = speciesind+1,

1 0

[hg] galaxy 1706: Add a SelectToolParameterWrapper object and a ...
by Greg Von Kuster 15 Jan '09

15 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/cfe79f4c6673 changeset: 1706:cfe79f4c6673 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Jan 14 17:20:22 2009 -0500 description: Add a SelectToolParameterWrapper object and a wrap method to Tool.execute() that allows for things like the following in tool configs: <outputs> <data format="input" name="output" label="Blat on {<input_param>.value_label}" /> </outputs> Also replace EOL chars with <br> in a tool's output data.info so that they are displayed in HTML. 3 file(s) affected in this change: lib/galaxy/datatypes/data.py lib/galaxy/tools/__init__.py lib/galaxy/tools/actions/__init__.py diffs (108 lines): diff -r 4a067f42c725 -r cfe79f4c6673 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py Wed Jan 14 16:15:51 2009 -0500 +++ b/lib/galaxy/datatypes/data.py Wed Jan 14 17:20:22 2009 -0500 @@ -132,7 +132,8 @@ def display_info(self, dataset): """Returns formated html of dataset info""" try: - return escape(dataset.info) + # Change new line chars to html + return escape( dataset.info ).replace( "\r", "\n" ).replace( "\n", "<br>" ) except: return "info unavailable" def validate(self, dataset): diff -r 4a067f42c725 -r cfe79f4c6673 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Wed Jan 14 16:15:51 2009 -0500 +++ b/lib/galaxy/tools/__init__.py Wed Jan 14 17:20:22 2009 -0500 @@ -1072,10 +1072,11 @@ datatypes_registry = self.app.datatypes_registry, tool = self, name = input.name ) + elif isinstance( input, SelectToolParameter ): + input_values[ input.name ] = SelectToolParameterWrapper( input, input_values[ input.name ], self.app ) else: - input_values[ input.name ] = \ - InputValueWrapper( input, input_values[ input.name ], param_dict ) - # HACK: only wrap if check_values is false, this deals with external + input_values[ input.name ] = InputValueWrapper( input, input_values[ input.name ], param_dict ) + # HACK: only wrap if check_values is not false, this deals with external # tools where the inputs don't even get passed through. These # tools (e.g. UCSC) should really be handled in a special way. if self.check_values: @@ -1416,7 +1417,21 @@ return self.input.to_param_dict_string( self.value, self._other_values ) def __getattr__( self, key ): return getattr( self.value, key ) - + +class SelectToolParameterWrapper( object ): + """ + Wraps a SelectTooParameter so that __str__ returns the selected value, but all other + attributes are accessible. + """ + def __init__( self, input, value, app ): + self.input = input + self.value = value + self.input.value_label = input.value_to_display_text( value, app ) + def __str__( self ): + return self.input.to_param_dict_string( self.value ) + def __getattr__( self, key ): + return getattr( self.input, key ) + class DatasetFilenameWrapper( object ): """ Wraps a dataset so that __str__ returns the filename, but all other diff -r 4a067f42c725 -r cfe79f4c6673 lib/galaxy/tools/actions/__init__.py --- a/lib/galaxy/tools/actions/__init__.py Wed Jan 14 16:15:51 2009 -0500 +++ b/lib/galaxy/tools/actions/__init__.py Wed Jan 14 17:20:22 2009 -0500 @@ -1,9 +1,11 @@ from galaxy.util.bunch import Bunch from galaxy.tools.parameters import * +from galaxy.tools.parameters.grouping import * from galaxy.util.template import fill_template from galaxy.util.none_like import NoneDataset from galaxy.web import url_for from galaxy.jobs import JOB_OK +import galaxy.tools import logging log = logging.getLogger( __name__ ) @@ -65,6 +67,26 @@ return input_datasets def execute(self, tool, trans, incoming={}, set_output_hid=True ): + def wrap_values( inputs, input_values ): + # Wrap tool inputs as necessary + for input in inputs.itervalues(): + if isinstance( input, Repeat ): + for d in input_values[ input.name ]: + wrap_values( input.inputs, d ) + elif isinstance( input, Conditional ): + values = input_values[ input.name ] + current = values["__current_case__"] + wrap_values( input.cases[current].inputs, values ) + elif isinstance( input, DataToolParameter ): + input_values[ input.name ] = \ + galaxy.tools.DatasetFilenameWrapper( input_values[ input.name ], + datatypes_registry = trans.app.datatypes_registry, + tool = tool, + name = input.name ) + elif isinstance( input, SelectToolParameter ): + input_values[ input.name ] = galaxy.tools.SelectToolParameterWrapper( input, input_values[ input.name ], tool.app ) + else: + input_values[ input.name ] = galaxy.tools.InputValueWrapper( input, input_values[ input.name ], incoming ) out_data = {} # Collect any input datasets from the incoming parameters inp_data = self.collect_input_datasets( tool, incoming, trans ) @@ -151,6 +173,11 @@ # Set output label if output.label: params = dict( incoming ) + # wrapping the params allows the tool config to contain things like + # <outputs> + # <data format="input" name="output" label="Blat on ${<input_param>.name}" /> + # </outputs> + wrap_values( tool.inputs, params ) params['tool'] = tool params['on_string'] = on_text data.name = fill_template( output.label, context=params )

1 0

[hg] galaxy 1705: When copying a history, only copy active_datas...
by Greg Von Kuster 14 Jan '09

14 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/4a067f42c725 changeset: 1705:4a067f42c725 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Jan 14 16:15:51 2009 -0500 description: When copying a history, only copy active_datasets - fixes sharing a history containing purged datasets. 1 file(s) affected in this change: lib/galaxy/model/__init__.py diffs (12 lines): diff -r 941266327bc3 -r 4a067f42c725 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py Wed Jan 14 10:49:14 2009 -0500 +++ b/lib/galaxy/model/__init__.py Wed Jan 14 16:15:51 2009 -0500 @@ -376,7 +376,7 @@ des.flush() des.name = self.name des.user_id = self.user_id - for data in self.datasets: + for data in self.active_datasets: new_data = data.copy( copy_children = True ) des.add_dataset( new_data ) new_data.flush()

1 0