- galaxy-dev - lists.galaxyproject.org

Adding the 'name' content of the selection parameter
by Assaf Gordon 15 Jan '09

15 Jan '09

Hello, Continuing the discussion from Dec. 19th, http://mail.bx.psu.edu/pipermail/galaxy-user/2008-December/000422.html The following hack adds an additional key/value which contains the 'name' of the selected option in a <select> input parameter. With this hack, the following tool configuration is possible (note the output/label option): ========================== <tool> .. <inputs> <input format="fasta" type="data" name="input"> <input type="select" name="database" label="Database"> <option value="/home/gordon/long/path/hg18.fa">Human (hg18)</option> <option value="/home/gordon/long/path/dm3.fa">Fly (dm3)</option> <option value="/home/gordon/long/path/mm9.fa">Mouse (mm9)</option> </input> </inputs> <outputs>  <data format="txt" name="output" label="Blat $input1 on $database" />  <data format="txt" name="output" label="Blat $input1 on $database_name" /> </outputs> ========================== Currently, using "$database" in the output/label part will put the *value* of the selected option (e.g. "/home/gordon/long/path/hg18.fa") in the dataset's name. This doesn't really help the user understand what the dataset contains. With this hack, for each <select> input, a new variable is created (with the "_name" suffix) which will contain the *name* of the selected option (e.g. "Human (hg18)"). For example, if the user selected the second option, Then "$database" will contain "/home/gordon/long/path/dm3.fa", and "$database_name" will contain "Fly (dm3)". I'm well aware this is an ugly hack. A better way would be to have each parameter (in the incoming dictionary) be an object (e.g. ToolParameter) and not a string. Then we could write something like "$database.name" or "$database.value". The hack contains two patches. The first add support for 'value_to_display_text' with dynamic options, in "Parameters/Basic.py": ============================================================== --- galaxy_prod/lib/galaxy/tools/parameters/basic.py +++ galaxy_devel/lib/galaxy/tools/parameters/basic.py @@ -576,7 +576,7 @@ elif len( value ) == 1: value = value[0] return value - def value_to_display_text( self, value, app ): + def value_to_display_text( self, value, app, other_values={} ): if isinstance( value, UnvalidatedValue ): suffix = "\n(value not yet validated)" value = value.value @@ -584,10 +584,11 @@ suffix = "" if not isinstance( value, list ): value = [ value ] - # FIXME: Currently only translating values back to labels if they - # are not dynamic if self.is_dynamic: - rval = map( str, value ) + rval = [ ] + for name in self.options.get_options ( app , other_values ) : + if ( name[1] in value ): + rval.append ( name[0] ) else: options = list( self.static_options ) rval = [] ============================================================== The second patch iterates the tool's parameters, and for each <select> type parameter, adds the additional key/value for the selected option: ============================================================== --- galaxy_prod/lib/galaxy/tools/actions/__init__.py +++ galaxy_devel/lib/galaxy/tools/actions/__init__.py @@ -96,6 +96,18 @@ on_text = "" # Add the dbkey to the incoming parameters incoming[ "dbkey" ] = input_dbkey + + ##28dec2008, gordon + ## For every 'Select' parameter, we get only the 'value' part of the selection from the HTTP request. + ## The following code gets the 'name' part for the selected option + selection_parameter_names = {} + for param_name, param_selected_value in incoming.iteritems(): + param_obj = tool.get_param ( param_name ) + if isinstance( param_obj, basic.SelectToolParameter): + param_selected_name = param_obj.value_to_display_text( param_selected_value, trans, incoming ) + selection_parameter_names [ param_name + "_name" ] = param_selected_name.replace("\n","") + incoming.update( selection_parameter_names ) + # Keep track of parent / child relationships, we'll create all the # datasets first, then create the associations parent_to_child_pairs = [] ============================================================== I tried to make the changes as non intrusive as possible. I hope the added 'other_value' parameter doesn't affect other places that call 'value_to_display_text' - but I'm not quite sure about it (I'm still very new to python). I've tested this hack with several tools, and it seems to work (or at least not crash horribly). Comments are welcomed, Gordon.

2 1

request: add newlines to dataset's info field
by Assaf Gordon 15 Jan '09

15 Jan '09

Hello, I'd like to request/suggest a tiny change in the dataset's info field. Whenever a tool outputs some lines into STDOUT, they are displayed in the dataset's info field. Currently, if the info field contains several lines, they are displayed as a single line (because HTML ignores new lines). The following patch causes the info field to be displayed correctly. ================================================================= --- galaxy_prod/lib/galaxy/datatypes/data.py +++ galaxy_devel/lib/galaxy/datatypes/data.py @@ -132,7 +132,7 @@ def display_info(self, dataset): """Returns formated html of dataset info""" try: - return escape(dataset.info) + return escape(dataset.info).replace("\n", "<br>") except: return "info unavailable" def validate(self, dataset): ================================================================== Thanks, gordon.

2 1

[hg] galaxy 1708: Change History.copy() back to use datasets rat...
by Greg Von Kuster 15 Jan '09

15 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/78c2f403cb43 changeset: 1708:78c2f403cb43 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Thu Jan 15 10:49:01 2009 -0500 description: Change History.copy() back to use datasets rather than active_datasets. Add checks in set_peek() to properly set peek on purged datasets. Fixes sharing a history that contains purged datasets. 9 file(s) affected in this change: lib/galaxy/datatypes/data.py lib/galaxy/datatypes/genetics.py lib/galaxy/datatypes/images.py lib/galaxy/datatypes/interval.py lib/galaxy/datatypes/qualityscore.py lib/galaxy/datatypes/sequence.py lib/galaxy/datatypes/xml.py lib/galaxy/model/__init__.py tools/data_source/echo.xml diffs (441 lines): diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/data.py Thu Jan 15 10:49:01 2009 -0500 @@ -103,8 +103,12 @@ return False def set_peek( self, dataset ): """Set the peek and blurb text""" - dataset.peek = '' - dataset.blurb = 'data' + if not dataset.dataset.purged: + dataset.peek = '' + dataset.blurb = 'data' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): """Create HTML table, used for displaying peek""" out = ['<table cellspacing="0" cellpadding="3">'] @@ -275,19 +279,27 @@ return 'text/plain' def set_peek( self, dataset, line_count=None ): - dataset.peek = get_file_peek( dataset.file_name ) - if line_count is None: - dataset.blurb = "%s lines" % util.commaify( str( get_line_count( dataset.file_name ) ) ) + if not dataset.dataset.purged: + # The file must exist on disk for the get_file_peek() method + dataset.peek = get_file_peek( dataset.file_name ) + if line_count is None: + dataset.blurb = "%s lines" % util.commaify( str( get_line_count( dataset.file_name ) ) ) + else: + dataset.blurb = "%s lines" % util.commaify( str( line_count ) ) else: - dataset.blurb = "%s lines" % util.commaify( str( line_count ) ) + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' class Binary( Data ): """Binary data""" - def set_peek( self, dataset ): """Set the peek and blurb text""" - dataset.peek = 'binary data' - dataset.blurb = 'data' + if not dataset.dataset.purged: + dataset.peek = 'binary data' + dataset.blurb = 'data' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def get_test_fname( fname ): """Returns test data filename""" diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/genetics.py --- a/lib/galaxy/datatypes/genetics.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/genetics.py Thu Jan 15 10:49:01 2009 -0500 @@ -43,11 +43,14 @@ def set_peek( self, dataset ): """Set the peek and blurb text""" - dataset.peek = data.get_file_peek( dataset.file_name ) - ## dataset.peek = self.make_html_table( dataset.peek ) - dataset.blurb = util.commaify( str( data.get_line_count( dataset.file_name ) ) ) + " rows" - #i don't think set_meta should not be called here, it should be called separately - self.set_meta( dataset ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = util.commaify( str( data.get_line_count( dataset.file_name ) ) ) + " rows" + #i don't think set_meta should not be called here, it should be called separately + self.set_meta( dataset ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def get_estimated_display_viewport( self, dataset ): """Return a chrom, start, stop tuple for viewing a file.""" @@ -130,8 +133,12 @@ file_ext="snpmatrix" def set_peek( self, dataset ): - dataset.peek = "Binary RGenetics file" - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = "Binary RGenetics file" + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ """ diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/images.py --- a/lib/galaxy/datatypes/images.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/images.py Thu Jan 15 10:49:01 2009 -0500 @@ -14,9 +14,13 @@ """Class describing an ab1 binary sequence file""" file_ext = "ab1" def set_peek( self, dataset ): - export_url = "/history_add_to?"+urlencode({'history_id':dataset.history_id,'ext':'ab1','name':'ab1 sequence','info':'Sequence file','dbkey':dataset.dbkey}) - dataset.peek = "Binary ab1 sequence file" - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + export_url = "/history_add_to?" + urlencode({'history_id':dataset.history_id,'ext':'ab1','name':'ab1 sequence','info':'Sequence file','dbkey':dataset.dbkey}) + dataset.peek = "Binary ab1 sequence file" + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -27,9 +31,13 @@ """Class describing an scf binary sequence file""" file_ext = "scf" def set_peek( self, dataset ): - export_url = "/history_add_to?"+urlencode({'history_id':dataset.history_id,'ext':'scf','name':'scf sequence','info':'Sequence file','dbkey':dataset.dbkey}) - dataset.peek = "Binary scf sequence file" - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + export_url = "/history_add_to?" + urlencode({'history_id':dataset.history_id,'ext':'scf','name':'scf sequence','info':'Sequence file','dbkey':dataset.dbkey}) + dataset.peek = "Binary scf sequence file" + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -40,10 +48,14 @@ """Class describing a zip archive of binary sequence files""" file_ext = "binseq.zip" def set_peek( self, dataset ): - zip_file = zipfile.ZipFile( dataset.file_name, "r" ) - num_files = len( zip_file.namelist() ) - dataset.peek = "Archive of %s binary sequence files" % ( str( num_files ) ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + zip_file = zipfile.ZipFile( dataset.file_name, "r" ) + num_files = len( zip_file.namelist() ) + dataset.peek = "Archive of %s binary sequence files" % ( str( num_files ) ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -57,10 +69,14 @@ """Class describing a zip archive of text sequence files""" file_ext = "txtseq.zip" def set_peek( self, dataset ): - zip_file = zipfile.ZipFile( dataset.file_name, "r" ) - num_files = len( zip_file.namelist() ) - dataset.peek = "Archive of %s text sequence files" % ( str( num_files ) ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + zip_file = zipfile.ZipFile( dataset.file_name, "r" ) + num_files = len( zip_file.namelist() ) + dataset.peek = "Archive of %s text sequence files" % ( str( num_files ) ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -73,8 +89,12 @@ class Image( data.Data ): """Class describing an image""" def set_peek( self, dataset ): - dataset.peek = 'Image in %s format' % dataset.extension - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = 'Image in %s format' % dataset.extension + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def create_applet_tag_peek( class_name, archive, params ): text = """ @@ -104,18 +124,22 @@ """Class describing a GMAJ Applet""" file_ext = "gmaj.zip" def set_peek( self, dataset ): - params = { - "bundle":"display?id=%s&tofile=yes&toext=.zip" % dataset.id, - "buttonlabel": "Launch GMAJ", - "nobutton": "false", - "urlpause" :"100", - "debug": "false", - "posturl": "history_add_to?%s" % urlencode( { 'history_id': dataset.history_id, 'ext': 'maf', 'name': 'GMAJ Output on data %s' % dataset.hid, 'info': 'Added by GMAJ', 'dbkey': dataset.dbkey } ) - } - class_name = "edu.psu.bx.gmaj.MajApplet.class" - archive = "/static/gmaj/gmaj.jar" - dataset.peek = create_applet_tag_peek( class_name, archive, params ) - dataset.blurb = 'GMAJ Multiple Alignment Viewer' + if not dataset.dataset.purged: + params = { + "bundle":"display?id=%s&tofile=yes&toext=.zip" % dataset.id, + "buttonlabel": "Launch GMAJ", + "nobutton": "false", + "urlpause" :"100", + "debug": "false", + "posturl": "history_add_to?%s" % urlencode( { 'history_id': dataset.history_id, 'ext': 'maf', 'name': 'GMAJ Output on data %s' % dataset.hid, 'info': 'Added by GMAJ', 'dbkey': dataset.dbkey } ) + } + class_name = "edu.psu.bx.gmaj.MajApplet.class" + archive = "/static/gmaj/gmaj.jar" + dataset.peek = create_applet_tag_peek( class_name, archive, params ) + dataset.blurb = 'GMAJ Multiple Alignment Viewer' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek @@ -147,8 +171,12 @@ """Class describing an html file""" file_ext = "html" def set_peek( self, dataset ): - dataset.peek = "HTML file" - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = "HTML file" + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def get_mime(self): """Returns the mime type of the datatype""" return 'text/html' @@ -176,17 +204,21 @@ """Class describing a LAJ Applet""" file_ext = "laj" def set_peek( self, dataset ): - params = { - "alignfile1": "display?id=%s" % dataset.id, - "buttonlabel": "Launch LAJ", - "title": "LAJ in Galaxy", - "posturl": "history_add_to?%s" % urlencode( { 'history_id': dataset.history_id, 'ext': 'lav', 'name': 'LAJ Output', 'info': 'Added by LAJ', 'dbkey': dataset.dbkey } ), - "noseq": "true" - } - class_name = "edu.psu.cse.bio.laj.LajApplet.class" - archive = "/static/laj/laj.jar" - dataset.peek = create_applet_tag_peek( class_name, archive, params ) - dataset.blurb = 'LAJ Multiple Alignment Viewer' + if not dataset.dataset.purged: + params = { + "alignfile1": "display?id=%s" % dataset.id, + "buttonlabel": "Launch LAJ", + "title": "LAJ in Galaxy", + "posturl": "history_add_to?%s" % urlencode( { 'history_id': dataset.history_id, 'ext': 'lav', 'name': 'LAJ Output', 'info': 'Added by LAJ', 'dbkey': dataset.dbkey } ), + "noseq": "true" + } + class_name = "edu.psu.cse.bio.laj.LajApplet.class" + archive = "/static/laj/laj.jar" + dataset.peek = create_applet_tag_peek( class_name, archive, params ) + dataset.blurb = 'LAJ Multiple Alignment Viewer' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: return dataset.peek diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/interval.py --- a/lib/galaxy/datatypes/interval.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/interval.py Thu Jan 15 10:49:01 2009 -0500 @@ -59,11 +59,15 @@ def set_peek( self, dataset, line_count=None ): """Set the peek and blurb text""" - dataset.peek = data.get_file_peek( dataset.file_name ) - if line_count is None: - dataset.blurb = "%s regions" % util.commaify( str( data.get_line_count( dataset.file_name ) ) ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + if line_count is None: + dataset.blurb = "%s regions" % util.commaify( str( data.get_line_count( dataset.file_name ) ) ) + else: + dataset.blurb = "%s regions" % util.commaify( str( line_count ) ) else: - dataset.blurb = "%s regions" % util.commaify( str( line_count ) ) + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def set_meta( self, dataset, overwrite = True, first_line_is_header = False, **kwd ): Tabular.set_meta( self, dataset, overwrite = overwrite, skip = 0 ) diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/qualityscore.py --- a/lib/galaxy/datatypes/qualityscore.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/qualityscore.py Thu Jan 15 10:49:01 2009 -0500 @@ -16,11 +16,15 @@ file_ext = "qual" def set_peek( self, dataset, line_count=None ): - dataset.peek = data.get_file_peek( dataset.file_name ) - if line_count is None: - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + if line_count is None: + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.blurb = "%s lines, Quality score file" % util.commaify( str( line_count ) ) else: - dataset.blurb = "%s lines, Quality score file" % util.commaify( str( line_count ) ) + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def display_peek(self, dataset): try: diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/sequence.py --- a/lib/galaxy/datatypes/sequence.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/sequence.py Thu Jan 15 10:49:01 2009 -0500 @@ -31,8 +31,12 @@ file_ext = "fasta" def set_peek( self, dataset ): - dataset.peek = data.get_file_peek( dataset.file_name ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ @@ -86,8 +90,12 @@ file_ext = "csfasta" def set_peek( self, dataset ): - dataset.peek = data.get_file_peek( dataset.file_name ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ @@ -108,8 +116,12 @@ file_ext = "fastqsolexa" def set_peek( self, dataset ): - dataset.peek = data.get_file_peek( dataset.file_name ) - dataset.blurb = data.nice_size( dataset.get_size() ) + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = data.nice_size( dataset.get_size() ) + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/datatypes/xml.py --- a/lib/galaxy/datatypes/xml.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/datatypes/xml.py Thu Jan 15 10:49:01 2009 -0500 @@ -12,8 +12,12 @@ file_ext = "blastxml" def set_peek( self, dataset ): """Set the peek and blurb text""" - dataset.peek = data.get_file_peek( dataset.file_name ) - dataset.blurb = 'NCBI Blast XML data' + if not dataset.dataset.purged: + dataset.peek = data.get_file_peek( dataset.file_name ) + dataset.blurb = 'NCBI Blast XML data' + else: + dataset.peek = 'file does not exist' + dataset.blurb = 'file purged from disk' def sniff( self, filename ): """ Determines whether the file is blastxml diff -r 4cf0aa26070f -r 78c2f403cb43 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py Thu Jan 15 10:42:22 2009 -0500 +++ b/lib/galaxy/model/__init__.py Thu Jan 15 10:49:01 2009 -0500 @@ -287,14 +287,26 @@ return self.datatype.find_conversion_destination( self, accepted_formats, datatypes_registry, **kwd ) def copy( self, copy_children = False, parent_id = None ): - des = HistoryDatasetAssociation( hid=self.hid, name=self.name, info=self.info, blurb=self.blurb, peek=self.peek, extension=self.extension, dbkey=self.dbkey, dataset = self.dataset, visible=self.visible, deleted=self.deleted, parent_id=parent_id, copied_from_history_dataset_association = self ) + des = HistoryDatasetAssociation( hid=self.hid, + name=self.name, + info=self.info, + blurb=self.blurb, + peek=self.peek, + extension=self.extension, + dbkey=self.dbkey, + dataset=self.dataset, + visible=self.visible, + deleted=self.deleted, + parent_id=parent_id, + copied_from_history_dataset_association=self ) des.flush() des.set_size() des.metadata = self.metadata #need to set after flushed, as MetadataFiles require dataset.id if copy_children: for child in self.children: child_copy = child.copy( copy_children = copy_children, parent_id = des.id ) - des.set_peek() #in some instances peek relies on dataset_id, i.e. gmaj.zip for viewing MAFs + # In some instances peek relies on dataset_id ( e.g., gmaj.zip for viewing MAFs ) + des.set_peek() des.flush() return des @@ -376,7 +388,7 @@ des.flush() des.name = self.name des.user_id = self.user_id - for data in self.active_datasets: + for data in self.datasets: new_data = data.copy( copy_children = True ) des.add_dataset( new_data ) new_data.flush() diff -r 4cf0aa26070f -r 78c2f403cb43 tools/data_source/echo.xml --- a/tools/data_source/echo.xml Thu Jan 15 10:42:22 2009 -0500 +++ b/tools/data_source/echo.xml Thu Jan 15 10:49:01 2009 -0500 @@ -6,14 +6,18 @@ echoes parameters </description> - <command interpreter="python">echo.py $input $output </command> + <command interpreter="python">echo.py $input $database $output </command> <inputs> <param format="tabular" name="input" type="data" label="Input stuff"/> + <param type="select" name="database" label="Database"> + <option value="alignseq.loc">Human (hg18)</option> + <option value="faseq.loc">Fly (dm3)</option> + </param> </inputs> <outputs> - <data format="input" name="output" /> + <data format="input" name="output" label="Blat on ${database.value_label}" /> </outputs> </tool>

1 0

[hg] galaxy 1706: Add a SelectToolParameterWrapper object and a ...
by Greg Von Kuster 15 Jan '09

15 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/cfe79f4c6673 changeset: 1706:cfe79f4c6673 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Jan 14 17:20:22 2009 -0500 description: Add a SelectToolParameterWrapper object and a wrap method to Tool.execute() that allows for things like the following in tool configs: <outputs> <data format="input" name="output" label="Blat on {<input_param>.value_label}" /> </outputs> Also replace EOL chars with <br> in a tool's output data.info so that they are displayed in HTML. 3 file(s) affected in this change: lib/galaxy/datatypes/data.py lib/galaxy/tools/__init__.py lib/galaxy/tools/actions/__init__.py diffs (108 lines): diff -r 4a067f42c725 -r cfe79f4c6673 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py Wed Jan 14 16:15:51 2009 -0500 +++ b/lib/galaxy/datatypes/data.py Wed Jan 14 17:20:22 2009 -0500 @@ -132,7 +132,8 @@ def display_info(self, dataset): """Returns formated html of dataset info""" try: - return escape(dataset.info) + # Change new line chars to html + return escape( dataset.info ).replace( "\r", "\n" ).replace( "\n", "<br>" ) except: return "info unavailable" def validate(self, dataset): diff -r 4a067f42c725 -r cfe79f4c6673 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Wed Jan 14 16:15:51 2009 -0500 +++ b/lib/galaxy/tools/__init__.py Wed Jan 14 17:20:22 2009 -0500 @@ -1072,10 +1072,11 @@ datatypes_registry = self.app.datatypes_registry, tool = self, name = input.name ) + elif isinstance( input, SelectToolParameter ): + input_values[ input.name ] = SelectToolParameterWrapper( input, input_values[ input.name ], self.app ) else: - input_values[ input.name ] = \ - InputValueWrapper( input, input_values[ input.name ], param_dict ) - # HACK: only wrap if check_values is false, this deals with external + input_values[ input.name ] = InputValueWrapper( input, input_values[ input.name ], param_dict ) + # HACK: only wrap if check_values is not false, this deals with external # tools where the inputs don't even get passed through. These # tools (e.g. UCSC) should really be handled in a special way. if self.check_values: @@ -1416,7 +1417,21 @@ return self.input.to_param_dict_string( self.value, self._other_values ) def __getattr__( self, key ): return getattr( self.value, key ) - + +class SelectToolParameterWrapper( object ): + """ + Wraps a SelectTooParameter so that __str__ returns the selected value, but all other + attributes are accessible. + """ + def __init__( self, input, value, app ): + self.input = input + self.value = value + self.input.value_label = input.value_to_display_text( value, app ) + def __str__( self ): + return self.input.to_param_dict_string( self.value ) + def __getattr__( self, key ): + return getattr( self.input, key ) + class DatasetFilenameWrapper( object ): """ Wraps a dataset so that __str__ returns the filename, but all other diff -r 4a067f42c725 -r cfe79f4c6673 lib/galaxy/tools/actions/__init__.py --- a/lib/galaxy/tools/actions/__init__.py Wed Jan 14 16:15:51 2009 -0500 +++ b/lib/galaxy/tools/actions/__init__.py Wed Jan 14 17:20:22 2009 -0500 @@ -1,9 +1,11 @@ from galaxy.util.bunch import Bunch from galaxy.tools.parameters import * +from galaxy.tools.parameters.grouping import * from galaxy.util.template import fill_template from galaxy.util.none_like import NoneDataset from galaxy.web import url_for from galaxy.jobs import JOB_OK +import galaxy.tools import logging log = logging.getLogger( __name__ ) @@ -65,6 +67,26 @@ return input_datasets def execute(self, tool, trans, incoming={}, set_output_hid=True ): + def wrap_values( inputs, input_values ): + # Wrap tool inputs as necessary + for input in inputs.itervalues(): + if isinstance( input, Repeat ): + for d in input_values[ input.name ]: + wrap_values( input.inputs, d ) + elif isinstance( input, Conditional ): + values = input_values[ input.name ] + current = values["__current_case__"] + wrap_values( input.cases[current].inputs, values ) + elif isinstance( input, DataToolParameter ): + input_values[ input.name ] = \ + galaxy.tools.DatasetFilenameWrapper( input_values[ input.name ], + datatypes_registry = trans.app.datatypes_registry, + tool = tool, + name = input.name ) + elif isinstance( input, SelectToolParameter ): + input_values[ input.name ] = galaxy.tools.SelectToolParameterWrapper( input, input_values[ input.name ], tool.app ) + else: + input_values[ input.name ] = galaxy.tools.InputValueWrapper( input, input_values[ input.name ], incoming ) out_data = {} # Collect any input datasets from the incoming parameters inp_data = self.collect_input_datasets( tool, incoming, trans ) @@ -151,6 +173,11 @@ # Set output label if output.label: params = dict( incoming ) + # wrapping the params allows the tool config to contain things like + # <outputs> + # <data format="input" name="output" label="Blat on ${<input_param>.name}" /> + # </outputs> + wrap_values( tool.inputs, params ) params['tool'] = tool params['on_string'] = on_text data.name = fill_template( output.label, context=params )

1 0

[hg] galaxy 1707: Fixed a bug in microsatellite mutability tool.
by Greg Von Kuster 15 Jan '09

15 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/4cf0aa26070f changeset: 1707:4cf0aa26070f user: guru date: Thu Jan 15 10:42:22 2009 -0500 description: Fixed a bug in microsatellite mutability tool. 1 file(s) affected in this change: tools/regVariation/microsats_mutability.py diffs (23 lines): diff -r cfe79f4c6673 -r 4cf0aa26070f tools/regVariation/microsats_mutability.py --- a/tools/regVariation/microsats_mutability.py Wed Jan 14 17:20:22 2009 -0500 +++ b/tools/regVariation/microsats_mutability.py Thu Jan 15 10:42:22 2009 -0500 @@ -409,19 +409,9 @@ blk=0 win=0 linestr="" - ff=open("junkmabc","w") if region == 'win': - """ - sorted_infile = tempfile.NamedTemporaryFile() - if winspecies == 1: - cmdline = "sort -n -k 3"+" -o "+sorted_infile.name+" "+infile - elif winspecies == 2: - cmdline = "sort -n -k 10"+" -o "+sorted_infile.name+" "+infile - os.system(cmdline) - print >>ff, "Finished sorting" - """ msats = NiceReaderWrapper( fileinput.FileInput( infile ), chrom_col = speciesind, start_col = speciesind+1,

1 0

[hg] galaxy 1705: When copying a history, only copy active_datas...
by Greg Von Kuster 14 Jan '09

14 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/4a067f42c725 changeset: 1705:4a067f42c725 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Jan 14 16:15:51 2009 -0500 description: When copying a history, only copy active_datasets - fixes sharing a history containing purged datasets. 1 file(s) affected in this change: lib/galaxy/model/__init__.py diffs (12 lines): diff -r 941266327bc3 -r 4a067f42c725 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py Wed Jan 14 10:49:14 2009 -0500 +++ b/lib/galaxy/model/__init__.py Wed Jan 14 16:15:51 2009 -0500 @@ -376,7 +376,7 @@ des.flush() des.name = self.name des.user_id = self.user_id - for data in self.datasets: + for data in self.active_datasets: new_data = data.copy( copy_children = True ) des.add_dataset( new_data ) new_data.flush()

1 0

[hg] galaxy 1704: Improvements to job recovery and clarification...
by Nate Coraor 14 Jan '09

14 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/941266327bc3 changeset: 1704:941266327bc3 user: Nate Coraor <nate(a)bx.psu.edu> date: Wed Jan 14 10:49:14 2009 -0500 description: Improvements to job recovery and clarification of the false path's job id 2 file(s) affected in this change: lib/galaxy/jobs/__init__.py lib/galaxy/tools/__init__.py diffs (79 lines): diff -r c259ce9cb50d -r 941266327bc3 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Wed Jan 14 08:50:31 2009 -0500 +++ b/lib/galaxy/jobs/__init__.py Wed Jan 14 10:49:14 2009 -0500 @@ -111,19 +111,19 @@ def __check_jobs_at_startup( self ): """ - Checks all jobs that are in the 'running' or 'queued' state in the - database and requeues or cleans up as necessary. Only run as the + Checks all jobs that are in the 'new', 'queued' or 'running' state in + the database and requeues or cleans up as necessary. Only run as the job manager starts. """ model = self.app.model - # Jobs in the NEW state won't be requeued unless we're tracking in the database - if not self.track_jobs_in_database: - for job in model.Job.filter( model.Job.c.state==model.Job.states.NEW ).all(): - log.debug( "no runner: %s is still in new state, adding to the jobs queue" %job.id ) + for job in model.Job.filter( model.Job.c.state==model.Job.states.NEW ).all(): + log.debug( "no runner: %s is still in new state, adding to the jobs queue" %job.id ) + self.queue.put( ( job.id, job.tool_id ) ) + for job in model.Job.filter( (model.Job.c.state == model.Job.states.RUNNING) | (model.Job.c.state == model.Job.states.QUEUED) ).all(): + if job.job_runner_name is None: + log.debug( "no runner: %s is still in queued state, adding to the jobs queue" %job.id ) self.queue.put( ( job.id, job.tool_id ) ) - for job in model.Job.filter( (model.Job.c.state == model.Job.states.RUNNING) | (model.Job.c.state == model.Job.states.QUEUED) ).all(): - if job.job_runner_name is not None: - # why are we passing the queue to the wrapper? + else: job_wrapper = JobWrapper( job, self.app.toolbox.tools_by_id[ job.tool_id ], self ) self.dispatcher.recover( job, job_wrapper ) @@ -383,12 +383,12 @@ if not job.state == model.Job.states.DELETED: for dataset_assoc in job.output_datasets: if self.app.config.outputs_to_working_directory: - false_path = os.path.abspath( os.path.join( self.working_directory, "galaxy_dataset_%d.dat" % dataset_assoc.dataset.id ) ) - if os.path.exists( false_path ): + false_path = os.path.abspath( os.path.join( self.working_directory, "galaxy_dataset_%d.dat" % dataset_assoc.dataset.dataset.id ) ) + try: shutil.move( false_path, dataset_assoc.dataset.file_name ) log.debug( "fail(): Moved %s to %s" % ( false_path, dataset_assoc.dataset.file_name ) ) - else: - log.warning( "fail(): Missing output file in working directory: %s" % false_path ) + except ( IOError, OSError ), e: + log.error( "fail(): Missing output file in working directory: %s" % e ) dataset = dataset_assoc.dataset dataset.refresh() dataset.state = dataset.states.ERROR @@ -453,12 +453,13 @@ job.state = 'ok' for dataset_assoc in job.output_datasets: if self.app.config.outputs_to_working_directory: - false_path = os.path.abspath( os.path.join( self.working_directory, "galaxy_dataset_%d.dat" % dataset_assoc.dataset.id ) ) - if os.path.exists( false_path ): + false_path = os.path.abspath( os.path.join( self.working_directory, "galaxy_dataset_%d.dat" % dataset_assoc.dataset.dataset.id ) ) + try: shutil.move( false_path, dataset_assoc.dataset.file_name ) log.debug( "finish(): Moved %s to %s" % ( false_path, dataset_assoc.dataset.file_name ) ) - else: - log.warning( "finish(): Missing output file in working directory: %s" % false_path ) + except ( IOError, OSError ): + self.fail( "The job's output dataset(s) could not be read" ) + return for dataset in dataset_assoc.dataset.dataset.history_associations: #need to update all associated output hdas, i.e. history was shared with job running dataset.blurb = 'done' dataset.peek = 'no peek' diff -r c259ce9cb50d -r 941266327bc3 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Wed Jan 14 08:50:31 2009 -0500 +++ b/lib/galaxy/tools/__init__.py Wed Jan 14 10:49:14 2009 -0500 @@ -1098,7 +1098,7 @@ for name, data in output_datasets.items(): # Write outputs to the working directory (for security purposes) if desired. if self.app.config.outputs_to_working_directory and working_directory is not None: - false_path = os.path.abspath( os.path.join( working_directory, "galaxy_dataset_%d.dat" % data.id ) ) + false_path = os.path.abspath( os.path.join( working_directory, "galaxy_dataset_%d.dat" % data.dataset.id ) ) param_dict[name] = DatasetFilenameWrapper( data, false_path = false_path ) open( false_path, 'w' ).close() else:

1 0

[hg] galaxy 1702: Set the working directory in the wrapper when ...
by Nate Coraor 14 Jan '09

14 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/b868c8d902d2 changeset: 1702:b868c8d902d2 user: Nate Coraor <nate(a)bx.psu.edu> date: Wed Jan 14 08:43:00 2009 -0500 description: Set the working directory in the wrapper when recovering jobs at startup. 1 file(s) affected in this change: lib/galaxy/jobs/__init__.py diffs (12 lines): diff -r c3a20d503bd3 -r b868c8d902d2 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Tue Jan 13 12:18:08 2009 -0500 +++ b/lib/galaxy/jobs/__init__.py Wed Jan 14 08:43:00 2009 -0500 @@ -125,6 +125,8 @@ if job.job_runner_name is not None: # why are we passing the queue to the wrapper? job_wrapper = JobWrapper( job, self.app.toolbox.tools_by_id[ job.tool_id ], self ) + job_wrapper.working_directory = \ + os.path.join( self.app.config.job_working_directory, str( job.id ) ) self.dispatcher.recover( job, job_wrapper ) def __monitor( self ):

1 0

[hg] galaxy 1703: Move setting of the working directory to the w...
by Nate Coraor 14 Jan '09

14 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/c259ce9cb50d changeset: 1703:c259ce9cb50d user: Nate Coraor <nate(a)bx.psu.edu> date: Wed Jan 14 08:50:31 2009 -0500 description: Move setting of the working directory to the wrapper's init method instead. 1 file(s) affected in this change: lib/galaxy/jobs/__init__.py diffs (37 lines): diff -r b868c8d902d2 -r c259ce9cb50d lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Wed Jan 14 08:43:00 2009 -0500 +++ b/lib/galaxy/jobs/__init__.py Wed Jan 14 08:50:31 2009 -0500 @@ -125,8 +125,6 @@ if job.job_runner_name is not None: # why are we passing the queue to the wrapper? job_wrapper = JobWrapper( job, self.app.toolbox.tools_by_id[ job.tool_id ], self ) - job_wrapper.working_directory = \ - os.path.join( self.app.config.job_working_directory, str( job.id ) ) self.dispatcher.recover( job, job_wrapper ) def __monitor( self ): @@ -300,9 +298,13 @@ self.queue = queue self.app = queue.app self.extra_filenames = [] - self.working_directory = None self.command_line = None self.galaxy_lib_dir = None + # With job outputs in the working directory, we need the working + # directory to be set before prepare is run, or else premature deletion + # and job recovery fail. + self.working_directory = \ + os.path.join( self.app.config.job_working_directory, str( self.job_id ) ) def get_param_dict( self ): """ @@ -319,9 +321,6 @@ config files. """ mapping.context.current.clear() #this prevents the metadata reverting that has been seen in conjunction with the PBS job runner - # Create the working directory - self.working_directory = \ - os.path.join( self.app.config.job_working_directory, str( self.job_id ) ) if not os.path.exists( self.working_directory ): os.mkdir( self.working_directory ) # Restore parameters from the database

1 0

[hg] galaxy 1700: Eliminate boilerplate code to track memory_usa...
by Greg Von Kuster 13 Jan '09

13 Jan '09

details: http://www.bx.psu.edu/hg/galaxy/rev/03f1e9e78f74 changeset: 1700:03f1e9e78f74 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Tue Jan 13 11:33:17 2009 -0500 description: Eliminate boilerplate code to track memory_usage, keeping the class and config settings for possible future use. 5 file(s) affected in this change: lib/galaxy/jobs/__init__.py lib/galaxy/tools/actions/__init__.py lib/galaxy/tools/actions/upload.py lib/galaxy/web/controllers/root.py lib/galaxy/web/controllers/user.py diffs (446 lines): diff -r 511c2b1966dc -r 03f1e9e78f74 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Tue Jan 13 10:55:08 2009 -0500 +++ b/lib/galaxy/jobs/__init__.py Tue Jan 13 11:33:17 2009 -0500 @@ -246,9 +246,6 @@ job can be dispatched. Otherwise, return JOB_WAIT indicating that input datasets are still being prepared. """ - if self.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() for dataset_assoc in job.input_datasets: idata = dataset_assoc.dataset if not idata: @@ -264,9 +261,6 @@ elif idata.state != idata.states.OK: # need to requeue return JOB_WAIT - if self.app.memory_usage: - m1 = self.app.memory_usage.memory( m0, pretty=True ) - log.info("End of check_if_ready_to_run for job id %d, memory used increased by %s" % ( job.id, m1 ) ) if job.state == model.Job.states.DELETED: return JOB_DELETED return JOB_READY @@ -322,9 +316,6 @@ Prepare the job to run by creating the working directory and the config files. """ - if self.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() mapping.context.current.clear() #this prevents the metadata reverting that has been seen in conjunction with the PBS job runner # Create the working directory self.working_directory = \ @@ -378,9 +369,6 @@ extra_filenames.append( param_filename ) self.param_dict = param_dict self.extra_filenames = extra_filenames - if self.app.memory_usage: - m1 = self.app.memory_usage.memory( m0, pretty=True ) - log.info("End of prepare for job id %d, memory used increased by %s" % ( job.id, m1 ) ) return extra_filenames def fail( self, message, exception=False ): @@ -388,9 +376,6 @@ Indicate job failure by setting state and message on all output datasets. """ - if self.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() job = model.Job.get( self.job_id ) job.refresh() # if the job was deleted, don't fail it @@ -417,16 +402,10 @@ if exception: job.traceback = traceback.format_exc() job.flush() - if self.app.memory_usage: - m1 = self.app.memory_usage.memory( m0, pretty=True ) - log.info("End of fail for job id %d, memory used increased by %s" % ( job.id, m1 ) ) # If the job was deleted, just clean up self.cleanup() def change_state( self, state, info = False ): - if self.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() job = model.Job.get( self.job_id ) job.refresh() for dataset_assoc in job.output_datasets: @@ -440,9 +419,6 @@ job.info = info job.state = state job.flush() - if self.app.memory_usage: - m1 = self.app.memory_usage.memory( m0, pretty=True ) - log.info("End of change_state for job id %d, memory used increased by %s" % ( job.id, m1 ) ) def get_state( self ): job = model.Job.get( self.job_id ) @@ -462,9 +438,6 @@ the output datasets based on stderr and stdout from the command, and the contents of the output files. """ - if self.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() # default post job setup mapping.context.current.clear() job = model.Job.get( self.job_id ) @@ -535,15 +508,9 @@ job.command_line = self.command_line mapping.context.current.flush() log.debug( 'job %d ended' % self.job_id ) - if self.app.memory_usage: - m1 = self.app.memory_usage.memory( m0, pretty=True ) - log.info("End of finish for job id %d, memory used increased by %s" % ( job.id, m1 ) ) self.cleanup() def cleanup( self ): - if self.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() # remove temporary files try: for fname in self.extra_filenames: @@ -552,9 +519,6 @@ os.rmdir( self.working_directory ) except: log.exception( "Unable to cleanup job %d" % self.job_id ) - if self.app.memory_usage: - m1 = self.app.memory_usage.memory( m0, pretty=True ) - log.info("End of cleanup for job id %d, memory used increased by %s" % ( self.job_id, m1 ) ) def get_command_line( self ): return self.command_line diff -r 511c2b1966dc -r 03f1e9e78f74 lib/galaxy/tools/actions/__init__.py --- a/lib/galaxy/tools/actions/__init__.py Tue Jan 13 10:55:08 2009 -0500 +++ b/lib/galaxy/tools/actions/__init__.py Tue Jan 13 11:33:17 2009 -0500 @@ -65,9 +65,6 @@ return input_datasets def execute(self, tool, trans, incoming={}, set_output_hid=True ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = trans.app.memory_usage.memory() out_data = {} # Collect any input datasets from the incoming parameters inp_data = self.collect_input_datasets( tool, incoming, trans ) @@ -201,9 +198,6 @@ for name, dataset in out_data.iteritems(): job.add_output_dataset( name, dataset ) trans.app.model.flush() - if trans.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info("End of tool %s execution for job id %d, memory used increased by %s" % ( tool.id, job.id, m1 ) ) # Some tools are not really executable, but jobs are still created for them ( for record keeping ). # Examples include tools that redirect to other applications ( epigraph ). These special tools must # include something that can be retrieved from the params ( e.g., REDIRECT_URL ) to keep the job diff -r 511c2b1966dc -r 03f1e9e78f74 lib/galaxy/tools/actions/upload.py --- a/lib/galaxy/tools/actions/upload.py Tue Jan 13 10:55:08 2009 -0500 +++ b/lib/galaxy/tools/actions/upload.py Tue Jan 13 11:33:17 2009 -0500 @@ -15,9 +15,6 @@ self.line_count = None def execute( self, tool, trans, incoming={}, set_output_hid = True ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = trans.app.memory_usage.memory() data_file = incoming['file_data'] file_type = incoming['file_type'] dbkey = incoming['dbkey'] @@ -129,9 +126,6 @@ job.flush() log.info( 'job id %d ended ok, file size: %s' % ( job.id, file_size_str ) ) trans.log_event( 'job id %d ended ok, file size: %s' % ( job.id, file_size_str ), tool_id=tool.id ) - if trans.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info("End of tool %s execution for job id %d, memory used increased by %s" % ( tool.id, job.id, m1 ) ) return dict( output=hda ) def upload_empty( self, trans, job, err_code, err_msg ): diff -r 511c2b1966dc -r 03f1e9e78f74 lib/galaxy/web/controllers/root.py --- a/lib/galaxy/web/controllers/root.py Tue Jan 13 10:55:08 2009 -0500 +++ b/lib/galaxy/web/controllers/root.py Tue Jan 13 11:33:17 2009 -0500 @@ -116,9 +116,6 @@ Returns data directly into the browser. Sets the mime-type according to the extension """ - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() if hid is not None: try: hid = int( hid ) @@ -149,9 +146,6 @@ fname = ''.join(c in valid_chars and c or '_' for c in fname)[0:150] trans.response.headers["Content-Disposition"] = "attachment; filename=GalaxyHistoryItem-%s-[%s]%s" % (data.hid, fname, toext) trans.log_event( "Display dataset id: %s" % str(id) ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/display, memory used increased by %s" % m1 ) try: return open( data.file_name ) except: @@ -328,9 +322,6 @@ @web.expose def history_delete( self, trans, id=None, **kwd): """Deletes a list of histories, ensures that histories are owned by current user""" - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() history_names = [] if id: if isinstance( id, list ): @@ -358,18 +349,12 @@ trans.log_event( "History id %s marked as deleted" % str(hid) ) else: return trans.show_message( "You must select at least one history to delete." ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_delete, memory used increased by %s" % m1 ) return trans.show_message( "History deleted: %s" % ",".join(history_names), refresh_frames=['history']) @web.expose def history_undelete( self, trans, id=[], **kwd): """Undeletes a list of histories, ensures that histories are owned by current user""" - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() history_names = [] errors = [] ok_msg = "" @@ -401,34 +386,22 @@ ok_msg = "Histories (%s) have been undeleted." % ", ".join( history_names ) else: errors.append( "You must select at least one history to undelete." ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_undelete, memory used increased by %s" % m1 ) return self.history_available( trans, id=','.join( id ), show_deleted=True, ok_msg = ok_msg, error_msg = " ".join( errors ) ) @web.expose def clear_history( self, trans ): """Clears the history for a user""" - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() history = trans.get_history() for dataset in history.datasets: dataset.deleted = True dataset.clear_associated_files() self.app.model.flush() trans.log_event( "History id %s cleared" % (str(history.id)) ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/clear_history, memory used increased by %s" % m1 ) trans.response.send_redirect( url_for("/index" ) ) @web.expose @web.require_login( "share histories with other users" ) def history_share( self, trans, id=None, email="", **kwd ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() send_to_err = "" if not id: id = trans.get_history().id @@ -455,9 +428,6 @@ trans.log_event( "History share, id: %s, name: '%s': to new id: %s" % (str(history.id), history.name, str(new_history.id)) ) self.app.model.flush() return trans.show_message( "History (%s) has been shared with: %s" % (",".join(history_names),email) ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_share, memory used increased by %s" % m1 ) return trans.fill_template( "/history/share.mako", histories=histories, email=email, send_to_err=send_to_err) @web.expose @@ -466,9 +436,6 @@ """ List all available histories """ - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() if as_xml: trans.response.set_content_type('text/xml') return trans.fill_template( "/history/list_as_xml.mako" ) @@ -476,9 +443,6 @@ id = id.split( "," ) trans.log_event( "History id %s available" % str( id ) ) history_operations = dict( share=self.history_share, rename=self.history_rename, delete=self.history_delete, undelete=self.history_undelete ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_available, memory used increased by %s" % m1 ) if do_operation in history_operations: return history_operations[do_operation]( trans, id=id, show_deleted=show_deleted, ok_msg=ok_msg, error_msg=error_msg, **kwd ) return trans.fill_template( "/history/list.mako", ids=id, @@ -491,9 +455,6 @@ @web.expose def history_import( self, trans, id=None, confirm=False, **kwd ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() msg = "" user = trans.get_user() user_history = trans.get_history() @@ -534,9 +495,6 @@ new_history.flush() trans.set_history( new_history ) trans.log_event( "History imported, id: %s, name: '%s': " % (str(new_history.id) , new_history.name ) ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_import, memory used increased by %s" % m1 ) return trans.show_ok_message( """ History "%s" has been imported. Click <a href="%s">here</a> to begin.""" % ( new_history.name, web.url_for( '/' ) ) ) @@ -551,9 +509,6 @@ if not id: return trans.response.send_redirect( web.url_for( action='history_available' ) ) else: - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() new_history = trans.app.model.History.get( id ) if new_history: galaxy_session = trans.get_galaxy_session() @@ -565,9 +520,6 @@ new_history.flush() trans.set_history( new_history ) trans.log_event( "History switched to id: %s, name: '%s'" % (str(new_history.id), new_history.name ) ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_switch, memory used increased by %s" % m1 ) return trans.show_message( "History switched to: %s" % new_history.name, refresh_frames=['history']) else: @@ -575,22 +527,13 @@ @web.expose def history_new( self, trans ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() trans.new_history() trans.log_event( "Created new History, id: %s." % str(trans.get_history().id) ) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_new, memory used increased by %s" % m1 ) return trans.show_message( "New history created", refresh_frames = ['history'] ) @web.expose @web.require_login( "renames histories" ) def history_rename( self, trans, id=None, name=None, **kwd ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() user = trans.get_user() if not isinstance( id, list ): @@ -627,17 +570,11 @@ change_msg = change_msg + "<p>You must specify a valid name for History: "+cur_names[i]+"</p>" else: change_msg = change_msg + "<p>History: "+cur_names[i]+" does not appear to belong to you.</p>" - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_rename, memory used increased by %s" % m1 ) return trans.show_message( "<p>%s" % change_msg, refresh_frames=['history'] ) @web.expose def history_add_to( self, trans, history_id=None, file_data=None, name="Data Added to History",info=None,ext="txt",dbkey="?",**kwd ): """Adds a POSTed file to a History""" - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() try: history = trans.app.model.History.get( history_id ) data = trans.app.model.HistoryDatasetAssociation( name = name, info = info, extension = ext, dbkey = dbkey, create_dataset = True ) @@ -656,9 +593,6 @@ data.set_peek() data.flush() trans.log_event("Added dataset %d to history %d" %(data.id, trans.history.id)) - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/history_add_to, memory used increased by %s" % m1 ) return trans.show_ok_message("Dataset "+str(data.hid)+" added to history "+str(history_id)+".") except: return trans.show_error_message("Adding File to History has Failed") @@ -666,9 +600,6 @@ @web.expose def dataset_make_primary( self, trans, id=None): """Copies a dataset and makes primary""" - if trans.app.memory_usage: - # Keep track of memory usage - m0 = self.app.memory_usage.memory() try: old_data = self.app.model.HistoryDatasetAssociation.get( id ) new_data = old_data.copy() @@ -677,9 +608,6 @@ history = trans.get_history() history.add_dataset(new_data) new_data.flush() - if self.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of root/dataset_make_primary, memory used increased by %s" % m1 ) return trans.show_message( "<p>Secondary dataset has been made primary.</p>", refresh_frames=['history'] ) except: return trans.show_error_message( "<p>Failed to make secondary dataset primary.</p>" ) diff -r 511c2b1966dc -r 03f1e9e78f74 lib/galaxy/web/controllers/user.py --- a/lib/galaxy/web/controllers/user.py Tue Jan 13 10:55:08 2009 -0500 +++ b/lib/galaxy/web/controllers/user.py Tue Jan 13 11:33:17 2009 -0500 @@ -70,9 +70,6 @@ @web.expose def login( self, trans, email='', password='' ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = trans.app.memory_usage.memory() email_error = password_error = None # Attempt login if email or password: @@ -88,9 +85,6 @@ trans.handle_user_login( user ) trans.log_event( "User logged in" ) return trans.show_ok_message( "Now logged in as " + user.email, refresh_frames=['masthead', 'history'] ) - if trans.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of user/login, memory used increased by %s" % m1 ) return trans.show_form( web.FormBuilder( web.url_for(), "Login", submit_text="Login" ) .add_text( "email", "Email address", value=email, error=email_error ) @@ -98,22 +92,13 @@ help="<a href='%s'>Forgot password? Reset here</a>" % web.url_for( action='reset_password' ) ) ) @web.expose def logout( self, trans ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = trans.app.memory_usage.memory() # Since logging an event requires a session, we'll log prior to ending the session trans.log_event( "User logged out" ) trans.handle_user_logout() - if trans.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of user/logout, memory used increased by %s" % m1 ) return trans.show_ok_message( "You are no longer logged in", refresh_frames=['masthead', 'history'] ) @web.expose def create( self, trans, email='', password='', confirm='',subscribe=False ): - if trans.app.memory_usage: - # Keep track of memory usage - m0 = trans.app.memory_usage.memory() email_error = password_error = confirm_error = None if email: if len( email ) == 0 or "@" not in email or "." not in email: @@ -139,9 +124,6 @@ mail.write("To: %s\nFrom: %s\nSubject: Join Mailing List\n\nJoin Mailing list." % (trans.app.config.mailing_join_addr,email) ) if mail.close(): return trans.show_warn_message( "Now logged in as " + user.email+". However, subscribing to the mailing list has failed.", refresh_frames=['masthead', 'history'] ) - if trans.app.memory_usage: - m1 = trans.app.memory_usage.memory( m0, pretty=True ) - log.info( "End of user/create, memory used increased by %s" % m1 ) return trans.show_ok_message( "Now logged in as " + user.email, refresh_frames=['masthead', 'history'] ) return trans.show_form( web.FormBuilder( web.url_for(), "Create account", submit_text="Create" )

1 0