- galaxy-dev - lists.galaxyproject.org

[hg] galaxy 1598: Fix a bug: errored jobs weren't actually being...
by Nate Coraor 03 Nov '08

03 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/ca640ce1abcf changeset: 1598:ca640ce1abcf user: Nate Coraor <nate(a)bx.psu.edu> date: Mon Nov 03 15:15:22 2008 -0500 description: Fix a bug: errored jobs weren't actually being deleted. 1 file(s) affected in this change: lib/galaxy/web/controllers/root.py diffs (19 lines): diff -r 21f98b638868 -r ca640ce1abcf lib/galaxy/web/controllers/root.py --- a/lib/galaxy/web/controllers/root.py Mon Nov 03 15:07:22 2008 -0500 +++ b/lib/galaxy/web/controllers/root.py Mon Nov 03 15:15:22 2008 -0500 @@ -281,11 +281,10 @@ if data.parent_id is None and len( data.creating_job_associations ) > 0: # Mark associated job for deletion job = data.creating_job_associations[0].job - if job.state not in [ model.Job.states.QUEUED, model.Job.states.RUNNING, model.Job.states.NEW ]: - return - # Are *all* of the job's other output datasets deleted? - if job.check_if_output_datasets_deleted(): - job.mark_deleted() + if job.state in [ model.Job.states.QUEUED, model.Job.states.RUNNING, model.Job.states.NEW ]: + # Are *all* of the job's other output datasets deleted? + if job.check_if_output_datasets_deleted(): + job.mark_deleted() self.app.model.flush() @web.expose

1 0

[hg] galaxy 1597: Backed out changeset 36a297157114
by Nate Coraor 03 Nov '08

03 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/21f98b638868 changeset: 1597:21f98b638868 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Mon Nov 03 15:07:22 2008 -0500 description: Backed out changeset 36a297157114 Temporarilly rolling this change back... 1 file(s) affected in this change: lib/galaxy/jobs/__init__.py diffs (18 lines): diff -r 36a297157114 -r 21f98b638868 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Mon Nov 03 14:48:25 2008 -0500 +++ b/lib/galaxy/jobs/__init__.py Mon Nov 03 15:07:22 2008 -0500 @@ -274,7 +274,6 @@ Prepare the job to run by creating the working directory and the config files. """ - mapping.context.current.clear() #this prevents the metadata reverting that has been seen in conjunction with the PBS job runner # Create the working directory self.working_directory = \ os.path.join( self.app.config.job_working_directory, str( self.job_id ) ) @@ -644,4 +643,4 @@ def put( self, *args ): return def shutdown( self ): - return + return \ No newline at end of file

1 0

[hg] galaxy 1596: A fix for the metadata reverting issue seen wi...
by Nate Coraor 03 Nov '08

03 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/36a297157114 changeset: 1596:36a297157114 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Mon Nov 03 14:48:25 2008 -0500 description: A fix for the metadata reverting issue seen with PBS job runner. 1 file(s) affected in this change: lib/galaxy/jobs/__init__.py diffs (18 lines): diff -r 56bc2f789894 -r 36a297157114 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Sat Nov 01 15:30:13 2008 -0400 +++ b/lib/galaxy/jobs/__init__.py Mon Nov 03 14:48:25 2008 -0500 @@ -274,6 +274,7 @@ Prepare the job to run by creating the working directory and the config files. """ + mapping.context.current.clear() #this prevents the metadata reverting that has been seen in conjunction with the PBS job runner # Create the working directory self.working_directory = \ os.path.join( self.app.config.job_working_directory, str( self.job_id ) ) @@ -643,4 +644,4 @@ def put( self, *args ): return def shutdown( self ): - return \ No newline at end of file + return

1 0

[hg] galaxy 1593: Small typo in one of the encode import tools.
by Greg Von Kuster 01 Nov '08

01 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/cb3507efced7 changeset: 1593:cb3507efced7 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Fri Oct 31 14:52:41 2008 -0400 description: Small typo in one of the encode import tools. 1 file(s) affected in this change: tools/data_source/encode_import_all_latest_datasets.xml diffs (19 lines): diff -r 6ddbf5c83efc -r cb3507efced7 tools/data_source/encode_import_all_latest_datasets.xml --- a/tools/data_source/encode_import_all_latest_datasets.xml Fri Oct 31 11:00:50 2008 -0400 +++ b/tools/data_source/encode_import_all_latest_datasets.xml Fri Oct 31 14:52:41 2008 -0400 @@ -2,7 +2,7 @@ <command interpreter="python">encode_import.py $hg17,$hg16 $output ${GALAXY_DATA_INDEX_DIR}</command> <inputs> <display> - <p><div class="toolFormTitle">hg1 (most recent datasets in bold)</div>$hg17</p> + <p><div class="toolFormTitle">hg17 (most recent datasets in bold)</div>$hg17</p> <p><div class="toolFormTitle">hg16 (most recent datasets in bold)</div>$hg16</p> </display> <param name="hg17" type="select" display="checkboxes" multiple="true"> @@ -63,4 +63,4 @@ </help> -</tool> \ No newline at end of file +</tool>

1 0

[hg] galaxy 1595: Fix History.add_dataset() to be compatible wit...
by Greg Von Kuster 01 Nov '08

01 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/56bc2f789894 changeset: 1595:56bc2f789894 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Sat Nov 01 15:30:13 2008 -0400 description: Fix History.add_dataset() to be compatible with sqlalchemy 4. 1 file(s) affected in this change: lib/galaxy/model/__init__.py diffs (26 lines): diff -r 881ac57dcc81 -r 56bc2f789894 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py Fri Oct 31 15:21:30 2008 -0400 +++ b/lib/galaxy/model/__init__.py Sat Nov 01 15:30:13 2008 -0400 @@ -353,17 +353,18 @@ dataset = HistoryDatasetAssociation( dataset = dataset ) dataset.flush() elif not isinstance( dataset, HistoryDatasetAssociation ): - raise TypeError, "You can only add Dataset and HistoryDatasetAssociation instances to a history." + raise TypeError, "You can only add Dataset and HistoryDatasetAssociation instances to a history ( you tried to add %s )." % str( dataset ) if parent_id: for data in self.datasets: if data.id == parent_id: dataset.hid = data.hid break else: - if set_hid: dataset.hid = self._next_hid() + if set_hid: + dataset.hid = self._next_hid() else: - if set_hid: dataset.hid = self._next_hid() - dataset.history = self + if set_hid: + dataset.hid = self._next_hid() if genome_build not in [None, '?']: self.genome_build = genome_build self.datasets.append( dataset )

1 0

[hg] galaxy 1594: Fix for use of deprecated <display> tags in to...
by Greg Von Kuster 01 Nov '08

01 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/881ac57dcc81 changeset: 1594:881ac57dcc81 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Fri Oct 31 15:21:30 2008 -0400 description: Fix for use of deprecated <display> tags in tool_form.mako. 1 file(s) affected in this change: templates/tool_form.mako diffs (12 lines): diff -r cb3507efced7 -r 881ac57dcc81 templates/tool_form.mako --- a/templates/tool_form.mako Fri Oct 31 14:52:41 2008 -0400 +++ b/templates/tool_form.mako Fri Oct 31 15:21:30 2008 -0400 @@ -127,7 +127,7 @@ <input type="hidden" name="tool_state" value="${util.object_to_string( tool_state.encode( tool, app ) )}"> %if tool.display_by_page[tool_state.page]: - ${trans.fill_template_string( tool.display_by_page[tool_state.page], other_values=tool.get_param_html_map( trans, tool_state.page, tool_state.inputs ) )} + ${trans.fill_template_string( tool.display_by_page[tool_state.page], context=tool.get_param_html_map( trans, tool_state.page, tool_state.inputs ) )} <input type="submit" class="primary-button" name="runtool_btn" value="Execute"> %else:

1 0

[hg] galaxy 1591: James' changes allowing the server to run w/o ...
by Greg Von Kuster 01 Nov '08

01 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/c4644668afff changeset: 1591:c4644668afff user: Nate Coraor <nate(a)bx.psu.edu> date: Fri Oct 31 10:22:46 2008 -0400 description: James' changes allowing the server to run w/o a job runner, and nginx-specific performance improvements. 11 file(s) affected in this change: lib/galaxy/app.py lib/galaxy/config.py lib/galaxy/jobs/__init__.py lib/galaxy/model/__init__.py lib/galaxy/tools/__init__.py lib/galaxy/tools/actions/upload.py lib/galaxy/tools/parameters/__init__.py lib/galaxy/tools/parameters/basic.py lib/galaxy/web/controllers/root.py lib/galaxy/web/framework/base.py universe_wsgi.ini.sample diffs (576 lines): diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/app.py --- a/lib/galaxy/app.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/app.py Fri Oct 31 10:22:46 2008 -0400 @@ -31,9 +31,11 @@ #Load datatype converters self.datatypes_registry.load_datatype_converters( self.toolbox ) # Start the job queue - job_dispatcher = jobs.DefaultJobDispatcher( self ) - self.job_queue = jobs.JobQueue( self, job_dispatcher ) - self.job_stop_queue = jobs.JobStopQueue( self, job_dispatcher ) + self.job_manager = jobs.JobManager( self ) + # FIXME: These are exposed directly for backward compatibility + self.job_queue = self.job_manager.job_queue + self.job_stop_queue = self.job_manager.job_stop_queue + # Heartbeat and memdump for thread / heap profiling self.heartbeat = None self.memdump = None # Start the heartbeat process if configured and available @@ -48,7 +50,6 @@ if memdump.Memdump: self.memdump = memdump.Memdump() def shutdown( self ): - self.job_stop_queue.shutdown() - self.job_queue.shutdown() + self.job_manager.shutdown() if self.heartbeat: self.heartbeat.shutdown() diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/config.py --- a/lib/galaxy/config.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/config.py Fri Oct 31 10:22:46 2008 -0400 @@ -64,6 +64,11 @@ self.bugs_email = kwargs.get( 'bugs_email', None ) self.blog_url = kwargs.get( 'blog_url', None ) self.screencasts_url = kwargs.get( 'screencasts_url', None ) + # Configuration options for taking advantage of nginx features + self.nginx_x_accel_redirect_base = kwargs.get( 'nginx_x_accel_redirect_base', False ) + self.nginx_upload_location = kwargs.get( 'nginx_upload_store', False ) + if self.nginx_upload_location: + self.nginx_upload_location = os.path.abspath( self.nginx_upload_location ) # Parse global_conf and save the parser global_conf = kwargs.get( 'global_conf', None ) global_conf_parser = ConfigParser.ConfigParser() @@ -78,6 +83,11 @@ self.datatypes_config = kwargs.get( 'datatypes_config_file', 'datatypes_conf.xml' ) def get( self, key, default ): return self.config_dict.get( key, default ) + def get_bool( self, key, default ): + if key in self.config_dict: + return string_as_bool( key ) + else: + return default def check( self ): # Check that required directories exist for path in self.root, self.file_path, self.tool_path, self.tool_data_path, self.template_path, self.job_working_directory: diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/jobs/__init__.py Fri Oct 31 10:22:46 2008 -0400 @@ -16,6 +16,27 @@ # States for running a job. These are NOT the same as data states JOB_WAIT, JOB_ERROR, JOB_INPUT_ERROR, JOB_INPUT_DELETED, JOB_OK, JOB_READY, JOB_DELETED = 'wait', 'error', 'input_error', 'input_deleted', 'ok', 'ready', 'deleted' + +class JobManager( object ): + """ + Highest level interface to job management. + + TODO: Currently the app accesses "job_queue" and "job_stop_queue" directly. + This should be decoupled. + """ + def __init__( self, app ): + self.app = app + if self.app.config.get_bool( "enable_job_running", True ): + # The dispatcher launches the underlying job runners + self.dispatcher = DefaultJobDispatcher( app ) + # Queues for starting and stopping jobs + self.job_queue = JobQueue( app, self.dispatcher ) + self.job_stop_queue = JobStopQueue( app, self.dispatcher ) + else: + self.job_queue = self.job_stop_queue = NoopQueue() + def shutdown( self ): + self.job_queue.shutdown() + self.job_stop_queue.shutdown() class Sleeper( object ): """ @@ -594,49 +615,11 @@ pass for job in jobs: - # jobs in a non queued/running/new state do not need to be stopped - if job.state not in [ model.Job.states.QUEUED, model.Job.states.RUNNING, model.Job.states.NEW ]: - return - # job has multiple datasets that aren't parent/child and not all of them are deleted. - if not self.check_if_output_datasets_deleted( job.id ): - return - self.mark_deleted( job.id ) # job is in JobQueue or FooJobRunner, will be dequeued due to state change above if job.job_runner_name is None: return # tell the dispatcher to stop the job self.dispatcher.stop( job ) - - def check_if_output_datasets_deleted( self, job_id ): - job = model.Job.get( job_id ) - for dataset_assoc in job.output_datasets: - dataset = dataset_assoc.dataset - dataset.refresh() - #only the originator of the job can delete a dataset to cause - #cancellation of the job, no need to loop through history_associations - if not dataset.deleted: - return False - return True - - def mark_deleted( self, job_id ): - job = model.Job.get( job_id ) - job.refresh() - job.state = job.states.DELETED - job.info = "Job output deleted by user before job completed." - job.flush() - for dataset_assoc in job.output_datasets: - dataset = dataset_assoc.dataset - dataset.refresh() - dataset.deleted = True - dataset.state = dataset.states.DISCARDED - dataset.dataset.flush() - for dataset in dataset.dataset.history_associations: - #propagate info across shared datasets - dataset.deleted = True - dataset.blurb = 'deleted' - dataset.peek = 'Job deleted' - dataset.info = 'Job output deleted by user before job completed' - dataset.flush() def put( self, job ): self.queue.put( job ) @@ -652,3 +635,12 @@ self.queue.put( self.STOP_SIGNAL ) self.sleeper.wake() log.info( "job stopper stopped" ) + +class NoopQueue( object ): + """ + Implements the JobQueue / JobStopQueue interface but does nothing + """ + def put( self, *args ): + return + def shutdown( self ): + return \ No newline at end of file diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/model/__init__.py Fri Oct 31 10:22:46 2008 -0400 @@ -85,7 +85,35 @@ tool = app.toolbox.tools_by_id[self.tool_id] param_dict = tool.params_from_strings( param_dict, app ) return param_dict - + def check_if_output_datasets_deleted( self ): + """ + Return true if all of the output datasets associated with this job are + in the deleted state + """ + for dataset_assoc in self.output_datasets: + dataset = dataset_assoc.dataset + # only the originator of the job can delete a dataset to cause + # cancellation of the job, no need to loop through history_associations + if not dataset.deleted: + return False + return True + def mark_deleted( self ): + """ + Mark this job as deleted, and mark any output datasets as discarded. + """ + self.state = Job.states.DELETED + self.info = "Job output deleted by user before job completed." + for dataset_assoc in self.output_datasets: + dataset = dataset_assoc.dataset + dataset.deleted = True + dataset.state = dataset.states.DISCARDED + for dataset in dataset.dataset.history_associations: + # propagate info across shared datasets + dataset.deleted = True + dataset.blurb = 'deleted' + dataset.peek = 'Job deleted' + dataset.info = 'Job output deleted by user before job completed' + class JobParameter( object ): def __init__( self, name, value ): self.name = name diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/tools/__init__.py Fri Oct 31 10:22:46 2008 -0400 @@ -797,7 +797,7 @@ # Deal with the 'test' element and see if it's value changed test_param_key = group_prefix + input.test_param.name test_param_error = None - test_incoming = incoming.get( test_param_key, None ) + test_incoming = get_incoming_value( incoming, test_param_key, None ) if test_param_key not in incoming \ and "__force_update__" + test_param_key not in incoming \ and update_only: @@ -878,7 +878,7 @@ except: pass if not incoming_value_generated: - incoming_value = incoming.get( key, None ) + incoming_value = get_incoming_value( incoming, key, None ) value, error = check_param( trans, input, incoming_value, context ) if input.dependent_params and state[ input.name ] != value: # We need to keep track of changed dependency parametrs ( parameters @@ -1362,3 +1362,12 @@ else: return val +def get_incoming_value( incoming, key, default ): + if "__" + key + "__is_composite" in incoming: + composite_keys = incoming["__" + key + "__keys"].split() + value = dict() + for composite_key in composite_keys: + value[composite_key] = incoming[key + "_" + composite_key] + return value + else: + return incoming.get( key, default ) \ No newline at end of file diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/tools/actions/upload.py --- a/lib/galaxy/tools/actions/upload.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/tools/actions/upload.py Fri Oct 31 10:22:46 2008 -0400 @@ -26,12 +26,22 @@ temp_name = "" data_list = [] - if 'filename' in dir( data_file ): + if 'local_filename' in dir( data_file ): + # Use the existing file try: file_name = data_file.filename file_name = file_name.split( '\\' )[-1] file_name = file_name.split( '/' )[-1] - data_list.append( self.add_file( trans, data_file.file, file_name, file_type, dbkey, space_to_tab=space_to_tab ) ) + data_list.append( self.add_file( trans, data_file.local_filename, file_name, file_type, dbkey, space_to_tab=space_to_tab ) ) + except Exception, e: + return self.upload_empty( trans, "Error:", str( e ) ) + elif 'filename' in dir( data_file ): + try: + file_name = data_file.filename + file_name = file_name.split( '\\' )[-1] + file_name = file_name.split( '/' )[-1] + temp_name = sniff.stream_to_file( data_file.file ) + data_list.append( self.add_file( trans, temp_name, file_name, file_type, dbkey, space_to_tab=space_to_tab ) ) except Exception, e: return self.upload_empty( trans, "Error:", str( e ) ) if url_paste not in [ None, "" ]: @@ -41,7 +51,8 @@ line = line.rstrip( '\r\n' ) if line: try: - data_list.append( self.add_file( trans, urllib.urlopen( line ), line, file_type, dbkey, info="uploaded url", space_to_tab=space_to_tab ) ) + temp_name = sniff.stream_to_file( urllib.urlopen( line ) ) + data_list.append( self.add_file( trans, temp_name, line, file_type, dbkey, info="uploaded url", space_to_tab=space_to_tab ) ) except Exception, e: return self.upload_empty( trans, "Error:", str( e ) ) else: @@ -53,7 +64,8 @@ break if is_valid: try: - data_list.append( self.add_file( trans, StringIO.StringIO( url_paste ), 'Pasted Entry', file_type, dbkey, info="pasted entry", space_to_tab=space_to_tab ) ) + temp_name = sniff.stream_to_file( StringIO.StringIO( url_paste ) ) + data_list.append( self.add_file( trans, temp_name, 'Pasted Entry', file_type, dbkey, info="pasted entry", space_to_tab=space_to_tab ) ) except Exception, e: return self.upload_empty( trans, "Error:", str( e ) ) else: @@ -77,9 +89,8 @@ trans.app.model.flush() return dict( output=data ) - def add_file( self, trans, file_obj, file_name, file_type, dbkey, info=None, space_to_tab=False ): + def add_file( self, trans, temp_name, file_name, file_type, dbkey, info=None, space_to_tab=False ): data_type = None - temp_name = sniff.stream_to_file( file_obj ) # See if we have an empty file if not os.path.getsize( temp_name ) > 0: diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/tools/parameters/__init__.py --- a/lib/galaxy/tools/parameters/__init__.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/tools/parameters/__init__.py Fri Oct 31 10:22:46 2008 -0400 @@ -16,9 +16,7 @@ value = incoming_value error = None try: - if param.name == 'file_data': - pass - elif value is not None or isinstance(param, DataToolParameter): + if value is not None or isinstance(param, DataToolParameter): # Convert value from HTML representation value = param.from_html( value, trans, param_values ) # Allow the value to be converted if neccesary diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/tools/parameters/basic.py --- a/lib/galaxy/tools/parameters/basic.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/tools/parameters/basic.py Fri Oct 31 10:22:46 2008 -0400 @@ -2,12 +2,13 @@ Basic tool parameters. """ -import logging, string, sys, os +import logging, string, sys, os, os.path from elementtree.ElementTree import XML, Element from galaxy import config, datatypes, util from galaxy.web import form_builder +from galaxy.util.bunch import Bunch import validation, dynamic_options @@ -294,6 +295,23 @@ self.name = elem.get( 'name' ) def get_html_field( self, trans=None, value=None, other_values={} ): return form_builder.FileField( self.name ) + def from_html( self, value, trans=None, other_values={} ): + # Middleware or proxies may encode files in special ways (TODO: this + # should be pluggable) + if type( value ) == dict: + upload_location = self.tool.app.config.nginx_upload_location + assert upload_location, \ + "Request appears to have been processed by nginx_upload_module \ + but Galaxy is not configured to recgonize it" + # Check that the file is in the right location + local_filename = os.path.abspath( value['path'] ) + assert local_filename.startswith( upload_location ), \ + "Filename provided by nginx is not in correct directory" + value = Bunch( + filename = value["name"], + local_filename = local_filename + ) + return value def get_required_enctype( self ): """ File upload elements require the multipart/form-data encoding diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/web/controllers/root.py --- a/lib/galaxy/web/controllers/root.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/web/controllers/root.py Fri Oct 31 10:22:46 2008 -0400 @@ -266,6 +266,28 @@ return trans.fill_template( "/dataset/edit_attributes.mako", data=data, datatypes=ldatatypes, err=None ) + def __delete_dataset( self, trans, id ): + data = self.app.model.HistoryDatasetAssociation.get( id ) + if data: + # Walk up parent datasets to find the containing history + topmost_parent = data + while topmost_parent.parent: + topmost_parent = topmost_parent.parent + assert topmost_parent in trans.history.datasets, "Data does not belong to current history" + # Mark deleted and cleanup + data.mark_deleted() + data.clear_associated_files() + trans.log_event( "Dataset id %s marked as deleted" % str(id) ) + if data.parent_id is None and len( data.creating_job_associations ) > 0: + # Mark associated job for deletion + job = data.creating_job_associations[0].job + if job.state not in [ model.Job.states.QUEUED, model.Job.states.RUNNING, model.Job.states.NEW ]: + return + # Are *all* of the job's other output datasets deleted? + if job.check_if_output_datasets_deleted(): + job.mark_deleted() + self.app.model.flush() + @web.expose def delete( self, trans, id = None, show_deleted_on_refresh = False, **kwd): if id: @@ -276,26 +298,10 @@ history = trans.get_history() for id in dataset_ids: try: - int( id ) + id = int( id ) except: continue - data = self.app.model.HistoryDatasetAssociation.get( id ) - if data: - # Walk up parent datasets to find the containing history - topmost_parent = data - while topmost_parent.parent: - topmost_parent = topmost_parent.parent - assert topmost_parent in history.datasets, "Data does not belong to current history" - # Mark deleted and cleanup - data.mark_deleted() - data.clear_associated_files() - self.app.model.flush() - trans.log_event( "Dataset id %s marked as deleted" % str(id) ) - if data.parent_id is None: - try: - self.app.job_stop_queue.put( data.creating_job_associations[0].job ) - except IndexError: - pass # upload tool will cause this since it doesn't have a job + self.__delete_dataset( trans, id ) return self.history( trans, show_deleted = show_deleted_on_refresh ) @web.expose @@ -305,24 +311,7 @@ int( id ) except: return "Dataset id '%s' is invalid" %str( id ) - history = trans.get_history() - data = self.app.model.HistoryDatasetAssociation.get( id ) - if data: - # Walk up parent datasets to find the containing history - topmost_parent = data - while topmost_parent.parent: - topmost_parent = topmost_parent.parent - assert topmost_parent in history.datasets, "Data does not belong to current history" - # Mark deleted and cleanup - data.mark_deleted() - data.clear_associated_files() - self.app.model.flush() - trans.log_event( "Dataset id %s marked as deleted async" % str(id) ) - if data.parent_id is None: - try: - self.app.job_stop_queue.put( data.creating_job_associations[0].job ) - except IndexError: - pass # upload tool will cause this since it doesn't have a job + self.__delete_dataset( trans, id ) return "OK" ## ---- History management ----------------------------------------------- diff -r 7e94f40ee9e8 -r c4644668afff lib/galaxy/web/framework/base.py --- a/lib/galaxy/web/framework/base.py Thu Oct 30 17:34:46 2008 -0400 +++ b/lib/galaxy/web/framework/base.py Fri Oct 31 10:22:46 2008 -0400 @@ -5,6 +5,7 @@ import socket import types import logging +import os.path import sys from Cookie import SimpleCookie @@ -132,16 +133,16 @@ if callable( body ): # Assume the callable is another WSGI application to run return body( environ, start_response ) + elif isinstance( body, types.FileType ): + # Stream the file back to the browser + return send_file( start_response, trans, body ) else: start_response( trans.response.wsgi_status(), trans.response.wsgi_headeritems() ) return self.make_body_iterable( trans, body ) def make_body_iterable( self, trans, body ): - if isinstance( body, types.FileType ): - # Stream the file back to the browser - return iterate_file( body ) - elif isinstance( body, ( types.GeneratorType, list, tuple ) ): + if isinstance( body, ( types.GeneratorType, list, tuple ) ): # Recursively stream the iterable return flatten( body ) elif isinstance( body, basestring ): @@ -302,6 +303,20 @@ CHUNK_SIZE = 2**16 +def send_file( start_response, trans, body ): + # If configured use X-Accel-Redirect header for nginx + base = trans.app.config.nginx_x_accel_redirect_base + if base: + trans.response.headers['X-Accel-Redirect'] = \ + base + os.path.abspath( body.name ) + body = [ "" ] + # Fall back on sending the file in chunks + else: + body = iterate_file( body ) + start_response( trans.response.wsgi_status(), + trans.response.wsgi_headeritems() ) + return body + def iterate_file( file ): """ Progressively return chunks from `file`. diff -r 7e94f40ee9e8 -r c4644668afff universe_wsgi.ini.sample --- a/universe_wsgi.ini.sample Thu Oct 30 17:34:46 2008 -0400 +++ b/universe_wsgi.ini.sample Fri Oct 31 10:22:46 2008 -0400 @@ -34,7 +34,7 @@ # Database connection database_file = database/universe.sqlite # You may use a SQLAlchemy connection string to specify an external database instead -## database_connection = postgres:///galaxy_test +## database_connection = postgres:///galaxy ## database_engine_option_echo = true ## database_engine_option_echo_pool = true ## database_engine_option_pool_size = 10 @@ -89,12 +89,15 @@ # Write thread status periodically to 'heartbeat.log' (careful, uses disk space rapidly!) ## use_heartbeat = True +# Enable the memory debugging interface (careful, negatively impacts server performance) +## use_memdump = True + # Profiling middleware (cProfile based) ## use_profile = True -# Mail -smtp_server = coltrane.bx.psu.edu -error_email_to = galaxy-bugs(a)bx.psu.edu +# For use by 'report this error' link on error-state datasets +#smtp_server = smtp.example.org +#error_email_to = galaxy-bugs(a)example.org # Use the new iframe / javascript based layout use_new_layout = true @@ -120,29 +123,34 @@ ## wiki_url: replaces the default galaxy main wiki ## bugs_email: replaces the default galaxy bugs email list #brand = Private local mirror -#wiki_url=/path/to/my/local/wiki -#bugs_email=mailto:bugmaster@this.site.com +#wiki_url = /path/to/my/local/wiki +#bugs_email = mailto:galaxy-bugs@example.org # ---- Job Runners ---------------------------------------------------------- # Clustering Galaxy is not a straightforward process and requires a lot of -# pre-configuration. See the ClusteringGalaxy Wiki before attempting to set any -# of these options. If running normally (without a cluster), do not change -# anything in this section. +# pre-configuration. See the ClusteringGalaxy Wiki before attempting to set +# any of these options: +# +# http://g2.trac.bx.psu.edu/wiki/ClusteringGalaxy +# +# If running normally (without a cluster), do not change anything in this +# section. # start_job_runners: Comma-separated list of job runners to start. local is # always started. If left commented, no jobs will be run on the cluster, even # if a cluster URL is explicitly defined in the [galaxy:tool_runners] section -# below. The only runner currently available is 'pbs'. +# below. The runners currently available are 'pbs' and 'sge'. #start_job_runners = pbs # default_cluster_job_runner: The URL for the default runner to use when a tool # doesn't explicity define a runner below. For help on the cluster URL format, -# see the ClusteringGalaxy Wiki. Leave commented if not using a cluster job runner. +# see the ClusteringGalaxy Wiki. Leave commented if not using a cluster job +# runner. #default_cluster_job_runner = pbs:/// # The PBS options are described in detail in the Galaxy Configuration section of -# the ClusteringGalaxy Wiki +# the ClusteringGalaxy Wiki, and are only necessary when using file staging. #pbs_application_server = #pbs_stage_path = #pbs_dataset_server = @@ -152,8 +160,6 @@ [galaxy:tool_runners] biomart = local:/// -blat2wig = pbs:///blast -blat_wrapper = pbs:///blast encode_db1 = local:/// encode_import_all_latest_datasets1 = local:/// encode_import_chromatin_and_chromosomes1 = local:/// @@ -161,13 +167,8 @@ encode_import_genes_and_transcripts1 = local:/// encode_import_multi-species_sequence_analysis1 = local:/// encode_import_transcription_regulation1 = local:/// -generate_coverage_report = pbs:///blast hbvar = local:/// -hist_high_quality_score = pbs:///blast -megablast_wrapper = pbs:///blast -megablast_xml_parser = pbs:///blast microbial_import1 = local:/// -quality_score_distribution = pbs:///blast ucsc_table_direct1 = local:/// ucsc_table_direct_archaea1 = local:/// ucsc_table_direct_test1 = local:///

1 0

[hg] galaxy 1592: Fixes for dealing with UnvalidatedValues (i.e....
by Greg Von Kuster 01 Nov '08

01 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/6ddbf5c83efc changeset: 1592:6ddbf5c83efc user: Dan Blankenberg <dan(a)bx.psu.edu> date: Fri Oct 31 11:00:50 2008 -0400 description: Fixes for dealing with UnvalidatedValues (i.e. when running workflows) and also when passing None trans to dynamic options. 4 file(s) affected in this change: lib/galaxy/jobs/__init__.py lib/galaxy/tools/__init__.py lib/galaxy/tools/parameters/dynamic_options.py tools/data_source/microbial_import_code.py diffs (77 lines): diff -r c4644668afff -r 6ddbf5c83efc lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Fri Oct 31 10:22:46 2008 -0400 +++ b/lib/galaxy/jobs/__init__.py Fri Oct 31 11:00:50 2008 -0400 @@ -464,7 +464,7 @@ # custom post process setup inp_data = dict( [ ( da.name, da.dataset ) for da in job.input_datasets ] ) out_data = dict( [ ( da.name, da.dataset ) for da in job.output_datasets ] ) - param_dict = dict( [ ( p.name, p.value ) for p in job.parameters ] ) # why not re-use self.param_dict here? + param_dict = dict( [ ( p.name, p.value ) for p in job.parameters ] ) # why not re-use self.param_dict here? ##dunno...probably should, this causes tools.parameters.basic.UnvalidatedValue to be used in following methods instead of validated and transformed values during i.e. running workflows param_dict = self.tool.params_from_strings( param_dict, self.app ) # Check for and move associated_files self.tool.collect_associated_files(out_data) diff -r c4644668afff -r 6ddbf5c83efc lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Fri Oct 31 10:22:46 2008 -0400 +++ b/lib/galaxy/tools/__init__.py Fri Oct 31 11:00:50 2008 -0400 @@ -948,7 +948,10 @@ # Regular tool parameter value = input_values[ input.name ] if isinstance( value, UnvalidatedValue ): - value = input.from_html( value.value, None, context ) + if value.value is None: #if value.value is None, it could not have been submited via html form and therefore .from_html can't be guaranteed to work + value = None + else: + value = input.from_html( value.value, None, context ) # Then do any further validation on the value input.validate( value, None ) input_values[ input.name ] = value diff -r c4644668afff -r 6ddbf5c83efc lib/galaxy/tools/parameters/dynamic_options.py --- a/lib/galaxy/tools/parameters/dynamic_options.py Fri Oct 31 10:22:46 2008 -0400 +++ b/lib/galaxy/tools/parameters/dynamic_options.py Fri Oct 31 11:00:50 2008 -0400 @@ -102,7 +102,7 @@ if self.multiple: return dataset_value in file_value.split( self.separator ) return file_value == dataset_value - assert self.ref_name in other_values or trans.workflow_building_mode, "Required dependency '%s' not found in incoming values" % self.ref_name + assert self.ref_name in other_values or ( trans is not None and trans.workflow_building_mode), "Required dependency '%s' not found in incoming values" % self.ref_name ref = other_values.get( self.ref_name, None ) if not isinstance( ref, self.dynamic_option.tool_param.tool.app.model.HistoryDatasetAssociation ): return [] #not a valid dataset @@ -146,9 +146,9 @@ def get_dependency_name( self ): return self.ref_name def filter_options( self, options, trans, other_values ): - if trans.workflow_building_mode: return [] + if trans is not None and trans.workflow_building_mode: return [] + assert self.ref_name in other_values, "Required dependency '%s' not found in incoming values" % self.ref_name ref = str( other_values.get( self.ref_name, None ) ) - assert ref is not None, "Required dependency '%s' not found in incoming values" % self.ref_name rval = [] for fields in options: if ( self.keep and fields[self.column] == ref ) or ( not self.keep and fields[self.column] != ref ): diff -r c4644668afff -r 6ddbf5c83efc tools/data_source/microbial_import_code.py --- a/tools/data_source/microbial_import_code.py Fri Oct 31 10:22:46 2008 -0400 +++ b/tools/data_source/microbial_import_code.py Fri Oct 31 11:00:50 2008 -0400 @@ -80,7 +80,7 @@ return microbe_info #post processing, set build for data and add additional data to history -from galaxy import datatypes, config, jobs +from galaxy import datatypes, config, jobs, tools from shutil import copyfile def exec_after_process(app, inp_data, out_data, param_dict, tool, stdout, stderr): @@ -95,7 +95,12 @@ #if not (kingdom or group or org): if not (kingdom or org): print "Parameters are not available." - + #workflow passes galaxy.tools.parameters.basic.UnvalidatedValue instead of values + if isinstance( kingdom, tools.parameters.basic.UnvalidatedValue ): + kingdom = kingdom.value + if isinstance( org, tools.parameters.basic.UnvalidatedValue ): + org = org.value + GALAXY_DATA_INDEX_DIR = app.config.tool_data_path microbe_info = load_microbial_data( GALAXY_DATA_INDEX_DIR, sep='\t' ) new_stdout = ""

1 0

[hg] galaxy 1590: Convert tool_form/tool_executed to mako (for p...
by Greg Von Kuster 01 Nov '08

01 Nov '08

details: http://www.bx.psu.edu/hg/galaxy/rev/7e94f40ee9e8 changeset: 1590:7e94f40ee9e8 user: James Taylor <james(a)jamestaylor.org> date: Thu Oct 30 17:34:46 2008 -0400 description: Convert tool_form/tool_executed to mako (for performance) 5 file(s) affected in this change: lib/galaxy/tools/__init__.py templates/tool_executed.mako templates/tool_executed.tmpl templates/tool_form.mako templates/tool_form.tmpl diffs (573 lines): diff -r 1d113c5386da -r 7e94f40ee9e8 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Thu Oct 30 16:17:46 2008 -0400 +++ b/lib/galaxy/tools/__init__.py Thu Oct 30 17:34:46 2008 -0400 @@ -676,7 +676,7 @@ # on the standard run form) or "URL" (a parameter provided by # external data source tools). if "runtool_btn" not in incoming and "URL" not in incoming: - return "tool_form.tmpl", dict( errors={}, tool_state=state, param_values={}, incoming={} ) + return "tool_form.mako", dict( errors={}, tool_state=state, param_values={}, incoming={} ) # Process incoming data if not( self.check_values ): # If `self.check_values` is false we don't do any checking or @@ -702,20 +702,20 @@ # error messages if errors: error_message = "One or more errors were found in the input you provided. The specific errors are marked below." - return "tool_form.tmpl", dict( errors=errors, tool_state=state, incoming=incoming, error_message=error_message ) + return "tool_form.mako", dict( errors=errors, tool_state=state, incoming=incoming, error_message=error_message ) # If we've completed the last page we can execute the tool elif state.page == self.last_page: out_data = self.execute( trans, incoming=params ) - return 'tool_executed.tmpl', dict( out_data=out_data ) + return 'tool_executed.mako', dict( out_data=out_data ) # Otherwise move on to the next page else: state.page += 1 # Fill in the default values for the next page self.fill_in_new_state( trans, self.inputs_by_page[ state.page ], state.inputs ) - return 'tool_form.tmpl', dict( errors=errors, tool_state=state ) + return 'tool_form.mako', dict( errors=errors, tool_state=state ) else: # Just a refresh, render the form with updated state and errors. - return 'tool_form.tmpl', dict( errors=errors, tool_state=state ) + return 'tool_form.mako', dict( errors=errors, tool_state=state ) def update_state( self, trans, inputs, state, incoming, prefix="", context=None, update_only=False, old_errors={}, changed_dependencies={} ): diff -r 1d113c5386da -r 7e94f40ee9e8 templates/tool_executed.mako --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/templates/tool_executed.mako Thu Oct 30 17:34:46 2008 -0400 @@ -0,0 +1,63 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> +<html> + +<head> +<title>Galaxy</title> +<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> +<link href="${h.url_for('/static/style/base.css')}" rel="stylesheet" type="text/css" /> +<script type="text/javascript"> + var inside_galaxy_frameset = false; + + if ( parent.frames && parent.frames.galaxy_history ) { + parent.frames.galaxy_history.location.href="${h.url_for( controller='root', action='history' )}"; + inside_galaxy_frameset = true; + } + + if ( parent.handle_minwidth_hint ) { + parent.handle_minwidth_hint( -1 ); + } + + function main() { + // If called from outside the galaxy frameset, redirect there + %if tool.options.refresh: + if ( ! inside_galaxy_frameset ) { + setTimeout( "refresh()", 1000 ); + document.getElementById( "refresh_message" ).style.display = "block"; + } + %endif + } + + function refresh() { + top.location.href = '${request.base}'; + } + +</script> + +</head> + +<body onLoad="main()"> + +<div class="donemessage"> + +<p>The following job has been succesfully added to the queue:</p> + +%for data in out_data.values(): + <div style="padding: 10px"><b> ${data.hid}: ${data.name}</b></div> +%endfor + +<p> +You can check the status of queued jobs and view the resulting +data by refreshing the <b>History</b> pane. When the job has been run +the status will change from 'running' to 'finished' if completed +succesfully or 'error' if problems were encountered. +</p> + +%if tool.options.refresh: +<p id="refresh_message" style="display: none;">You are now being redirected back to <a href="${request.base}">Galaxy</a></div> +%endif + +</div> + +</body> + +</html> \ No newline at end of file diff -r 1d113c5386da -r 7e94f40ee9e8 templates/tool_executed.tmpl --- a/templates/tool_executed.tmpl Thu Oct 30 16:17:46 2008 -0400 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,70 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> -<html> - -<head> -<title>Galaxy</title> -<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> -<link href="$h.url_for('/static/style/base.css')" rel="stylesheet" type="text/css" /> -<script type="text/javascript"> - var inside_galaxy_frameset = false; - - if ( parent.frames && parent.frames.galaxy_history ) - { - parent.frames.galaxy_history.location.href="$h.url_for( controller='root', action='history' )"; - - inside_galaxy_frameset = true; - } - - if ( parent.handle_minwidth_hint ) - { - parent.handle_minwidth_hint( -1 ); - } - - function main() - { - // If called from outside the galaxy frameset, redirect there - #if $tool.options.refresh - if ( ! inside_galaxy_frameset ) - { - setTimeout( "refresh()", 1000 ); - document.getElementById( "refresh_message" ).style.display = "block"; - } - #end if - } - - function refresh() - { - top.location.href = '$request.base'; - } - -</script> - -</head> - -<body onLoad="main()"> - - -<div class="donemessage"> - -<p>The following job has been succesfully added to the queue:</p> - -#for $data in $out_data.values - <div style="padding: 10px"><b> $data.hid: $data.name</b></div> -#end for - -<p> -You can check the status of queued jobs and view the resulting -data by refreshing the <b>History</b> pane. When the job has been run -the status will change from 'running' to 'finished' if completed -succesfully or 'error' if problems were encountered. -</p> - -#if $tool.options.refresh -<p id="refresh_message" style="display: none;">You are now being redirected back to <a href="$request.base">Galaxy</a></div> -#end if - -</div> - -</body> - -</html> \ No newline at end of file diff -r 1d113c5386da -r 7e94f40ee9e8 templates/tool_form.mako --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/templates/tool_form.mako Thu Oct 30 17:34:46 2008 -0400 @@ -0,0 +1,194 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> + +<% +from galaxy.util.expressions import ExpressionContext +%> + +<html> + +<head> +<title>Galaxy</title> +<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> +<link href="${h.url_for('/static/style/base.css')}" rel="stylesheet" type="text/css" /> +<script type='text/javascript' src="${h.url_for('/static/scripts/jquery.js')}"> </script> +<script type="text/javascript"> +$( function() { + $( "select[@refresh_on_change='true']").change( function() { + $( "#tool_form" ).submit(); + }); +}); +%if not add_frame.debug: + if( window.name != "galaxy_main" ) { + location.replace( '${h.url_for( controller='root', action='index', tool_id=tool.id )}' ); + } +%endif +</script> +</head> + +<body> + +<%def name="do_inputs( inputs, tool_state, errors, prefix, other_values=None )"> + <% other_values = ExpressionContext( tool_state, other_values ) %> + %for input_index, input in enumerate( inputs.itervalues() ): + %if input.type == "repeat": + <div class="repeat-group"> + <div class="form-title-row"><b>${input.title_plural}</b></div> + <% repeat_state = tool_state[input.name] %> + %for i in range( len( repeat_state ) ): + <div class="repeat-group-item"> + <% + if input.name in errors: + rep_errors = errors[input.name][i] + else: + rep_errors = dict() + index = repeat_state[i]['__index__'] + %> + <div class="form-title-row"><b>${input.title} ${i + 1}</b></div> + ${do_inputs( input.inputs, repeat_state[i], rep_errors, prefix + input.name + "_" + str(index) + "|", other_values )} + <div class="form-row"><input type="submit" name="${prefix}${input.name}_${index}_remove" value="Remove ${input.title} ${i+1}"></div> + </div> + %endfor + <div class="form-row"><input type="submit" name="${prefix}${input.name}_add" value="Add new ${input.title}"></div> + </div> + %elif input.type == "conditional": + <% + group_state = tool_state[input.name] + group_errors = errors.get( input.name, {} ) + current_case = group_state['__current_case__'] + group_prefix = prefix + input.name + "|" + %> + ${row_for_param( group_prefix, input.test_param, group_state, group_errors, other_values )} + ${do_inputs( input.cases[current_case].inputs, group_state, group_errors, group_prefix, other_values )} + %else: + ${row_for_param( prefix, input, tool_state, errors, other_values )} + %endif + %endfor +</%def> + +<%def name="row_for_param( prefix, param, parent_state, parent_errors, other_values )"> + <% + if parent_errors.has_key( param.name ): + cls = "form-row form-row-error" + else: + cls = "form-row" + %> + <div class="${cls}"> + <% label = param.get_label() %> + %if label: + <label> + ${label}: + </label> + %endif + <% + field = param.get_html_field( trans, parent_state[ param.name ], other_values ) + field.refresh_on_change = param.refresh_on_change + %> + <div style="float: left; width: 250px; margin-right: 10px;">${field.get_html( prefix )}</div> + %if parent_errors.has_key( param.name ): + <div style="float: left; color: red; font-weight: bold; padding-top: 1px; padding-bottom: 3px;"> + <div style="width: 300px;"><img style="vertical-align: middle;" src="${h.url_for('/static/style/error_small.png')}"> <span style="vertical-align: middle;">${parent_errors[param.name]}</span></div> + </div> + %endif + + %if param.help: + <div class="toolParamHelp" style="clear: both;"> + ${param.help} + </div> + %endif + + <div style="clear: both"></div> + + </div> +</%def> + +%if add_frame.from_noframe: + <div class="warningmessage"> + <strong>Welcome to Galaxy</strong> + <hr/> + It appears that you found this tool from a link outside of Galaxy. + If you're not familiar with Galaxy, please consider visiting the + <a href="${h.url_for( controller='root' )}" target="_top">welcome page</a>. + To learn more about what Galaxy is and what it can do for you, please visit + the <a href="$add_frame.wiki_url" target="_top">Galaxy wiki</a>. + </div> + <br/> +%endif + +<div class="toolForm" id="$tool.id"> + %if tool.has_multiple_pages: + <div class="toolFormTitle">${tool.name} (step ${tool_state.page+1} of ${tool.npages})</div> + %else: + <div class="toolFormTitle">${tool.name}</div> + %endif + <div class="toolFormBody"> + <form id="tool_form" name="tool_form" action="${h.url_for( tool.action )}" enctype="${tool.enctype}" target="${tool.target}" method="${tool.method}"> + <input type="hidden" name="tool_id" value="${tool.id}"> + <input type="hidden" name="tool_state" value="${util.object_to_string( tool_state.encode( tool, app ) )}"> + + %if tool.display_by_page[tool_state.page]: + ${trans.fill_template_string( tool.display_by_page[tool_state.page], other_values=tool.get_param_html_map( trans, tool_state.page, tool_state.inputs ) )} + <input type="submit" class="primary-button" name="runtool_btn" value="Execute"> + + %else: + + <div style="display: none;"> + %if tool_state.page == tool.last_page: + <input type="submit" class="primary-button" name="runtool_btn" value="Execute"> + %else: + <input type="submit" class="primary-button" name="runtool_btn" value="Next step"> + %endif + </div> + ${do_inputs( tool.inputs_by_page[ tool_state.page ], tool_state.inputs, errors, "" )} + <div class="form-row"> + %if tool_state.page == tool.last_page: + <input type="submit" class="primary-button" name="runtool_btn" value="Execute"> + %else: + <input type="submit" class="primary-button" name="runtool_btn" value="Next step"> + %endif + </div> + + %endif + + </form> + </div> +</div> + +%if tool.help: +<div class="toolHelp"> + <div class="toolHelpBody"> + %if tool.has_multiple_pages: + ${tool.help_by_page[tool_state.page]} + %else: + ${tool.help} + %endif + </div> +</div> +%endif + +</body> + +<script type="text/javascript"> + $( function() { + $( 'li > ul' ).each( function( i ) { + if ( $( this )[0].className == 'toolParameterExpandableCollapsable' ) + { + var parent_li = $( this ).parent( 'li' ); + var sub_ul = $( this ).remove(); + parent_li.find( 'span' ).wrapInner( '<a/>' ).find( 'a' ).click( function() { + sub_ul.toggle(); + $( this )[0].innerHTML = ( sub_ul[0].style.display=='none' ) ? '[+]' : '[-]'; + }); + parent_li.append( sub_ul ); + } + }); + $( 'ul ul' ).each( function(i) { + if ( $( this )[0].className == 'toolParameterExpandableCollapsable' && this.attributes.getNamedItem( 'default_state' ).value == 'collapsed' ) + { + $( this ).hide(); + } + }); + }); +</script> + +</html> diff -r 1d113c5386da -r 7e94f40ee9e8 templates/tool_form.tmpl --- a/templates/tool_form.tmpl Thu Oct 30 16:17:46 2008 -0400 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,191 +0,0 @@ - -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> - -#from galaxy.util.expressions import ExpressionContext - -<html> - -<head> -<title>Galaxy</title> -<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> -<link href="$h.url_for('/static/style/base.css')" rel="stylesheet" type="text/css" /> -<script type='text/javascript' src="$h.url_for('/static/scripts/jquery.js')"> </script> -<script type="text/javascript"> -jQuery( function() { - jQuery( "select[@refresh_on_change='true']").change( function() { - jQuery( "#tool_form" ).submit(); - }); -}); -#if not $add_frame.debug - if( window.name != "galaxy_main" ) { - location.replace( '$h.url_for( controller='root', action='index', tool_id=$tool.id )' ); - } -#end if -</script> -</head> - -<body> - -## #if $getVar( 'error_message', None ) -## <div class="errormessagesmall">$error_message</div> -## <p></p> -## #end if - -#def do_inputs( $inputs, $tool_state, $errors, $prefix, $context=None ) - #set $context = ExpressionContext( $tool_state, $context ) - #for $input_index, $input in enumerate( $inputs.itervalues() ) - #if $input.type == "repeat" - <div class="repeat-group"> - <div class="form-title-row"><b>${input.title_plural}</b></div> - #set repeat_state = $tool_state[$input.name] - #for i in range( len( $repeat_state ) ): - <div class="repeat-group-item"> - #if $input.name in errors - #set rep_errors = $errors[$input.name][$i] - #else - #set rep_errors = dict() - #end if - #set index = $repeat_state[$i]['__index__'] - <div class="form-title-row"><b>${input.title} ${i + 1}</b></div> - $do_inputs( $input.inputs, $repeat_state[$i], $rep_errors, $prefix + $input.name + "_" + str($index) + "|", $context ) - <div class="form-row"><input type="submit" name="${prefix}${input.name}_${index}_remove" value="Remove ${input.title} ${i+1}"></div> - </div> - #end for - <div class="form-row"><input type="submit" name="${prefix}${input.name}_add" value="Add new ${input.title}"></div> - </div> - #elif $input.type == "conditional" - #set group_state = $tool_state[$input.name] - #set group_errors = $errors.get( $input.name, {} ) - #set current_case = $group_state['__current_case__'] - #set group_prefix = $prefix + $input.name + "|" - $row_for_param( $group_prefix, $input.test_param, $group_state, $group_errors, $context ) - $do_inputs( $input.cases[$current_case].inputs, $group_state, $group_errors, $group_prefix, $context ) - #else - $row_for_param( $prefix, $input, $tool_state, $errors, $context ) - #end if - #end for -#end def - -#def row_for_param( $prefix, $param, $parent_state, $parent_errors, $context ) - #if $parent_errors.has_key( $param.name ): - #set cls = "form-row form-row-error" - #else - #set cls = "form-row" - #end if - <div class="$cls"> - #set label = $param.get_label() - #if $label: - <label> - $label: - </label> - #end if - #set field = $param.get_html_field( $caller, $parent_state[ $param.name ], $context ) - #set $field.refresh_on_change = $param.refresh_on_change - <div style="float: left; width: 250px; margin-right: 10px;">$field.get_html( $prefix )</div> - #if $parent_errors.has_key( $param.name ): - <div style="float: left; color: red; font-weight: bold; padding-top: 1px; padding-bottom: 3px;"> - <div style="width: 300px;"><img style="vertical-align: middle;" src="$h.url_for('/static/style/error_small.png')"> <span style="vertical-align: middle;">$parent_errors[$param.name]</span></div> - </div> - #elif $param.help - ##<div class="toolParamHelp" style="float: right;">$param.help</div> - #end if - - #if $param.help - <div class="toolParamHelp" style="clear: both;"> - $param.help - </div> - - #end if - - <div style="clear: both"></div> - - </div> -#end def - -#if $add_frame.from_noframe - <div class="warningmessage"> - <strong>Welcome to Galaxy</strong> - <hr/> - It appears that you found this tool from a link outside of Galaxy. If you're not familiar with Galaxy, please consider visiting the <a href="$h.url_for( controller='root' )" target="_top">welcome page</a>. To learn more about what Galaxy is and what it can do for you, please visit the <a href="$add_frame.wiki_url" target="_top">Galaxy wiki</a>. - </div> - <br/> -#end if - -<div class="toolForm" id="$tool.id"> - #if $tool.has_multiple_pages - <div class="toolFormTitle">$tool.name (step #echo $tool_state.page+1 # of $tool.npages)</div> - #else - <div class="toolFormTitle">$tool.name</div> - #end if - <div class="toolFormBody"> - <form id="tool_form" name="tool_form" action="$h.url_for( $tool.action )" enctype="$tool.enctype" target="$tool.target" method="$tool.method"> - <input type="hidden" name="tool_id" value="$tool.id"> - <input type="hidden" name="tool_state" value="$util.object_to_string( $tool_state.encode( $tool, $app ) )"> - - #if $tool.display_by_page[$tool_state.page] - - $caller.fill_template_string( $tool.display_by_page[$tool_state.page], context=$tool.get_param_html_map( $caller, $tool_state.page, $tool_state.inputs ) ) - <input type="submit" class="primary-button" name="runtool_btn" value="Execute"> - - #else - - <div style="display: none;"> - #if $tool_state.page == $tool.last_page - <input type="submit" class="primary-button" name="runtool_btn" value="Execute"> - #else - <input type="submit" class="primary-button" name="runtool_btn" value="Next step"> - #end if - </div> - $do_inputs( $tool.inputs_by_page[ $tool_state.page ], $tool_state.inputs, $errors, "" ) - <div class="form-row"> - #if $tool_state.page == $tool.last_page - <input type="submit" class="primary-button" name="runtool_btn" value="Execute"> - #else - <input type="submit" class="primary-button" name="runtool_btn" value="Next step"> - #end if - </div> - - #end if - - </form> - </div> -</div> - -#if $tool.help -<div class="toolHelp"> - <div class="toolHelpBody"> - #if $tool.has_multiple_pages - $tool.help_by_page[$tool_state.page] - #else - $tool.help - #end if - </div> -</div> -#end if - -</body> - -<script type="text/javascript"> - \$( function() { - \$( 'li > ul' ).each( function( i ) { - if ( \$( this )[0].className == 'toolParameterExpandableCollapsable' ) - { - var parent_li = \$( this ).parent( 'li' ); - var sub_ul = \$( this ).remove(); - parent_li.find( 'span' ).wrapInner( '<a/>' ).find( 'a' ).click( function() { - sub_ul.toggle(); - \$( this )[0].innerHTML = ( sub_ul[0].style.display=='none' ) ? '[+]' : '[-]'; - }); - parent_li.append( sub_ul ); - } - }); - \$( 'ul ul' ).each( function(i) { - if ( \$( this )[0].className == 'toolParameterExpandableCollapsable' && this.attributes.getNamedItem( 'default_state' ).value == 'collapsed' ) - { - \$( this ).hide(); - } - }); - }); -</script> - -</html>

1 0

[hg] galaxy 1587: Added new functional tests to all gops tools u...
by Greg Von Kuster 30 Oct '08

30 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/0da612f8a78e changeset: 1587:0da612f8a78e user: guru date: Thu Oct 30 16:03:11 2008 -0400 description: Added new functional tests to all gops tools using BitsetSafeReaderWrapper. These tests check cases where datasets have the chr,start and end values in non-standard columns. 12 file(s) affected in this change: test-data/2_mod.bed test-data/gops_complement_out_diffCols.dat test-data/gops_coverage_out_diffCols.interval test-data/gops_intersect_diffCols.bed test-data/gops_intersect_p_diffCols.bed test-data/gops_merge_diffCols.dat test-data/gops_subtract_diffCols.dat tools/new_operations/complement.xml tools/new_operations/coverage.xml tools/new_operations/intersect.xml tools/new_operations/merge.xml tools/new_operations/subtract.xml diffs (530 lines): diff -r 4e44d29377e3 -r 0da612f8a78e test-data/2_mod.bed --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/2_mod.bed Thu Oct 30 16:03:11 2008 -0400 @@ -0,0 +1,69 @@ +#chr name score strand start end +chr1 NM_005997_cds_0_0_chr1_147962193_r 0 - 147962192 147962580 +chr1 BC007833_cds_0_0_chr1_147984546_f 0 + 147984545 147984630 +chr1 AJ011123_cds_0_0_chr1_148078401_r 0 - 148078400 148078582 +chr1 NM_002796_cds_0_0_chr1_148185137_f 0 + 148185136 148185276 +chr10 AY029205_cds_0_0_chr10_55251624_r 0 - 55251623 55253124 +chr11 AK057832_cds_0_0_chr11_116124408_r 0 - 116124407 116124501 +chr11 NM_000040_cds_1_0_chr11_116206509_f 0 + 116206508 116206563 +chr11 BC005380_cds_0_0_chr11_116211734_r 0 - 116211733 116212337 +chr11 AY358331_cds_0_0_chr11_130745912_f 0 + 130745911 130745993 +chr12 NM_052885_cds_0_0_chr12_38440095_r 0 - 38440094 38440321 +chr12 AY792511_cds_0_0_chr12_38905201_f 0 + 38905200 38905351 +chr13 NM_207440_cds_1_0_chr13_112381695_f 0 + 112381694 112381953 +chr13 NM_032116_cds_0_0_chr13_29680677_r 0 - 29680676 29680875 +chr14 U88895_cds_0_0_chr14_98521865_f 0 + 98521864 98521922 +chr14 NM_022898_cds_0_0_chr14_98710241_r 0 - 98710240 98712285 +chr15 BX537418_cds_0_0_chr15_41486873_r 0 - 41486872 41487060 +chr15 AK223365_cds_0_0_chr15_41673709_f 0 + 41673708 41673857 +chr15 NM_153700_cds_0_0_chr15_41679162_r 0 - 41679161 41679250 +chr15 AK223365_cds_0_0_chr15_41773541_f 0 + 41773540 41773689 +chr16 NM_005332_cds_0_0_chr16_142909_f 0 + 142908 143003 +chr16 BC065198_cds_0_0_chr16_179198_r 0 - 179197 179339 +chr16 AK057165_cds_2_0_chr16_244414_f 0 + 244413 244681 +chr16 AB016929_cds_0_0_chr16_259269_r 0 - 259268 259383 +chr18 NM_001792_cds_0_0_chr18_23786115_r 0 - 23786114 23786321 +chr18 NM_012397_cds_1_0_chr18_59406882_f 0 + 59406881 59407046 +chr18 AB046400_cds_0_0_chr18_59455933_r 0 - 59455932 59456337 +chr18 AY792326_cds_0_0_chr18_59528408_f 0 + 59528407 59528575 +chr19 BC013995_cds_1_0_chr19_59068596_f 0 + 59068595 59069564 +chr19 NM_198481_cds_0_0_chr19_59236027_r 0 - 59236026 59236146 +chr19 NM_004542_cds_0_0_chr19_59297999_f 0 + 59297998 59298008 +chr19 AK128544_cds_3_0_chr19_59318206_r 0 - 59318205 59318718 +chr2 NM_006773_cds_0_0_chr2_118288584_f 0 + 118288583 118288668 +chr2 BC005078_cds_0_0_chr2_118390396_r 0 - 118390395 118390500 +chr2 AY125465_cds_0_0_chr2_220108690_f 0 + 220108689 220109267 +chr2 NM_024536_cds_0_0_chr2_220229610_r 0 - 220229609 220230869 +chr20 NM_181466_cds_0_0_chr20_33330414_r 0 - 33330413 33330423 +chr20 BC085019_cds_1_0_chr20_33485371_f 0 + 33485370 33486123 +chr20 NM_000557_cds_1_0_chr20_33488492_r 0 - 33488491 33489122 +chr20 AF022655_cds_1_0_chr20_33513607_f 0 + 33513606 33513792 +chr21 NM_032910_cds_0_0_chr21_32687403_f 0 + 32687402 32687588 +chr21 NM_018277_cds_3_0_chr21_32869642_r 0 - 32869641 32870022 +chr21 NM_005806_cds_1_0_chr21_33321041_f 0 + 33321040 33322012 +chr21 AK129657_cds_0_0_chr21_33728359_r 0 - 33728358 33728724 +chr22 NM_004147_cds_0_0_chr22_30120224_f 0 + 30120223 30120265 +chr22 BC032941_cds_0_0_chr22_30160420_r 0 - 30160419 30160661 +chr22 NM_001007467_cds_1_0_chr22_30228825_f 0 + 30228824 30228916 +chr22 CR456540_cds_0_0_chr22_30340152_r 0 - 30340151 30340376 +chr5 AF099740_cds_11_0_chr5_131311207_r 0 - 131311206 131311254 +chr5 NM_000588_cds_0_0_chr5_131424299_f 0 + 131424298 131424460 +chr5 BC035813_cds_0_0_chr5_131556602_r 0 - 131556601 131556672 +chr5 BC003096_cds_0_0_chr5_131621327_f 0 + 131621326 131621419 +chr6 NM_007214_cds_0_0_chr6_108299601_r 0 - 108299600 108299744 +chr6 NM_003269_cds_0_0_chr6_108594663_f 0 + 108594662 108594687 +chr6 NM_003795_cds_0_0_chr6_108640046_r 0 - 108640045 108640151 +chr6 NM_145315_cds_0_0_chr6_108722977_f 0 + 108722976 108723115 +chr7 AF467257_cds_1_0_chr7_113660518_f 0 + 113660517 113660685 +chr7 NM_003391_cds_0_0_chr7_116512160_r 0 - 116512159 116512389 +chr7 NM_000492_cds_0_0_chr7_116714100_f 0 + 116714099 116714152 +chr7 AF377960_cds_0_0_chr7_116945542_r 0 - 116945541 116945787 +chr8 NM_000127_cds_0_0_chr8_118881132_r 0 - 118881131 118881317 +chr9 BC051300_cds_0_0_chr9_128764157_f 0 + 128764156 128764189 +chr9 NM_014908_cds_0_0_chr9_128787520_r 0 - 128787519 128789136 +chr9 NM_015354_cds_0_0_chr9_128789553_f 0 + 128789552 128789584 +chr9 AB058751_cds_0_0_chr9_128850517_r 0 - 128850516 128850624 +chrX NM_001167_cds_1_0_chrX_122745048_f 0 + 122745047 122745924 +chrX NM_000425_cds_0_0_chrX_152648965_r 0 - 152648964 152649196 +chrX AF101728_cds_0_0_chrX_152691447_f 0 + 152691446 152691471 +chrX BC052303_cds_0_0_chrX_152694030_r 0 - 152694029 152694263 diff -r 4e44d29377e3 -r 0da612f8a78e test-data/gops_complement_out_diffCols.dat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/gops_complement_out_diffCols.dat Thu Oct 30 16:03:11 2008 -0400 @@ -0,0 +1,88 @@ +chr7 . . + 0 113660517 +chr7 . . + 113660685 116512159 +chr7 . . + 116512389 116714099 +chr7 . . + 116714152 116945541 +chr7 . . + 116945787 536870912 +chr6 . . + 0 108299600 +chr6 . . + 108299744 108594662 +chr6 . . + 108594687 108640045 +chr6 . . + 108640151 108722976 +chr6 . . + 108723115 536870912 +chr5 . . + 0 131311206 +chr5 . . + 131311254 131424298 +chr5 . . + 131424460 131556601 +chr5 . . + 131556672 131621326 +chr5 . . + 131621419 536870912 +chrX . . + 0 122745047 +chrX . . + 122745924 152648964 +chrX . . + 152649196 152691446 +chrX . . + 152691471 152694029 +chrX . . + 152694263 536870912 +chr2 . . + 0 118288583 +chr2 . . + 118288668 118390395 +chr2 . . + 118390500 220108689 +chr2 . . + 220109267 220229609 +chr2 . . + 220230869 536870912 +chr1 . . + 0 147962192 +chr1 . . + 147962580 147984545 +chr1 . . + 147984630 148078400 +chr1 . . + 148078582 148185136 +chr1 . . + 148185276 536870912 +chr21 . . + 0 32687402 +chr21 . . + 32687588 32869641 +chr21 . . + 32870022 33321040 +chr21 . . + 33322012 33728358 +chr21 . . + 33728724 536870912 +chr9 . . + 0 128764156 +chr9 . . + 128764189 128787519 +chr9 . . + 128789136 128789552 +chr9 . . + 128789584 128850516 +chr9 . . + 128850624 536870912 +chr8 . . + 0 118881131 +chr8 . . + 118881317 536870912 +chr13 . . + 0 29680676 +chr13 . . + 29680875 112381694 +chr13 . . + 112381953 536870912 +chr12 . . + 0 38440094 +chr12 . . + 38440321 38905200 +chr12 . . + 38905351 536870912 +chr11 . . + 0 116124407 +chr11 . . + 116124501 116206508 +chr11 . . + 116206563 116211733 +chr11 . . + 116212337 130745911 +chr11 . . + 130745993 536870912 +chr10 . . + 0 55251623 +chr10 . . + 55253124 536870912 +chr22 . . + 0 30120223 +chr22 . . + 30120265 30160419 +chr22 . . + 30160661 30228824 +chr22 . . + 30228916 30340151 +chr22 . . + 30340376 536870912 +chr16 . . + 0 142908 +chr16 . . + 143003 179197 +chr16 . . + 179339 244413 +chr16 . . + 244681 259268 +chr16 . . + 259383 536870912 +chr15 . . + 0 41486872 +chr15 . . + 41487060 41673708 +chr15 . . + 41673857 41679161 +chr15 . . + 41679250 41773540 +chr15 . . + 41773689 536870912 +chr14 . . + 0 98521864 +chr14 . . + 98521922 98710240 +chr14 . . + 98712285 536870912 +chr20 . . + 0 33330413 +chr20 . . + 33330423 33485370 +chr20 . . + 33486123 33488491 +chr20 . . + 33489122 33513606 +chr20 . . + 33513792 536870912 +chr19 . . + 0 59068595 +chr19 . . + 59069564 59236026 +chr19 . . + 59236146 59297998 +chr19 . . + 59298008 59318205 +chr19 . . + 59318718 536870912 +chr18 . . + 0 23786114 +chr18 . . + 23786321 59406881 +chr18 . . + 59407046 59455932 +chr18 . . + 59456337 59528407 +chr18 . . + 59528575 536870912 diff -r 4e44d29377e3 -r 0da612f8a78e test-data/gops_coverage_out_diffCols.interval --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/gops_coverage_out_diffCols.interval Thu Oct 30 16:03:11 2008 -0400 @@ -0,0 +1,65 @@ +chr1 147962192 147962580 CCDS989.1_cds_0_0_chr1_147962193_r 0 - 388 1.0 +chr1 147984545 147984630 CCDS990.1_cds_0_0_chr1_147984546_f 0 + 85 1.0 +chr1 148078400 148078582 CCDS993.1_cds_0_0_chr1_148078401_r 0 - 182 1.0 +chr1 148185136 148185276 CCDS996.1_cds_0_0_chr1_148185137_f 0 + 140 1.0 +chr10 55251623 55253124 CCDS7248.1_cds_0_0_chr10_55251624_r 0 - 1501 1.0 +chr11 116124407 116124501 CCDS8374.1_cds_0_0_chr11_116124408_r 0 - 94 1.0 +chr11 116206508 116206563 CCDS8377.1_cds_0_0_chr11_116206509_f 0 + 55 1.0 +chr11 116211733 116212337 CCDS8378.1_cds_0_0_chr11_116211734_r 0 - 604 1.0 +chr11 1812377 1812407 CCDS7726.1_cds_0_0_chr11_1812378_f 0 + 0 0.0 +chr12 38440094 38440321 CCDS8736.1_cds_0_0_chr12_38440095_r 0 - 227 1.0 +chr13 112381694 112381953 CCDS9526.1_cds_0_0_chr13_112381695_f 0 + 259 1.0 +chr14 98710240 98712285 CCDS9949.1_cds_0_0_chr14_98710241_r 0 - 2045 1.0 +chr15 41486872 41487060 CCDS10096.1_cds_0_0_chr15_41486873_r 0 - 188 1.0 +chr15 41673708 41673857 CCDS10097.1_cds_0_0_chr15_41673709_f 0 + 149 1.0 +chr15 41679161 41679250 CCDS10098.1_cds_0_0_chr15_41679162_r 0 - 89 1.0 +chr15 41826029 41826196 CCDS10101.1_cds_0_0_chr15_41826030_f 0 + 0 0.0 +chr16 142908 143003 CCDS10397.1_cds_0_0_chr16_142909_f 0 + 95 1.0 +chr16 179963 180135 CCDS10401.1_cds_0_0_chr16_179964_r 0 - 0 0.0 +chr16 244413 244681 CCDS10402.1_cds_0_0_chr16_244414_f 0 + 268 1.0 +chr16 259268 259383 CCDS10403.1_cds_0_0_chr16_259269_r 0 - 115 1.0 +chr18 23786114 23786321 CCDS11891.1_cds_0_0_chr18_23786115_r 0 - 207 1.0 +chr18 59406881 59407046 CCDS11985.1_cds_0_0_chr18_59406882_f 0 + 165 1.0 +chr18 59455932 59456337 CCDS11986.1_cds_0_0_chr18_59455933_r 0 - 405 1.0 +chr18 59600586 59600754 CCDS11988.1_cds_0_0_chr18_59600587_f 0 + 0 0.0 +chr19 59068595 59069564 CCDS12866.1_cds_0_0_chr19_59068596_f 0 + 969 1.0 +chr19 59236026 59236146 CCDS12872.1_cds_0_0_chr19_59236027_r 0 - 120 1.0 +chr19 59297998 59298008 CCDS12877.1_cds_0_0_chr19_59297999_f 0 + 10 1.0 +chr19 59302168 59302288 CCDS12878.1_cds_0_0_chr19_59302169_r 0 - 0 0.0 +chr2 118288583 118288668 CCDS2120.1_cds_0_0_chr2_118288584_f 0 + 85 1.0 +chr2 118394148 118394202 CCDS2121.1_cds_0_0_chr2_118394149_r 0 - 0 0.0 +chr2 220190202 220190242 CCDS2441.1_cds_0_0_chr2_220190203_f 0 + 0 0.0 +chr2 220229609 220230869 CCDS2443.1_cds_0_0_chr2_220229610_r 0 - 1260 1.0 +chr20 33330413 33330423 CCDS13249.1_cds_0_0_chr20_33330414_r 0 - 10 1.0 +chr20 33513606 33513792 CCDS13255.1_cds_0_0_chr20_33513607_f 0 + 186 1.0 +chr20 33579500 33579527 CCDS13256.1_cds_0_0_chr20_33579501_r 0 - 0 0.0 +chr20 33593260 33593348 CCDS13257.1_cds_0_0_chr20_33593261_f 0 + 0 0.0 +chr21 32707032 32707192 CCDS13614.1_cds_0_0_chr21_32707033_f 0 + 0 0.0 +chr21 32869641 32870022 CCDS13615.1_cds_0_0_chr21_32869642_r 0 - 381 1.0 +chr21 33321040 33322012 CCDS13620.1_cds_0_0_chr21_33321041_f 0 + 972 1.0 +chr21 33744994 33745040 CCDS13625.1_cds_0_0_chr21_33744995_r 0 - 0 0.0 +chr22 30120223 30120265 CCDS13897.1_cds_0_0_chr22_30120224_f 0 + 42 1.0 +chr22 30160419 30160661 CCDS13898.1_cds_0_0_chr22_30160420_r 0 - 242 1.0 +chr22 30665273 30665360 CCDS13901.1_cds_0_0_chr22_30665274_f 0 + 0 0.0 +chr22 30939054 30939266 CCDS13903.1_cds_0_0_chr22_30939055_r 0 - 0 0.0 +chr5 131424298 131424460 CCDS4149.1_cds_0_0_chr5_131424299_f 0 + 162 1.0 +chr5 131556601 131556672 CCDS4151.1_cds_0_0_chr5_131556602_r 0 - 71 1.0 +chr5 131621326 131621419 CCDS4152.1_cds_0_0_chr5_131621327_f 0 + 93 1.0 +chr5 131847541 131847666 CCDS4155.1_cds_0_0_chr5_131847542_r 0 - 0 0.0 +chr6 108299600 108299744 CCDS5061.1_cds_0_0_chr6_108299601_r 0 - 144 1.0 +chr6 108594662 108594687 CCDS5063.1_cds_0_0_chr6_108594663_f 0 + 25 1.0 +chr6 108640045 108640151 CCDS5064.1_cds_0_0_chr6_108640046_r 0 - 106 1.0 +chr6 108722976 108723115 CCDS5067.1_cds_0_0_chr6_108722977_f 0 + 139 1.0 +chr7 113660517 113660685 CCDS5760.1_cds_0_0_chr7_113660518_f 0 + 168 1.0 +chr7 116512159 116512389 CCDS5771.1_cds_0_0_chr7_116512160_r 0 - 230 1.0 +chr7 116714099 116714152 CCDS5773.1_cds_0_0_chr7_116714100_f 0 + 53 1.0 +chr7 116945541 116945787 CCDS5774.1_cds_0_0_chr7_116945542_r 0 - 246 1.0 +chr8 118881131 118881317 CCDS6324.1_cds_0_0_chr8_118881132_r 0 - 186 1.0 +chr9 128764156 128764189 CCDS6914.1_cds_0_0_chr9_128764157_f 0 + 33 1.0 +chr9 128787519 128789136 CCDS6915.1_cds_0_0_chr9_128787520_r 0 - 1617 1.0 +chr9 128882427 128882523 CCDS6917.1_cds_0_0_chr9_128882428_f 0 + 0 0.0 +chr9 128937229 128937445 CCDS6919.1_cds_0_0_chr9_128937230_r 0 - 0 0.0 +chrX 122745047 122745924 CCDS14606.1_cds_0_0_chrX_122745048_f 0 + 877 1.0 +chrX 152648964 152649196 CCDS14733.1_cds_0_0_chrX_152648965_r 0 - 232 1.0 +chrX 152691446 152691471 CCDS14735.1_cds_0_0_chrX_152691447_f 0 + 25 1.0 +chrX 152694029 152694263 CCDS14736.1_cds_0_0_chrX_152694030_r 0 - 234 1.0 diff -r 4e44d29377e3 -r 0da612f8a78e test-data/gops_intersect_diffCols.bed --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/gops_intersect_diffCols.bed Thu Oct 30 16:03:11 2008 -0400 @@ -0,0 +1,49 @@ +chr1 147962192 147962580 CCDS989.1_cds_0_0_chr1_147962193_r 0 - +chr1 147984545 147984630 CCDS990.1_cds_0_0_chr1_147984546_f 0 + +chr1 148078400 148078582 CCDS993.1_cds_0_0_chr1_148078401_r 0 - +chr1 148185136 148185276 CCDS996.1_cds_0_0_chr1_148185137_f 0 + +chr10 55251623 55253124 CCDS7248.1_cds_0_0_chr10_55251624_r 0 - +chr11 116124407 116124501 CCDS8374.1_cds_0_0_chr11_116124408_r 0 - +chr11 116206508 116206563 CCDS8377.1_cds_0_0_chr11_116206509_f 0 + +chr11 116211733 116212337 CCDS8378.1_cds_0_0_chr11_116211734_r 0 - +chr12 38440094 38440321 CCDS8736.1_cds_0_0_chr12_38440095_r 0 - +chr13 112381694 112381953 CCDS9526.1_cds_0_0_chr13_112381695_f 0 + +chr14 98710240 98712285 CCDS9949.1_cds_0_0_chr14_98710241_r 0 - +chr15 41486872 41487060 CCDS10096.1_cds_0_0_chr15_41486873_r 0 - +chr15 41673708 41673857 CCDS10097.1_cds_0_0_chr15_41673709_f 0 + +chr15 41679161 41679250 CCDS10098.1_cds_0_0_chr15_41679162_r 0 - +chr16 142908 143003 CCDS10397.1_cds_0_0_chr16_142909_f 0 + +chr16 244413 244681 CCDS10402.1_cds_0_0_chr16_244414_f 0 + +chr16 259268 259383 CCDS10403.1_cds_0_0_chr16_259269_r 0 - +chr18 23786114 23786321 CCDS11891.1_cds_0_0_chr18_23786115_r 0 - +chr18 59406881 59407046 CCDS11985.1_cds_0_0_chr18_59406882_f 0 + +chr18 59455932 59456337 CCDS11986.1_cds_0_0_chr18_59455933_r 0 - +chr19 59068595 59069564 CCDS12866.1_cds_0_0_chr19_59068596_f 0 + +chr19 59236026 59236146 CCDS12872.1_cds_0_0_chr19_59236027_r 0 - +chr19 59297998 59298008 CCDS12877.1_cds_0_0_chr19_59297999_f 0 + +chr2 118288583 118288668 CCDS2120.1_cds_0_0_chr2_118288584_f 0 + +chr2 220229609 220230869 CCDS2443.1_cds_0_0_chr2_220229610_r 0 - +chr20 33330413 33330423 CCDS13249.1_cds_0_0_chr20_33330414_r 0 - +chr20 33513606 33513792 CCDS13255.1_cds_0_0_chr20_33513607_f 0 + +chr21 32869641 32870022 CCDS13615.1_cds_0_0_chr21_32869642_r 0 - +chr21 33321040 33322012 CCDS13620.1_cds_0_0_chr21_33321041_f 0 + +chr22 30120223 30120265 CCDS13897.1_cds_0_0_chr22_30120224_f 0 + +chr22 30160419 30160661 CCDS13898.1_cds_0_0_chr22_30160420_r 0 - +chr5 131424298 131424460 CCDS4149.1_cds_0_0_chr5_131424299_f 0 + +chr5 131556601 131556672 CCDS4151.1_cds_0_0_chr5_131556602_r 0 - +chr5 131621326 131621419 CCDS4152.1_cds_0_0_chr5_131621327_f 0 + +chr6 108299600 108299744 CCDS5061.1_cds_0_0_chr6_108299601_r 0 - +chr6 108594662 108594687 CCDS5063.1_cds_0_0_chr6_108594663_f 0 + +chr6 108640045 108640151 CCDS5064.1_cds_0_0_chr6_108640046_r 0 - +chr6 108722976 108723115 CCDS5067.1_cds_0_0_chr6_108722977_f 0 + +chr7 113660517 113660685 CCDS5760.1_cds_0_0_chr7_113660518_f 0 + +chr7 116512159 116512389 CCDS5771.1_cds_0_0_chr7_116512160_r 0 - +chr7 116714099 116714152 CCDS5773.1_cds_0_0_chr7_116714100_f 0 + +chr7 116945541 116945787 CCDS5774.1_cds_0_0_chr7_116945542_r 0 - +chr8 118881131 118881317 CCDS6324.1_cds_0_0_chr8_118881132_r 0 - +chr9 128764156 128764189 CCDS6914.1_cds_0_0_chr9_128764157_f 0 + +chr9 128787519 128789136 CCDS6915.1_cds_0_0_chr9_128787520_r 0 - +chrX 122745047 122745924 CCDS14606.1_cds_0_0_chrX_122745048_f 0 + +chrX 152648964 152649196 CCDS14733.1_cds_0_0_chrX_152648965_r 0 - +chrX 152691446 152691471 CCDS14735.1_cds_0_0_chrX_152691447_f 0 + +chrX 152694029 152694263 CCDS14736.1_cds_0_0_chrX_152694030_r 0 - diff -r 4e44d29377e3 -r 0da612f8a78e test-data/gops_intersect_p_diffCols.bed --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/gops_intersect_p_diffCols.bed Thu Oct 30 16:03:11 2008 -0400 @@ -0,0 +1,49 @@ +chr1 147962192 147962580 CCDS989.1_cds_0_0_chr1_147962193_r 0 - +chr1 147984545 147984630 CCDS990.1_cds_0_0_chr1_147984546_f 0 + +chr1 148078400 148078582 CCDS993.1_cds_0_0_chr1_148078401_r 0 - +chr1 148185136 148185276 CCDS996.1_cds_0_0_chr1_148185137_f 0 + +chr10 55251623 55253124 CCDS7248.1_cds_0_0_chr10_55251624_r 0 - +chr11 116124407 116124501 CCDS8374.1_cds_0_0_chr11_116124408_r 0 - +chr11 116206508 116206563 CCDS8377.1_cds_0_0_chr11_116206509_f 0 + +chr11 116211733 116212337 CCDS8378.1_cds_0_0_chr11_116211734_r 0 - +chr12 38440094 38440321 CCDS8736.1_cds_0_0_chr12_38440095_r 0 - +chr13 112381694 112381953 CCDS9526.1_cds_0_0_chr13_112381695_f 0 + +chr14 98710240 98712285 CCDS9949.1_cds_0_0_chr14_98710241_r 0 - +chr15 41486872 41487060 CCDS10096.1_cds_0_0_chr15_41486873_r 0 - +chr15 41673708 41673857 CCDS10097.1_cds_0_0_chr15_41673709_f 0 + +chr15 41679161 41679250 CCDS10098.1_cds_0_0_chr15_41679162_r 0 - +chr16 142908 143003 CCDS10397.1_cds_0_0_chr16_142909_f 0 + +chr16 244413 244681 CCDS10402.1_cds_0_0_chr16_244414_f 0 + +chr16 259268 259383 CCDS10403.1_cds_0_0_chr16_259269_r 0 - +chr18 23786114 23786321 CCDS11891.1_cds_0_0_chr18_23786115_r 0 - +chr18 59406881 59407046 CCDS11985.1_cds_0_0_chr18_59406882_f 0 + +chr18 59455932 59456337 CCDS11986.1_cds_0_0_chr18_59455933_r 0 - +chr19 59068595 59069564 CCDS12866.1_cds_0_0_chr19_59068596_f 0 + +chr19 59236026 59236146 CCDS12872.1_cds_0_0_chr19_59236027_r 0 - +chr19 59297998 59298008 CCDS12877.1_cds_0_0_chr19_59297999_f 0 + +chr2 118288583 118288668 CCDS2120.1_cds_0_0_chr2_118288584_f 0 + +chr2 220229609 220230869 CCDS2443.1_cds_0_0_chr2_220229610_r 0 - +chr20 33330413 33330423 CCDS13249.1_cds_0_0_chr20_33330414_r 0 - +chr20 33513606 33513792 CCDS13255.1_cds_0_0_chr20_33513607_f 0 + +chr21 32869641 32870022 CCDS13615.1_cds_0_0_chr21_32869642_r 0 - +chr21 33321040 33322012 CCDS13620.1_cds_0_0_chr21_33321041_f 0 + +chr22 30120223 30120265 CCDS13897.1_cds_0_0_chr22_30120224_f 0 + +chr22 30160419 30160661 CCDS13898.1_cds_0_0_chr22_30160420_r 0 - +chr5 131424298 131424460 CCDS4149.1_cds_0_0_chr5_131424299_f 0 + +chr5 131556601 131556672 CCDS4151.1_cds_0_0_chr5_131556602_r 0 - +chr5 131621326 131621419 CCDS4152.1_cds_0_0_chr5_131621327_f 0 + +chr6 108299600 108299744 CCDS5061.1_cds_0_0_chr6_108299601_r 0 - +chr6 108594662 108594687 CCDS5063.1_cds_0_0_chr6_108594663_f 0 + +chr6 108640045 108640151 CCDS5064.1_cds_0_0_chr6_108640046_r 0 - +chr6 108722976 108723115 CCDS5067.1_cds_0_0_chr6_108722977_f 0 + +chr7 113660517 113660685 CCDS5760.1_cds_0_0_chr7_113660518_f 0 + +chr7 116512159 116512389 CCDS5771.1_cds_0_0_chr7_116512160_r 0 - +chr7 116714099 116714152 CCDS5773.1_cds_0_0_chr7_116714100_f 0 + +chr7 116945541 116945787 CCDS5774.1_cds_0_0_chr7_116945542_r 0 - +chr8 118881131 118881317 CCDS6324.1_cds_0_0_chr8_118881132_r 0 - +chr9 128764156 128764189 CCDS6914.1_cds_0_0_chr9_128764157_f 0 + +chr9 128787519 128789136 CCDS6915.1_cds_0_0_chr9_128787520_r 0 - +chrX 122745047 122745924 CCDS14606.1_cds_0_0_chrX_122745048_f 0 + +chrX 152648964 152649196 CCDS14733.1_cds_0_0_chrX_152648965_r 0 - +chrX 152691446 152691471 CCDS14735.1_cds_0_0_chrX_152691447_f 0 + +chrX 152694029 152694263 CCDS14736.1_cds_0_0_chrX_152694030_r 0 - diff -r 4e44d29377e3 -r 0da612f8a78e test-data/gops_merge_diffCols.dat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/gops_merge_diffCols.dat Thu Oct 30 16:03:11 2008 -0400 @@ -0,0 +1,69 @@ +#chr name score strand start end +chr7 113660517 113660685 +chr7 116512159 116512389 +chr7 116714099 116714152 +chr7 116945541 116945787 +chr6 108299600 108299744 +chr6 108594662 108594687 +chr6 108640045 108640151 +chr6 108722976 108723115 +chr5 131311206 131311254 +chr5 131424298 131424460 +chr5 131556601 131556672 +chr5 131621326 131621419 +chrX 122745047 122745924 +chrX 152648964 152649196 +chrX 152691446 152691471 +chrX 152694029 152694263 +chr2 118288583 118288668 +chr2 118390395 118390500 +chr2 220108689 220109267 +chr2 220229609 220230869 +chr1 147962192 147962580 +chr1 147984545 147984630 +chr1 148078400 148078582 +chr1 148185136 148185276 +chr21 32687402 32687588 +chr21 32869641 32870022 +chr21 33321040 33322012 +chr21 33728358 33728724 +chr9 128764156 128764189 +chr9 128787519 128789136 +chr9 128789552 128789584 +chr9 128850516 128850624 +chr8 118881131 118881317 +chr13 29680676 29680875 +chr13 112381694 112381953 +chr12 38440094 38440321 +chr12 38905200 38905351 +chr11 116124407 116124501 +chr11 116206508 116206563 +chr11 116211733 116212337 +chr11 130745911 130745993 +chr10 55251623 55253124 +chr22 30120223 30120265 +chr22 30160419 30160661 +chr22 30228824 30228916 +chr22 30340151 30340376 +chr16 142908 143003 +chr16 179197 179339 +chr16 244413 244681 +chr16 259268 259383 +chr15 41486872 41487060 +chr15 41673708 41673857 +chr15 41679161 41679250 +chr15 41773540 41773689 +chr14 98521864 98521922 +chr14 98710240 98712285 +chr20 33330413 33330423 +chr20 33485370 33486123 +chr20 33488491 33489122 +chr20 33513606 33513792 +chr19 59068595 59069564 +chr19 59236026 59236146 +chr19 59297998 59298008 +chr19 59318205 59318718 +chr18 23786114 23786321 +chr18 59406881 59407046 +chr18 59455932 59456337 +chr18 59528407 59528575 diff -r 4e44d29377e3 -r 0da612f8a78e test-data/gops_subtract_diffCols.dat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/gops_subtract_diffCols.dat Thu Oct 30 16:03:11 2008 -0400 @@ -0,0 +1,16 @@ +chr11 1812377 1812407 CCDS7726.1_cds_0_0_chr11_1812378_f 0 + +chr15 41826029 41826196 CCDS10101.1_cds_0_0_chr15_41826030_f 0 + +chr16 179963 180135 CCDS10401.1_cds_0_0_chr16_179964_r 0 - +chr18 59600586 59600754 CCDS11988.1_cds_0_0_chr18_59600587_f 0 + +chr19 59302168 59302288 CCDS12878.1_cds_0_0_chr19_59302169_r 0 - +chr2 118394148 118394202 CCDS2121.1_cds_0_0_chr2_118394149_r 0 - +chr2 220190202 220190242 CCDS2441.1_cds_0_0_chr2_220190203_f 0 + +chr20 33579500 33579527 CCDS13256.1_cds_0_0_chr20_33579501_r 0 - +chr20 33593260 33593348 CCDS13257.1_cds_0_0_chr20_33593261_f 0 + +chr21 32707032 32707192 CCDS13614.1_cds_0_0_chr21_32707033_f 0 + +chr21 33744994 33745040 CCDS13625.1_cds_0_0_chr21_33744995_r 0 - +chr22 30665273 30665360 CCDS13901.1_cds_0_0_chr22_30665274_f 0 + +chr22 30939054 30939266 CCDS13903.1_cds_0_0_chr22_30939055_r 0 - +chr5 131847541 131847666 CCDS4155.1_cds_0_0_chr5_131847542_r 0 - +chr9 128882427 128882523 CCDS6917.1_cds_0_0_chr9_128882428_f 0 + +chr9 128937229 128937445 CCDS6919.1_cds_0_0_chr9_128937230_r 0 - diff -r 4e44d29377e3 -r 0da612f8a78e tools/new_operations/complement.xml --- a/tools/new_operations/complement.xml Thu Oct 30 14:46:57 2008 -0400 +++ b/tools/new_operations/complement.xml Thu Oct 30 16:03:11 2008 -0400 @@ -17,6 +17,11 @@ <param name="input1" value="1.bed" /> <param name="allchroms" value="true" /> <output name="output" file="gops_complement_out.bed" /> + </test> + <test> + <param name="input1" value="2_mod.bed" ftype="interval"/> + <param name="allchroms" value="true" /> + <output name="output" file="gops_complement_out_diffCols.dat" /> </test> <test> <param name="input1" value="gops_bigint.interval" /> diff -r 4e44d29377e3 -r 0da612f8a78e tools/new_operations/coverage.xml --- a/tools/new_operations/coverage.xml Thu Oct 30 14:46:57 2008 -0400 +++ b/tools/new_operations/coverage.xml Thu Oct 30 16:03:11 2008 -0400 @@ -18,6 +18,11 @@ <param name="input1" value="1.bed" /> <param name="input2" value="2.bed" /> <output name="output" file="gops_coverage_out.interval" /> + </test> + <test> + <param name="input1" value="1.bed" /> + <param name="input2" value="2_mod.bed" ftype="interval"/> + <output name="output" file="gops_coverage_out_diffCols.interval" /> </test> <test> <param name="input1" value="gops_bigint.interval" /> diff -r 4e44d29377e3 -r 0da612f8a78e tools/new_operations/intersect.xml --- a/tools/new_operations/intersect.xml Thu Oct 30 14:46:57 2008 -0400 +++ b/tools/new_operations/intersect.xml Thu Oct 30 16:03:11 2008 -0400 @@ -21,13 +21,27 @@ <data format="input" name="output" metadata_source="input1" /> </outputs> <code file="operation_filter.py"/> - <tests> + <tests> + <test> + <param name="input1" value="1.bed" /> + <param name="input2" value="2.bed" /> + <param name="min" value="1" /> + <param name="returntype" value="" /> + <output name="output" file="gops_intersect_out.bed" /> + </test> <test> <param name="input1" value="1.bed" /> - <param name="input2" value="2.bed" /> + <param name="input2" value="2_mod.bed" ftype="interval"/> <param name="min" value="1" /> - <param name="returntype" value="" /> - <output name="output" file="gops_intersect_out.bed" /> + <param name="returntype" value="" /> + <output name="output" file="gops_intersect_diffCols.bed" /> + </test> + <test> + <param name="input1" value="1.bed" /> + <param name="input2" value="2_mod.bed" ftype="interval"/> + <param name="min" value="1" /> + <param name="returntype" value="Overlapping pieces of Intervals" /> + <output name="output" file="gops_intersect_p_diffCols.bed" /> </test> <test> <param name="input1" value="1.bed" /> diff -r 4e44d29377e3 -r 0da612f8a78e tools/new_operations/merge.xml --- a/tools/new_operations/merge.xml Thu Oct 30 14:46:57 2008 -0400 +++ b/tools/new_operations/merge.xml Thu Oct 30 16:03:11 2008 -0400 @@ -19,6 +19,11 @@ <test> <param name="input1" value="1.bed" /> <output name="output" file="gops-merge.dat" /> + <param name="returntype" value="true" /> + </test> + <test> + <param name="input1" value="2_mod.bed" ftype="interval"/> + <output name="output" file="gops_merge_diffCols.dat" /> <param name="returntype" value="true" /> </test> <test> diff -r 4e44d29377e3 -r 0da612f8a78e tools/new_operations/subtract.xml --- a/tools/new_operations/subtract.xml Thu Oct 30 14:46:57 2008 -0400 +++ b/tools/new_operations/subtract.xml Thu Oct 30 16:03:11 2008 -0400 @@ -31,6 +31,13 @@ <param name="min" value="1" /> <param name="returntype" value="" /> <output name="output" file="gops-subtract.dat" /> + </test> + <test> + <param name="input1" value="1.bed" /> + <param name="input2" value="2_mod.bed" ftype="interval"/> + <param name="min" value="1" /> + <param name="returntype" value="" /> + <output name="output" file="gops_subtract_diffCols.dat" /> </test> <test> <param name="input1" value="gops_subtract_bigint.bed" />

1 0