October 2008 - galaxy-dev - lists.galaxyproject.org

[hg] galaxy 1579: Add new EpiGRAPH tools that point to EpiGRAPH ...
by Greg Von Kuster 29 Oct '08

29 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/778fcc385ab7 changeset: 1579:778fcc385ab7 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Tue Oct 28 11:27:42 2008 -0400 description: Add new EpiGRAPH tools that point to EpiGRAPH production server. 6 file(s) affected in this change: tool_conf.xml.main tool_conf.xml.sample tools/data_destination/epigraph.xml tools/data_destination/epigraph_test.xml tools/data_source/epigraph_import.xml tools/data_source/epigraph_import_test.xml diffs (144 lines): diff -r 75312c1cab05 -r 778fcc385ab7 tool_conf.xml.main --- a/tool_conf.xml.main Tue Oct 28 10:55:39 2008 -0400 +++ b/tool_conf.xml.main Tue Oct 28 11:27:42 2008 -0400 @@ -7,6 +7,7 @@ <tool file="data_source/microbial_import.xml" /> <tool file="data_source/biomart.xml" /> <tool file="data_source/encode_db.xml" /> + <tool file="data_source/epigraph_import.xml" /> </section> <section name="Get ENCODE Data" id="encode"> <tool file="data_source/encode_import_chromatin_and_chromosomes.xml"/> @@ -15,6 +16,9 @@ <tool file="data_source/encode_import_transcription_regulation.xml"/> <tool file="data_source/encode_import_all_latest_datasets.xml" /> <tool file="data_source/encode_import_gencode.xml" /> + </section> + <section name="Send Data" id="send"> + <tool file="data_destination/epigraph.xml" /> </section> <section name="ENCODE Tools" id="EncodeTools"> <tool file="encode/gencode_partition.xml" /> diff -r 75312c1cab05 -r 778fcc385ab7 tool_conf.xml.sample --- a/tool_conf.xml.sample Tue Oct 28 10:55:39 2008 -0400 +++ b/tool_conf.xml.sample Tue Oct 28 11:27:42 2008 -0400 @@ -11,8 +11,9 @@ <tool file="data_source/biomart_test.xml" /> <tool file="data_source/gbrowse_elegans.xml" /> <tool file="data_source/flymine.xml" /> + <tool file="data_source/encode_db.xml" /> <tool file="data_source/epigraph_import.xml" /> - <tool file="data_source/encode_db.xml" /> + <tool file="data_source/epigraph_import_test.xml" /> <tool file="data_source/hbvar.xml" /> <tool file="validation/fix_errors.xml" /> </section> @@ -26,6 +27,7 @@ </section> <section name="Send Data" id="send"> <tool file="data_destination/epigraph.xml" /> + <tool file="data_destination/epigraph_test.xml" /> </section> <section name="ENCODE Tools" id="EncodeTools"> <tool file="encode/gencode_partition.xml" /> diff -r 75312c1cab05 -r 778fcc385ab7 tools/data_destination/epigraph.xml --- a/tools/data_destination/epigraph.xml Tue Oct 28 10:55:39 2008 -0400 +++ b/tools/data_destination/epigraph.xml Tue Oct 28 11:27:42 2008 -0400 @@ -6,7 +6,7 @@ <param format="bed" name="input1" type="data" label="Send this dataset to EpiGRAPH"> <validator type="unspecified_build" /> </param> - <param name="REDIRECT_URL" type="hidden" value="http://epigraph.mpi-inf.mpg.de/WebGRAPH_Public_Test/faces/DataImport.jsp" /> + <param name="REDIRECT_URL" type="hidden" value="http://epigraph.mpi-inf.mpg.de/WebGRAPH/faces/DataImport.jsp" /> <param name="DATA_URL" type="baseurl" value="/datasets" /> <param name="GALAXY_URL" type="baseurl" value="/tool_runner?tool_id=epigraph_import" /> </inputs> diff -r 75312c1cab05 -r 778fcc385ab7 tools/data_destination/epigraph_test.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/data_destination/epigraph_test.xml Tue Oct 28 11:27:42 2008 -0400 @@ -0,0 +1,40 @@ +<?xml version="1.0"?> +<tool name="Perform genome analysis" id="epigraph_test_export"> + <description> and prediction with EpiGRAPH Test</description> + <redirect_url_params>GENOME=${input1.dbkey} NAME=${input1.name} INFO=${input1.info}</redirect_url_params> + <inputs> + <param format="bed" name="input1" type="data" label="Send this dataset to EpiGRAPH"> + <validator type="unspecified_build" /> + </param> + <param name="REDIRECT_URL" type="hidden" value="http://epigraph.mpi-inf.mpg.de/WebGRAPH_Public_Test/faces/DataImport.jsp" /> + <param name="DATA_URL" type="baseurl" value="/datasets" /> + <param name="GALAXY_URL" type="baseurl" value="/tool_runner?tool_id=epigraph_import" /> + </inputs> + <outputs/> + <help> + +.. class:: warningmark + +After clicking the **Execute** button, you will be redirected to the EpiGRAPH test website. Please be patient while the dataset is being imported. Inside EpiGRAPH, buttons are available to send the results of the EpiGRAPH analysis back to Galaxy. In addition, you can always abandon an EpiGRAPH session and return to Galaxy by directing your browser to your current Galaxy instance. + +----- + +.. class:: infomark + +**What it does** + +This tool sends the selected dataset to EpiGRAPH in order to perform an in-depth analysis with statistical and machine learning methods. + +----- + +.. class:: infomark + +**EpiGRAPH outline** + +The EpiGRAPH_ web service enables biologists to uncover hidden associations in vertebrate genome and epigenome datasets. Users can upload or import sets of genomic regions and EpiGRAPH will test a wide range of attributes (including DNA sequence and structure, gene density, chromatin modifications and evolutionary conservation) for enrichment or depletion among these regions. Furthermore, EpiGRAPH learns to predictively identify genomic regions that exhibit similar properties. + +.. _EpiGRAPH: http://epigraph.mpi-inf.mpg.de/ + + </help> +</tool> + diff -r 75312c1cab05 -r 778fcc385ab7 tools/data_source/epigraph_import.xml --- a/tools/data_source/epigraph_import.xml Tue Oct 28 10:55:39 2008 -0400 +++ b/tools/data_source/epigraph_import.xml Tue Oct 28 11:27:42 2008 -0400 @@ -7,7 +7,7 @@ <tool name="EpiGRAPH" id="epigraph_import" tool_type="data_source" URL_method="get"> <description> server</description> <command interpreter="python">data_source.py $output</command> - <inputs action="http://epigraph.mpi-inf.mpg.de/WebGRAPH_Public_Test/faces/Login.jsp" check_values="false" method="get"> + <inputs action="http://epigraph.mpi-inf.mpg.de/WebGRAPH/faces/Login.jsp" check_values="false" method="get"> <display>go to EpiGRAPH server $GALAXY_URL</display> <param name="GALAXY_URL" type="baseurl" value="/tool_runner?tool_id=epigraph_import" /> </inputs> diff -r 75312c1cab05 -r 778fcc385ab7 tools/data_source/epigraph_import_test.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/data_source/epigraph_import_test.xml Tue Oct 28 11:27:42 2008 -0400 @@ -0,0 +1,29 @@ +<?xml version="1.0"?> + +<tool name="EpiGRAPH" id="epigraph_import_test" tool_type="data_source" URL_method="get"> + <description> test server</description> + <command interpreter="python">data_source.py $output</command> + <inputs action="http://epigraph.mpi-inf.mpg.de/WebGRAPH_Public_Test/faces/Login.jsp" check_values="false" method="get"> + <display>go to EpiGRAPH server $GALAXY_URL</display> + <param name="GALAXY_URL" type="baseurl" value="/tool_runner?tool_id=epigraph_import_test" /> + </inputs> + <request_param_translation> + <request_param galaxy_name="URL" remote_name="URL" missing="" /> + <request_param galaxy_name="dbkey" remote_name="GENOME" missing="?" /> + <request_param galaxy_name="organism" remote_name="organism" missing="" /> + <request_param galaxy_name="table" remote_name="table" missing="" /> + <request_param galaxy_name="description" remote_name="description" missing="" /> + <request_param galaxy_name="name" remote_name="NAME" missing="EpiGRAPH query" /> + <request_param galaxy_name="info" remote_name="INFO" missing="" /> + <request_param galaxy_name="data_type" remote_name="data_type" missing="txt" /> + </request_param_translation> + <uihints minwidth="800"/> + <outputs> + <data name="output" format="txt" /> + </outputs> + <options sanitize="False" refresh="True"/> +</tool>

1 0

[hg] galaxy 1578: Tweaks to my last commit - only persist job st...
by Greg Von Kuster 29 Oct '08

29 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/75312c1cab05 changeset: 1578:75312c1cab05 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Tue Oct 28 10:55:39 2008 -0400 description: Tweaks to my last commit - only persist job states defined in model, not job. 1 file(s) affected in this change: lib/galaxy/jobs/__init__.py diffs (71 lines): diff -r 0f4fd4c20cd6 -r 75312c1cab05 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Tue Oct 28 10:21:02 2008 -0400 +++ b/lib/galaxy/jobs/__init__.py Tue Oct 28 10:55:39 2008 -0400 @@ -117,8 +117,8 @@ while self.running: try: self.monitor_step() - except: - log.exception( "Exception in monitor_step" ) + except Exception, e: + log.exception( "Exception in monitor_step: %s" % str( e ) ) # Sleep self.sleeper.sleep( 1 ) @@ -308,7 +308,7 @@ self.extra_filenames = extra_filenames return extra_filenames - def fail( self, message, state=None, exception=False ): + def fail( self, message, exception=False ): """ Indicate job failure by setting state and message on all output datasets. @@ -316,7 +316,7 @@ job = model.Job.get( self.job_id ) job.refresh() # if the job was deleted, don't fail it - if not job.state == job.states.DELETED: + if not job.state == model.Job.states.DELETED: for dataset_assoc in job.output_datasets: dataset = dataset_assoc.dataset dataset.refresh() @@ -325,10 +325,7 @@ dataset.info = message dataset.set_size() dataset.flush() - if state is not None: - job.state = state - else: - job.state = model.Job.states.ERROR + job.state = model.Job.states.ERROR job.command_line = self.command_line job.info = message # If the failure is due to a Galaxy framework exception, save the traceback @@ -385,13 +382,11 @@ idata.dataset.refresh() #we need to refresh the base Dataset, since that is where 'state' is stored # don't run jobs for which the input dataset was deleted if idata.deleted: - msg = "input data %d was deleted before this job started" % idata.hid - self.fail( msg, state=JOB_INPUT_DELETED ) + self.fail( "input data %d was deleted before this job started" % idata.hid ) return JOB_INPUT_DELETED # an error in the input data causes us to bail immediately elif idata.state == idata.states.ERROR: - msg = "input data %d is in an error state" % idata.hid - self.fail( msg, state=JOB_INPUT_ERROR ) + self.fail( "input data %d is in an error state" % idata.hid ) return JOB_INPUT_ERROR elif idata.state != idata.states.OK: # need to requeue @@ -578,8 +573,8 @@ while self.running: try: self.monitor_step() - except: - log.exception( "Exception in monitor_step" ) + except Exception, e: + log.exception( "Exception in monitor_step: %s" % str( e ) ) # Sleep self.sleeper.sleep( 1 )

1 0

[hg] galaxy 1577: Improve job error messaging, some fixes for se...
by Greg Von Kuster 29 Oct '08

29 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/0f4fd4c20cd6 changeset: 1577:0f4fd4c20cd6 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Tue Oct 28 10:21:02 2008 -0400 description: Improve job error messaging, some fixes for setting job state, job info, dataset state, and dataset info when job ends in error. 2 file(s) affected in this change: lib/galaxy/jobs/__init__.py templates/dataset/errors.tmpl diffs (293 lines): diff -r 8eec48aaca6e -r 0f4fd4c20cd6 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Mon Oct 27 17:03:50 2008 -0400 +++ b/lib/galaxy/jobs/__init__.py Tue Oct 28 10:21:02 2008 -0400 @@ -176,11 +176,17 @@ self.dispatcher.put( job ) log.debug( "job %d dispatched" % job.job_id) elif job_state == JOB_DELETED: - log.debug( "job %d deleted by user while still queued" % job.job_id ) + msg = "job %d deleted by user while still queued" % job.job_id + job.info = msg + log.debug( msg ) else: - log.error( "unknown job state '%s' for job %d" % ( job_state, job.job_id )) - except: - log.exception( "failure running job %d" % job.job_id ) + msg = "unknown job state '%s' for job %d" % ( job_state, job.job_id ) + job.info = msg + log.error( msg ) + except Exception, e: + msg = "failure running job %d: %s" % ( job.job_id, str( e ) ) + job.info = msg + log.exception( msg ) # Update the waiting list self.waiting = new_waiting # If special (e.g. fair) scheduling is enabled, dispatch all jobs @@ -194,9 +200,10 @@ except Empty: # squeue is empty, so stop dispatching break - except: # if something else breaks while dispatching - job.fail( "failure dispatching job" ) - log.exception( "failure running job %d" % sjob.job_id ) + except Exception, e: # if something else breaks while dispatching + msg = "failure running job %d: %s" % ( sjob.job_id, str( e ) ) + job.fail( msg ) + log.exception( msg ) def put( self, job_id, tool ): """Add a job to the queue (by job identifier)""" @@ -301,7 +308,7 @@ self.extra_filenames = extra_filenames return extra_filenames - def fail( self, message, exception=False ): + def fail( self, message, state=None, exception=False ): """ Indicate job failure by setting state and message on all output datasets. @@ -309,25 +316,26 @@ job = model.Job.get( self.job_id ) job.refresh() # if the job was deleted, don't fail it - if job.state == job.states.DELETED: - self.cleanup() - return - for dataset_assoc in job.output_datasets: - dataset = dataset_assoc.dataset - dataset.refresh() - dataset.state = dataset.states.ERROR - dataset.blurb = 'tool error' - dataset.info = message - dataset.set_size() - dataset.flush() - job.state = model.Job.states.ERROR - job.command_line = self.command_line - job.info = message - # If the failure is due to a Galaxy framework exception, save - # the traceback - if exception: - job.traceback = traceback.format_exc() - job.flush() + if not job.state == job.states.DELETED: + for dataset_assoc in job.output_datasets: + dataset = dataset_assoc.dataset + dataset.refresh() + dataset.state = dataset.states.ERROR + dataset.blurb = 'tool error' + dataset.info = message + dataset.set_size() + dataset.flush() + if state is not None: + job.state = state + else: + job.state = model.Job.states.ERROR + job.command_line = self.command_line + job.info = message + # If the failure is due to a Galaxy framework exception, save the traceback + if exception: + job.traceback = traceback.format_exc() + job.flush() + # If the job was deleted, just clean up self.cleanup() def change_state( self, state, info = False ): @@ -371,16 +379,19 @@ job.refresh() for dataset_assoc in job.input_datasets: idata = dataset_assoc.dataset - if not idata: continue + if not idata: + continue idata.refresh() idata.dataset.refresh() #we need to refresh the base Dataset, since that is where 'state' is stored # don't run jobs for which the input dataset was deleted - if idata.deleted == True: - self.fail( "input data %d was deleted before this job ran" % idata.hid ) + if idata.deleted: + msg = "input data %d was deleted before this job started" % idata.hid + self.fail( msg, state=JOB_INPUT_DELETED ) return JOB_INPUT_DELETED # an error in the input data causes us to bail immediately elif idata.state == idata.states.ERROR: - self.fail( "error in input data %d" % idata.hid ) + msg = "input data %d is in an error state" % idata.hid + self.fail( msg, state=JOB_INPUT_ERROR ) return JOB_INPUT_ERROR elif idata.state != idata.states.OK: # need to requeue @@ -467,8 +478,8 @@ os.remove( fname ) if self.working_directory is not None: os.rmdir( self.working_directory ) - except: - log.exception( "Unable to cleanup job %s" % self.job_id ) + except Exception, e: + log.exception( "Unable to cleanup job %s, exception: %s" % ( str( self.job_id ), str( e ) ) ) def get_command_line( self ): return self.command_line @@ -617,7 +628,7 @@ job = model.Job.get( job_id ) job.refresh() job.state = job.states.DELETED - job.info = "Job deleted by user before it completed." + job.info = "Job output deleted by user before job completed." job.flush() for dataset_assoc in job.output_datasets: dataset = dataset_assoc.dataset @@ -630,7 +641,7 @@ dataset.deleted = True dataset.blurb = 'deleted' dataset.peek = 'Job deleted' - dataset.info = 'Job deleted by user before it completed' + dataset.info = 'Job output deleted by user before job completed' dataset.flush() def put( self, job ): diff -r 8eec48aaca6e -r 0f4fd4c20cd6 templates/dataset/errors.tmpl --- a/templates/dataset/errors.tmpl Mon Oct 27 17:03:50 2008 -0400 +++ b/templates/dataset/errors.tmpl Tue Oct 28 10:21:02 2008 -0400 @@ -1,79 +1,69 @@ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> + <head> + <title>Dataset generation errors</title> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> + <link href="/static/style/base.css" rel="stylesheet" type="text/css" /> + <style> + pre + { + background: white; + color: black; + border: dotted black 1px; + overflow: auto; + padding: 10px; + } + </style> + </head> -<head> -<title>Dataset generation errors</title> -<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> -<link href="/static/style/base.css" rel="stylesheet" type="text/css" /> -<style> -pre -{ - background: white; - color: black; - border: dotted black 1px; - overflow: auto; - padding: 10px; -} -</style> -</head> + <body> + <h2>Dataset generation errors</h2> + <p><b>Dataset $dataset.hid: $dataset.display_name</b></p> -<body> - - <h2>Dataset generation errors</h2> - - <p><b>Dataset $dataset.hid: $dataset.display_name</b></p> - - #if $dataset.creating_job_associations - - #set job = $dataset.creating_job_associations[0].job - - #if job.traceback - The Galaxy framework encountered the following error while attempting - to run the tool: + #if $dataset.creating_job_associations + #set job = $dataset.creating_job_associations[0].job + #if job.traceback + The Galaxy framework encountered the following error while attempting to run the tool: + <pre>${job.traceback}</pre> + #end if + #if $job.stderr or $job.info + Tool execution generated the following error message: + #if $job.stderr + <pre>${job.stderr}</pre> + #elif $job.info + <pre>${job.info}</pre> + #end if + #else + Tool execution did not generate any error messages. + #end if + #if $job.stdout + The tool produced the following additional output: + <pre>${job.stdout}</pre> + #end if + #else + The tool did not create any additional job / error info. + #end if - <pre>${job.traceback}</pre> - - #end if - - #if $job.stderr - Tool execution generated the following error message: - <pre>${job.stderr}</pre> - #else - Tool execution did not generate any error messages. - #end if - - #if $job.stdout - The tool produced the following additional output: - <pre>${job.stdout}</pre> - #end if - - #else - - The tool did not create any additional job / error info. - - #end if - - <h2>Report this error to the Galaxy Team</h2> - - <p>The Galaxy team regularly reviews errors that occur in the application. - However, if you would like to provide additional information (such as - what you were trying to do when the error occurred) and a contact e-mail - address, we will be better able to investigate your problem and get back - to you.</p> - - <div class="toolForm"> - <div class="toolFormTitle">Error Report</div> - <div class="toolFormBody"> - <form name="report_error" action="${h.url_for( action='report_error')}" method="post" > - <input type="hidden" name="id" value="$dataset.id" /> - <table> - <tr valign="top"><td>Your Email:</td><td><input type="text" name="email" size="40" /></td></tr> - <tr valign="top"><td>Message:</td><td><textarea name="message", rows="10" cols="40" /></textarea></td></tr> - <tr><td></td><td><input type="submit" value="Report"> - </table> - </form> - </div> - </div> - -</body> + <h2>Report this error to the Galaxy Team</h2> + <p> + The Galaxy team regularly reviews errors that occur in the application. + However, if you would like to provide additional information (such as + what you were trying to do when the error occurred) and a contact e-mail + address, we will be better able to investigate your problem and get back + to you. + </p> + <div class="toolForm"> + <div class="toolFormTitle">Error Report</div> + <div class="toolFormBody"> + <form name="report_error" action="${h.url_for( action='report_error')}" method="post" > + <input type="hidden" name="id" value="$dataset.id" /> + <table> + <tr valign="top"><td>Your Email:</td><td><input type="text" name="email" size="40" /></td></tr> + <tr valign="top"><td>Message:</td><td><textarea name="message", rows="10" cols="40" /></textarea></td></tr> + <tr><td></td><td><input type="submit" value="Report"> + </table> + </form> + </div> + </div> + </body> </html>

1 0

[hg] galaxy 1576: Add a memdump page to the admin controller, ma...
by Nate Coraor 28 Oct '08

28 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/8eec48aaca6e changeset: 1576:8eec48aaca6e user: Nate Coraor <nate(a)bx.psu.edu> date: Mon Oct 27 17:03:50 2008 -0400 description: Add a memdump page to the admin controller, makes heapy output slightly browsable and provides an entry point to heapy (via editing the template) while the server is running. 4 file(s) affected in this change: lib/galaxy/app.py lib/galaxy/util/memdump.py lib/galaxy/web/controllers/admin.py templates/admin/memdump.mako diffs (174 lines): diff -r 415cc6dc8e35 -r 8eec48aaca6e lib/galaxy/app.py --- a/lib/galaxy/app.py Mon Oct 27 16:03:43 2008 -0400 +++ b/lib/galaxy/app.py Mon Oct 27 17:03:50 2008 -0400 @@ -35,6 +35,7 @@ self.job_queue = jobs.JobQueue( self, job_dispatcher ) self.job_stop_queue = jobs.JobStopQueue( self, job_dispatcher ) self.heartbeat = None + self.memdump = None # Start the heartbeat process if configured and available if self.config.use_heartbeat: from galaxy.util import heartbeat diff -r 415cc6dc8e35 -r 8eec48aaca6e lib/galaxy/util/memdump.py --- a/lib/galaxy/util/memdump.py Mon Oct 27 16:03:43 2008 -0400 +++ b/lib/galaxy/util/memdump.py Mon Oct 27 17:03:50 2008 -0400 @@ -22,22 +22,29 @@ self.fname = fname signal.signal( signum, self.dump ) self.heapy = guppy.hpy() + self.heap = None def dump( self, signum, stack ): file = open( self.fname, "a" ) print >> file, "Memdump for pid %d at %s" % ( os.getpid(), time.asctime() ) print >> file try: - h = self.heapy.heap() + self.heap = self.heapy.heap() print >> file, "heap():" - print >> file, h + print >> file, self.heap print >> file, "\nbyrcs:" - print >> file, h.byrcs + print >> file, self.heap.byrcs print >> file, "\nbyrcs[0].byid:" - print >> file, h.byrcs[0].byid + print >> file, self.heap.byrcs[0].byid print >> file, "\nget_rp():" - print >> file, h.get_rp() + print >> file, self.heap.get_rp() self.heapy.setref() except AssertionError: pass print >> file, "\nEnd dump\n" file.close() + def setref( self ): + self.heapy.setref() + def get( self, update=False ): + if update: + self.heap = self.heapy.heap() + return self.heap diff -r 415cc6dc8e35 -r 8eec48aaca6e lib/galaxy/web/controllers/admin.py --- a/lib/galaxy/web/controllers/admin.py Mon Oct 27 16:03:43 2008 -0400 +++ b/lib/galaxy/web/controllers/admin.py Mon Oct 27 17:03:50 2008 -0400 @@ -22,3 +22,42 @@ else: msg = 'Invalid password' return msg + @web.expose + def memdump( self, trans, ids = 'None', sorts = 'None', pages = 'None', new_id = None, new_sort = None, **kwd ): + if self.app.memdump is None: + return trans.show_error_message( "Memdump is not enabled (set <code>use_memdump = True</code> in universe_wsgi.ini)" ) + heap = self.app.memdump.get() + p = util.Params( kwd ) + msg = None + if p.dump: + heap = self.app.memdump.get( update = True ) + msg = "Heap dump complete" + elif p.setref: + self.app.memdump.setref() + msg = "Reference point set (dump to see delta from this point)" + ids = ids.split( ',' ) + sorts = sorts.split( ',' ) + if new_id is not None: + ids.append( new_id ) + sorts.append( 'None' ) + elif new_sort is not None: + sorts[-1] = new_sort + breadcrumb = "<a href='%s' class='breadcrumb'>heap</a>" % web.url_for() + # new lists so we can assemble breadcrumb links + new_ids = [] + new_sorts = [] + for id, sort in zip( ids, sorts ): + new_ids.append( id ) + if id != 'None': + breadcrumb += "<a href='%s' class='breadcrumb'>[%s]</a>" % ( web.url_for( ids=','.join( new_ids ), sorts=','.join( new_sorts ) ), id ) + heap = heap[int(id)] + new_sorts.append( sort ) + if sort != 'None': + breadcrumb += "<a href='%s' class='breadcrumb'>.by('%s')</a>" % ( web.url_for( ids=','.join( new_ids ), sorts=','.join( new_sorts ) ), sort ) + heap = heap.by( sort ) + ids = ','.join( new_ids ) + sorts = ','.join( new_sorts ) + if p.theone: + breadcrumb += ".theone" + heap = heap.theone + return trans.fill_template( '/admin/memdump.mako', heap = heap, ids = ids, sorts = sorts, breadcrumb = breadcrumb, msg = msg ) diff -r 415cc6dc8e35 -r 8eec48aaca6e templates/admin/memdump.mako --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/templates/admin/memdump.mako Mon Oct 27 17:03:50 2008 -0400 @@ -0,0 +1,75 @@ +<%inherit file="/base.mako"/> + +<%def name="title()">Memory Profiling</%def> +<% + import re + from xml.sax.saxutils import escape, unescape +%> + +<style type="text/css"> + a.breadcrumb:link, + a.breadcrumb:visited, + a.breadcrumb:active { + text-decoration: none; + } + a.breadcrumb:hover { + text-decoration: underline; + } +</style> + +<h2>Memory Profiling</h2> + +<ul class="manage-table-actions"> + <li><a class="action-button" href="${h.url_for( controller='admin', action='memdump', dump=True )}">Dump memory (warning: hangs server!)</a></li> + <li><a class="action-button" href="${h.url_for( controller='admin', action='memdump', setref=True )}">Set reference point</a></li> +</ul> + +<%def name="htmlize( heap )"> +<% + s = escape( str( heap ) ) + new_s = "" + id_re = re.compile('^(\s+)([0-9]+)') + for line in s.split( '\n' ): + try: + id = id_re.search( line ).group( 2 ) + except: + id = None + new_s += re.sub( id_re, r'\1<a href="' + h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_id=id ) + r'">\2</a>', line ) + if id and heap[int(id)].count == 1: + new_s += " <a href='%s'>theone</a>\n" % h.url_for( ids=ids, sorts=sorts, new_id=id, theone=True ) + else: + new_s += "\n" + return new_s +%> +</%def> + +%if msg: + <div class="donemessage">${msg}</div> +%endif + +%if heap is None: + No memory dump available. Click "Dump memory" to create one. +%else: + <pre> + <br/> +You are here: ${breadcrumb}<br/> + %if breadcrumb.endswith( 'theone' ): + ${heap} + %else: + <nobr> +Sort: + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Class')}">Class</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Clodo' )}">Clodo</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Id' )}">Id</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Idset' )}">Idset</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Module' )}">Module</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Unity' )}">Unity</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Rcs' )}">Rcs</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Size' )}">Size</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Type' )}">Type</a> | + <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Via' )}">Via</a> + </nobr> + ${htmlize( heap )} + %endif + </pre> +%endif

1 0

[hg] galaxy 1575: Add a new URL_method attribute to data_source ...
by Greg Von Kuster 28 Oct '08

28 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/415cc6dc8e35 changeset: 1575:415cc6dc8e35 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Mon Oct 27 16:03:43 2008 -0400 description: Add a new URL_method attribute to data_source tool types whose value is either "get" or "post" ( some require a get request while others require a post request ). This fixes the Biomart problem ( along with a new, well documented hack that can be eliminated when Biomart encodes the value of URL in the initial response - they'll tell us when they've fixed this ). Also added some requested info to the "send to EpiGRAPH" tool. 12 file(s) affected in this change: lib/galaxy/tools/__init__.py lib/galaxy/util/__init__.py lib/galaxy/web/controllers/tool_runner.py tools/data_destination/epigraph.xml tools/data_source/biomart.xml tools/data_source/biomart_test.xml tools/data_source/data_source.py tools/data_source/epigraph_import.xml tools/data_source/flymine.xml tools/data_source/ucsc_tablebrowser.xml tools/data_source/ucsc_tablebrowser_archaea.xml tools/data_source/ucsc_tablebrowser_test.xml diffs (299 lines): diff -r 36a7ff82faf0 -r 415cc6dc8e35 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Fri Oct 24 12:32:32 2008 -0400 +++ b/lib/galaxy/tools/__init__.py Mon Oct 27 16:03:43 2008 -0400 @@ -228,19 +228,21 @@ self.version = "1.0.0" # Type of tool self.tool_type = root.get( "tool_type", None ) - if self.tool_type is not None: - # data_source tool - if self.tool_type == "data_source": - self.param_trans_dict = {} - req_param_trans = root.find( "request_param_translation" ) - if req_param_trans is not None: - for req_param in req_param_trans.findall( "request_param" ): - # req_param tags must look like <request_param galaxy_name="dbkey" remote_name="GENOME" missing="" /> - trans_list = [] - remote_name = req_param.get( "remote_name" ) - trans_list.append( req_param.get( "galaxy_name" ) ) - trans_list.append( req_param.get( "missing" ) ) - self.param_trans_dict[ remote_name ] = trans_list + # data_source tool + if self.tool_type == "data_source": + self.URL_method = root.get( "URL_method", "get" ) # get is the default + # TODO: Biomart hack - eliminate when they encode URL - they'll let us know when... + self.add_to_URL = root.get( "add_to_URL", None ) + self.param_trans_dict = {} + req_param_trans = root.find( "request_param_translation" ) + if req_param_trans is not None: + for req_param in req_param_trans.findall( "request_param" ): + # req_param tags must look like <request_param galaxy_name="dbkey" remote_name="GENOME" missing="" /> + trans_list = [] + remote_name = req_param.get( "remote_name" ) + trans_list.append( req_param.get( "galaxy_name" ) ) + trans_list.append( req_param.get( "missing" ) ) + self.param_trans_dict[ remote_name ] = trans_list # Command line (template). Optional for tools that do not invoke a local program command = root.find("command") if command is not None and command.text is not None: diff -r 36a7ff82faf0 -r 415cc6dc8e35 lib/galaxy/util/__init__.py --- a/lib/galaxy/util/__init__.py Fri Oct 24 12:32:32 2008 -0400 +++ b/lib/galaxy/util/__init__.py Mon Oct 27 16:03:43 2008 -0400 @@ -143,7 +143,7 @@ # different parameters can be sanitized in different ways. NEVER_SANITIZE = ['file_data', 'url_paste', 'URL'] - def __init__( self, params, safe=True, sanitize=True, tool_type=None, param_trans_dict={} ): + def __init__( self, params, safe=True, sanitize=True, tool=None ): if safe: for key, value in params.items(): # Check to see if we should translate certain parameter names. For example, @@ -152,21 +152,27 @@ # param_trans_dict looks like { "GENOME" : [ "dbkey" "?" ] } new_key = key new_value = value - if tool_type == 'data_source': - if key in param_trans_dict: - new_key = param_trans_dict[ key ][0] + if tool and tool.tool_type == 'data_source': + if key in tool.param_trans_dict: + new_key = tool.param_trans_dict[ key ][0] if not value: - new_value = param_trans_dict[ key ][1] + new_value = tool.param_trans_dict[ key ][1] if key not in self.NEVER_SANITIZE and sanitize: self.__dict__[ new_key ] = sanitize_param( new_value ) else: self.__dict__[ new_key ] = new_value - for key, value in param_trans_dict.items(): - # Make sure that all translated values used in Galaxy are added to the params - galaxy_name = param_trans_dict[ key ][0] - if galaxy_name not in self.__dict__: - # This will set the galaxy_name to the "missing" value - self.__dict__[ galaxy_name ] = param_trans_dict[ key ][1] + if tool and tool.tool_type == 'data_source': + # Add the tool's URL_method to params + self.__dict__[ 'URL_method' ] = tool.URL_method + # TODO: Biomart hack - eliminate when they encode URL - they'll let us know when... + if tool.add_to_URL is not None: + self.__dict__[ 'add_to_URL' ] = tool.add_to_URL + for key, value in tool.param_trans_dict.items(): + # Make sure that all translated values used in Galaxy are added to the params + galaxy_name = tool.param_trans_dict[ key ][0] + if galaxy_name not in self.__dict__: + # This will set the galaxy_name to the "missing" value + self.__dict__[ galaxy_name ] = tool.param_trans_dict[ key ][1] else: self.__dict__.update(params) diff -r 36a7ff82faf0 -r 415cc6dc8e35 lib/galaxy/web/controllers/tool_runner.py --- a/lib/galaxy/web/controllers/tool_runner.py Fri Oct 24 12:32:32 2008 -0400 +++ b/lib/galaxy/web/controllers/tool_runner.py Mon Oct 27 16:03:43 2008 -0400 @@ -39,11 +39,7 @@ log.error( "index called with tool id '%s' but no such tool exists", tool_id ) trans.log_event( "Tool id '%s' does not exist" % tool_id ) return "Tool '%s' does not exist, kwd=%s " % (tool_id, kwd) - try: - param_trans_dict = tool.param_trans_dict - except: - param_trans_dict = {} - params = util.Params( kwd, sanitize=tool.options.sanitize, tool_type=tool.tool_type, param_trans_dict=param_trans_dict ) + params = util.Params( kwd, sanitize=tool.options.sanitize, tool=tool ) history = trans.get_history() trans.ensure_valid_galaxy_session() template, vars = tool.handle_input( trans, params.__dict__ ) diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_destination/epigraph.xml --- a/tools/data_destination/epigraph.xml Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_destination/epigraph.xml Mon Oct 27 16:03:43 2008 -0400 @@ -1,21 +1,40 @@ <?xml version="1.0"?> -<tool name="Perform genome" id="epigraph_export"> - <description> analysis and prediction with EpiGRAPH</description> - <redirect_url_params>GENOME=${input1.dbkey} NAME=${input1.name} INFO=${input1.info}</redirect_url_params> - <inputs> - <param format="bed" name="input1" type="data" label="Send this dataset to EpiGRAPH"> - <validator type="unspecified_build" /> - </param> - <param name="REDIRECT_URL" type="hidden" value="http://epigraph.mpi-inf.mpg.de/WebGRAPH_Public_Test/faces/DataImport.jsp" /> - <param name="DATA_URL" type="baseurl" value="/datasets" /> - <param name="GALAXY_URL" type="baseurl" value="/tool_runner?tool_id=epigraph_import" /> - </inputs> - <outputs/> - <help> +<tool name="Perform genome analysis" id="epigraph_export"> + <description> and prediction with EpiGRAPH</description> + <redirect_url_params>GENOME=${input1.dbkey} NAME=${input1.name} INFO=${input1.info}</redirect_url_params> + <inputs> + <param format="bed" name="input1" type="data" label="Send this dataset to EpiGRAPH"> + <validator type="unspecified_build" /> + </param> + <param name="REDIRECT_URL" type="hidden" value="http://epigraph.mpi-inf.mpg.de/WebGRAPH_Public_Test/faces/DataImport.jsp" /> + <param name="DATA_URL" type="baseurl" value="/datasets" /> + <param name="GALAXY_URL" type="baseurl" value="/tool_runner?tool_id=epigraph_import" /> + </inputs> + <outputs/> + <help> + +.. class:: warningmark + +After clicking the **Execute** button, you will be redirected to the EpiGRAPH website. Please be patient while the dataset is being imported. Inside EpiGRAPH, buttons are available to send the results of the EpiGRAPH analysis back to Galaxy. In addition, you can always abandon an EpiGRAPH session and return to Galaxy by directing your browser to your current Galaxy instance. + +----- + +.. class:: infomark + **What it does** -This tool sends the selected dataset to EpiGRAPH for in-depth analysis and prediction. +This tool sends the selected dataset to EpiGRAPH in order to perform an in-depth analysis with statistical and machine learning methods. - </help> +----- + +.. class:: infomark + +**EpiGRAPH outline** + +The EpiGRAPH_ web service enables biologists to uncover hidden associations in vertebrate genome and epigenome datasets. Users can upload or import sets of genomic regions and EpiGRAPH will test a wide range of attributes (including DNA sequence and structure, gene density, chromatin modifications and evolutionary conservation) for enrichment or depletion among these regions. Furthermore, EpiGRAPH learns to predictively identify genomic regions that exhibit similar properties. + +.. _EpiGRAPH: http://epigraph.mpi-inf.mpg.de/ + + </help> </tool> diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_source/biomart.xml --- a/tools/data_source/biomart.xml Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_source/biomart.xml Mon Oct 27 16:03:43 2008 -0400 @@ -1,5 +1,13 @@ <?xml version="1.0"?> -<tool name="BioMart" id="biomart" tool_type="data_source"> + +<tool name="BioMart" id="biomart" tool_type="data_source" URL_method="get" add_to_URL="biomart_hack"> <description>Central server</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://www.biomart.org/biomart/martview" check_values="false" method="get" target="_top"> diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_source/biomart_test.xml --- a/tools/data_source/biomart_test.xml Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_source/biomart_test.xml Mon Oct 27 16:03:43 2008 -0400 @@ -1,5 +1,13 @@ <?xml version="1.0"?> -<tool name="BioMart" id="biomart" tool_type="data_source"> + +<tool name="BioMart" id="biomart_test" tool_type="data_source" URL_method="get" add_to_URL="biomart_hack"> <description>Test server</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://test.biomart.org/biomart/martview" check_values="false" method="get" target="_top"> diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_source/data_source.py --- a/tools/data_source/data_source.py Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_source/data_source.py Mon Oct 27 16:03:43 2008 -0400 @@ -33,10 +33,20 @@ if not URL: open( filename, 'w' ).write( "" ) stop_err( 'The remote data source application has not sent back a URL parameter in the request.' ) + # TODO: Hack to get biomart to work - this can be eliminated when the Biomart team encodes URL prior to sending, meanwhile + # everything including and beyond the first '&' is truncated from URL. They said they'll let us know when this is fixed + # at their end. + add_to_URL = params.get( 'add_to_URL', None ) + if add_to_URL: + URL += '&_export=1&GALAXY_URL=0' + URL_method = params.get( 'URL_method', None ) out = open( filename, 'w' ) CHUNK_SIZE = 2**20 # 1Mb try: - page = urllib.urlopen( URL, urllib.urlencode( params ) ) + if not URL_method or URL_method == 'get': + page = urllib.urlopen( URL ) + elif URL_method == 'post': + page = urllib.urlopen( URL, urllib.urlencode( params ) ) except: stop_err( 'It appears that the remote data source application is currently off line. Please try again later.' ) while 1: diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_source/epigraph_import.xml --- a/tools/data_source/epigraph_import.xml Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_source/epigraph_import.xml Mon Oct 27 16:03:43 2008 -0400 @@ -1,5 +1,10 @@ <?xml version="1.0"?> -<tool name="EpiGRAPH" id="epigraph_import" tool_type="data_source"> + +<tool name="EpiGRAPH" id="epigraph_import" tool_type="data_source" URL_method="get"> <description> server</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://epigraph.mpi-inf.mpg.de/WebGRAPH_Public_Test/faces/Login.jsp" check_values="false" method="get"> diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_source/flymine.xml --- a/tools/data_source/flymine.xml Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_source/flymine.xml Mon Oct 27 16:03:43 2008 -0400 @@ -1,5 +1,10 @@ <?xml version="1.0"?> -<tool name="Flymine" id="flymine" tool_type="data_source"> + +<tool name="Flymine" id="flymine" tool_type="data_source" URL_method="post"> <description>server</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://preview.flymine.org/preview/begin.do" check_values="false" method="get" target="_top"> diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_source/ucsc_tablebrowser.xml --- a/tools/data_source/ucsc_tablebrowser.xml Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_source/ucsc_tablebrowser.xml Mon Oct 27 16:03:43 2008 -0400 @@ -1,5 +1,10 @@ <?xml version="1.0"?> -<tool name="UCSC Main" id="ucsc_table_direct1" tool_type="data_source"> + +<tool name="UCSC Main" id="ucsc_table_direct1" tool_type="data_source" URL_method="post"> <description>table browser</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://genome.ucsc.edu/cgi-bin/hgTables" check_values="false" method="get"> diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_source/ucsc_tablebrowser_archaea.xml --- a/tools/data_source/ucsc_tablebrowser_archaea.xml Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_source/ucsc_tablebrowser_archaea.xml Mon Oct 27 16:03:43 2008 -0400 @@ -1,5 +1,10 @@ <?xml version="1.0"?> -<tool name="UCSC Archaea" id="ucsc_table_direct_archaea1" tool_type="data_source"> + +<tool name="UCSC Archaea" id="ucsc_table_direct_archaea1" tool_type="data_source" URL_method="post"> <description>table browser</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://archaea.ucsc.edu/cgi-bin/hgTables" check_values="false" method="get"> diff -r 36a7ff82faf0 -r 415cc6dc8e35 tools/data_source/ucsc_tablebrowser_test.xml --- a/tools/data_source/ucsc_tablebrowser_test.xml Fri Oct 24 12:32:32 2008 -0400 +++ b/tools/data_source/ucsc_tablebrowser_test.xml Mon Oct 27 16:03:43 2008 -0400 @@ -1,5 +1,10 @@ <?xml version="1.0"?> -<tool name="UCSC Test" id="ucsc_table_direct_test1" tool_type="data_source"> + +<tool name="UCSC Test" id="ucsc_table_direct_test1" tool_type="data_source" URL_method="post"> <description>table browser</description> <command interpreter="python">data_source.py $output</command> <inputs action="http://genome-test.cse.ucsc.edu/cgi-bin/hgTables" check_values="false" method="get">

1 0

[hg] galaxy 1574: WebHelpers imports simplejson. Import simplej...
by Nate Coraor 25 Oct '08

25 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/36a7ff82faf0 changeset: 1574:36a7ff82faf0 user: Nate Coraor <nate(a)bx.psu.edu> date: Fri Oct 24 12:32:32 2008 -0400 description: WebHelpers imports simplejson. Import simplejson before importing WebHelpers so we can ensure we get our version. 4 file(s) affected in this change: lib/galaxy/eggs/__init__.py lib/galaxy/model/custom_types.py lib/galaxy/web/controllers/workflow.py lib/galaxy/web/framework/__init__.py diffs (67 lines): diff -r c278ca8f9b73 -r 36a7ff82faf0 lib/galaxy/eggs/__init__.py --- a/lib/galaxy/eggs/__init__.py Fri Oct 24 11:41:23 2008 -0400 +++ b/lib/galaxy/eggs/__init__.py Fri Oct 24 12:32:32 2008 -0400 @@ -528,8 +528,6 @@ else: pkg_resources.working_set.require( "%s==%s" % ( name, egg.get_vertag() ) ) return - except UserWarning, e: - sys.exit( 1 ) except pkg_resources.VersionConflict, e: # there's a conflicting egg on the pythonpath, remove it dist = pkg_resources.get_distribution( name ) diff -r c278ca8f9b73 -r 36a7ff82faf0 lib/galaxy/model/custom_types.py --- a/lib/galaxy/model/custom_types.py Fri Oct 24 11:41:23 2008 -0400 +++ b/lib/galaxy/model/custom_types.py Fri Oct 24 12:32:32 2008 -0400 @@ -1,4 +1,6 @@ from sqlalchemy.types import * +import pkg_resources +pkg_resources.require("simplejson") import simplejson import pickle from galaxy.util.bunch import Bunch diff -r c278ca8f9b73 -r 36a7ff82faf0 lib/galaxy/web/controllers/workflow.py --- a/lib/galaxy/web/controllers/workflow.py Fri Oct 24 11:41:23 2008 -0400 +++ b/lib/galaxy/web/controllers/workflow.py Fri Oct 24 12:32:32 2008 -0400 @@ -1,5 +1,7 @@ from galaxy.web.base.controller import * +import pkg_resources +pkg_resources.require( "simplejson" ) import simplejson from galaxy.tools.parameters import * @@ -736,4 +738,4 @@ prefix = "%s|" % ( key ) cleanup( prefix, input.cases[current_case].inputs, group_values ) cleanup( "", inputs, values ) - return associations \ No newline at end of file + return associations diff -r c278ca8f9b73 -r 36a7ff82faf0 lib/galaxy/web/framework/__init__.py --- a/lib/galaxy/web/framework/__init__.py Fri Oct 24 11:41:23 2008 -0400 +++ b/lib/galaxy/web/framework/__init__.py Fri Oct 24 12:32:32 2008 -0400 @@ -11,18 +11,18 @@ import pickle from galaxy import util +pkg_resources.require( "simplejson" ) +import simplejson + pkg_resources.require( "WebHelpers" ) +import webhelpers + pkg_resources.require( "PasteDeploy" ) - -import webhelpers from paste.deploy.converters import asbool pkg_resources.require( "Mako" ) import mako.template import mako.lookup - -pkg_resources.require( "simplejson" ) -import simplejson pkg_resources.require( "sqlalchemy>=0.3" ) from sqlalchemy import desc

1 0

[hg] galaxy 1573: Add some missing 'url_for's.
by Nate Coraor 25 Oct '08

25 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/c278ca8f9b73 changeset: 1573:c278ca8f9b73 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Fri Oct 24 11:41:23 2008 -0400 description: Add some missing 'url_for's. 4 file(s) affected in this change: lib/galaxy/web/controllers/root.py lib/galaxy/web/controllers/tool_runner.py templates/tool_form.tmpl templates/workflow/editor.mako diffs (80 lines): diff -r f2ce31c74346 -r c278ca8f9b73 lib/galaxy/web/controllers/root.py --- a/lib/galaxy/web/controllers/root.py Thu Oct 23 15:19:48 2008 -0400 +++ b/lib/galaxy/web/controllers/root.py Fri Oct 24 11:41:23 2008 -0400 @@ -114,7 +114,7 @@ ## ---- Dataset display / editing ---------------------------------------- @web.expose - def display(self, trans, id=None, hid=None, tofile=None, toext=".txt"): + def display( self, trans, id=None, hid=None, tofile=None, toext=".txt", **kwd ): """ Returns data directly into the browser. Sets the mime-type according to the extension diff -r f2ce31c74346 -r c278ca8f9b73 lib/galaxy/web/controllers/tool_runner.py --- a/lib/galaxy/web/controllers/tool_runner.py Thu Oct 23 15:19:48 2008 -0400 +++ b/lib/galaxy/web/controllers/tool_runner.py Fri Oct 24 11:41:23 2008 -0400 @@ -30,7 +30,7 @@ def index(self, trans, tool_id=None, from_noframe=None, **kwd): # No tool id passed, redirect to main page if tool_id is None: - return trans.response.send_redirect( "/static/welcome.html" ) + return trans.response.send_redirect( url_for( "/static/welcome.html" ) ) # Load the tool toolbox = self.get_toolbox() tool = toolbox.tools_by_id.get( tool_id, None ) diff -r f2ce31c74346 -r c278ca8f9b73 templates/tool_form.tmpl --- a/templates/tool_form.tmpl Thu Oct 23 15:19:48 2008 -0400 +++ b/templates/tool_form.tmpl Fri Oct 24 11:41:23 2008 -0400 @@ -9,7 +9,7 @@ <title>Galaxy</title> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> <link href="$h.url_for('/static/style/base.css')" rel="stylesheet" type="text/css" /> -<script type='text/javascript' src="/static/scripts/jquery.js"> </script> +<script type='text/javascript' src="$h.url_for('/static/scripts/jquery.js')"> </script> <script type="text/javascript"> jQuery( function() { jQuery( "select[@refresh_on_change='true']").change( function() { diff -r f2ce31c74346 -r c278ca8f9b73 templates/workflow/editor.mako --- a/templates/workflow/editor.mako Thu Oct 23 15:19:48 2008 -0400 +++ b/templates/workflow/editor.mako Fri Oct 24 11:41:23 2008 -0400 @@ -3,7 +3,7 @@ <%def name="title()">Galaxy Workflow Editor</%def> <%def name="late_javascripts()"> - <script type='text/javascript' src="/static/scripts/galaxy.panels.js"> </script> + <script type='text/javascript' src="${h.url_for('/static/scripts/galaxy.panels.js')}"> </script> <script type="text/javascript"> ensure_dd_helper(); make_left_panel( $("#left"), $("#center"), $("#left-border" ) ); @@ -17,16 +17,16 @@ ${parent.javascripts()}  - <script type='text/javascript' src="/static/scripts/jquery.js"> </script> - <script type='text/javascript' src="/static/scripts/jquery.ui.js"> </script> - <script type='text/javascript' src="/static/scripts/galaxy.ui.scrollPanel.js"> </script> - <script type='text/javascript' src="/static/scripts/jquery.hoverIntent.js"> </script> - <script type='text/javascript' src="/static/scripts/jquery.form.js"> </script> - <script type='text/javascript' src="/static/scripts/jquery.json.js"> </script> + <script type='text/javascript' src="${h.url_for('/static/scripts/jquery.js')}"> </script> + <script type='text/javascript' src="${h.url_for('/static/scripts/jquery.ui.js')}"> </script> + <script type='text/javascript' src="${h.url_for('/static/scripts/galaxy.ui.scrollPanel.js')}"> </script> + <script type='text/javascript' src="${h.url_for('/static/scripts/jquery.hoverIntent.js')}"> </script> + <script type='text/javascript' src="${h.url_for('/static/scripts/jquery.form.js')}"> </script> + <script type='text/javascript' src="${h.url_for('/static/scripts/jquery.json.js')}"> </script> - <script type='text/javascript' src="/static/scripts/galaxy.workflow_editor.canvas.js"> </script> + <script type='text/javascript' src="${h.url_for('/static/scripts/galaxy.workflow_editor.canvas.js')}"> </script> <!--[if lt IE 7]> <script type='text/javascript'> @@ -587,4 +587,4 @@ <div class="unified-panel-body" style="overflow: auto;"> <div id="right-content"></div> </div> -</%def> \ No newline at end of file +</%def>

1 0

[hg] galaxy 1572: Fix a bug from the last commit, what I changed...
by Nate Coraor 24 Oct '08

24 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/f2ce31c74346 changeset: 1572:f2ce31c74346 user: Nate Coraor <nate(a)bx.psu.edu> date: Thu Oct 23 15:19:48 2008 -0400 description: Fix a bug from the last commit, what I changed doesn't work right. 2 file(s) affected in this change: lib/galaxy/datatypes/interval.py templates/root/history_common.mako diffs (50 lines): diff -r 6e592c876f4d -r f2ce31c74346 lib/galaxy/datatypes/interval.py --- a/lib/galaxy/datatypes/interval.py Thu Oct 23 15:02:23 2008 -0400 +++ b/lib/galaxy/datatypes/interval.py Thu Oct 23 15:19:48 2008 -0400 @@ -9,6 +9,7 @@ import data from galaxy import util from galaxy.datatypes.sniff import * +from galaxy.web import url_for from cgi import escape import urllib from bx.intervals.io import * @@ -217,7 +218,7 @@ stop = viewport_tuple[2] for site_name, site_url in util.get_ucsc_by_build(dataset.dbkey): if site_name in app.config.ucsc_display_sites: - display_url = urllib.quote_plus( "%s/display_as?id=%i&display_app=%s" % (base_url, dataset.id, type) ) + display_url = urllib.quote_plus( "%s%s/display_as?id=%i&display_app=%s" % (base_url, url_for( controller='root' ), dataset.id, type) ) link = "%sdb=%s&position=%s:%s-%s&hgt.customText=%s" % (site_url, dataset.dbkey, chrom, start, stop, display_url ) ret_val.append( (site_name, link) ) return ret_val @@ -810,7 +811,7 @@ stop = viewport_tuple[2] for site_name, site_url in util.get_ucsc_by_build(dataset.dbkey): if site_name in app.config.ucsc_display_sites: - display_url = urllib.quote_plus( "%s/display_as?id=%i&display_app=%s" % (base_url, dataset.id, type) ) + display_url = urllib.quote_plus( "%s%s/display_as?id=%i&display_app=%s" % (base_url, url_for( controller='root' ), dataset.id, type) ) link = "%sdb=%s&position=%s:%s-%s&hgt.customText=%s" % (site_url, dataset.dbkey, chrom, start, stop, display_url ) ret_val.append( (site_name, link) ) return ret_val @@ -889,7 +890,7 @@ stop = viewport_tuple[2] for site_name, site_url in util.get_gbrowse_sites_by_build(dataset.dbkey): if site_name in app.config.gbrowse_display_sites: - display_url = urllib.quote_plus( "%s/display_as?id=%i&display_app=%s" % (base_url, dataset.id, type) ) + display_url = urllib.quote_plus( "%s%s/display_as?id=%i&display_app=%s" % (base_url, url_for( controller='root' ), dataset.id, type) ) link = "%sname=%s&ref=%s:%s..%s&eurl=%s" % (site_url, dataset.dbkey, chrom, start, stop, display_url ) ret_val.append( (site_name, link) ) return ret_val diff -r 6e592c876f4d -r f2ce31c74346 templates/root/history_common.mako --- a/templates/root/history_common.mako Thu Oct 23 15:02:23 2008 -0400 +++ b/templates/root/history_common.mako Thu Oct 23 15:19:48 2008 -0400 @@ -67,7 +67,7 @@ %if data.has_data: <a href="${h.url_for( action='display', id=data.id, tofile='yes', toext=data.ext )}" target="_blank">save</a> %for display_app in data.datatype.get_display_types(): - <% display_links = data.datatype.get_display_links( data, display_app, app, request.base + h.url_for() ) %> + <% display_links = data.datatype.get_display_links( data, display_app, app, request.base ) %> %if len( display_links ) > 0: | ${data.datatype.get_display_label(display_app)} %for display_name, display_link in display_links:

1 0

[hg] galaxy 1571: When creating display links, use url_for to ad...
by Nate Coraor 24 Oct '08

24 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/6e592c876f4d changeset: 1571:6e592c876f4d user: Nate Coraor <nate(a)bx.psu.edu> date: Thu Oct 23 15:02:23 2008 -0400 description: When creating display links, use url_for to add the proxy prefix, if any. 1 file(s) affected in this change: templates/root/history_common.mako diffs (12 lines): diff -r 3e391f7192f6 -r 6e592c876f4d templates/root/history_common.mako --- a/templates/root/history_common.mako Wed Oct 22 14:37:28 2008 -0400 +++ b/templates/root/history_common.mako Thu Oct 23 15:02:23 2008 -0400 @@ -67,7 +67,7 @@ %if data.has_data: <a href="${h.url_for( action='display', id=data.id, tofile='yes', toext=data.ext )}" target="_blank">save</a> %for display_app in data.datatype.get_display_types(): - <% display_links = data.datatype.get_display_links( data, display_app, app, request.base ) %> + <% display_links = data.datatype.get_display_links( data, display_app, app, request.base + h.url_for() ) %> %if len( display_links ) > 0: | ${data.datatype.get_display_label(display_app)} %for display_name, display_link in display_links:

1 0

[hg] galaxy 1567: Like the local runner, if job preparation fail...
by Nate Coraor 24 Oct '08

24 Oct '08

details: http://www.bx.psu.edu/hg/galaxy/rev/b02b8d9196a8 changeset: 1567:b02b8d9196a8 user: Nate Coraor <nate(a)bx.psu.edu> date: Wed Oct 22 12:15:06 2008 -0400 description: Like the local runner, if job preparation fails, catch and log. 2 file(s) affected in this change: lib/galaxy/jobs/runners/pbs.py lib/galaxy/jobs/runners/sge.py diffs (42 lines): diff -r 510ff36caaf3 -r b02b8d9196a8 lib/galaxy/jobs/runners/pbs.py --- a/lib/galaxy/jobs/runners/pbs.py Tue Oct 21 10:40:08 2008 -0400 +++ b/lib/galaxy/jobs/runners/pbs.py Wed Oct 22 12:15:06 2008 -0400 @@ -114,8 +114,15 @@ def queue_job( self, job_wrapper ): """Create PBS script for a job and submit it to the PBS queue""" - job_wrapper.prepare() - command_line = job_wrapper.get_command_line() + + try: + job_wrapper.prepare() + command_line = job_wrapper.get_command_line() + except: + job_wrapper.fail( "failure preparing job", exception=True ) + log.exception("failure running job %d" % job_wrapper.job_id) + return + runner_url = job_wrapper.tool.job_runner # This is silly, why would we queue a job with no command line? diff -r 510ff36caaf3 -r b02b8d9196a8 lib/galaxy/jobs/runners/sge.py --- a/lib/galaxy/jobs/runners/sge.py Tue Oct 21 10:40:08 2008 -0400 +++ b/lib/galaxy/jobs/runners/sge.py Wed Oct 22 12:15:06 2008 -0400 @@ -103,8 +103,15 @@ def queue_job( self, job_wrapper ): """Create SGE script for a job and submit it to the SGE queue""" - job_wrapper.prepare() - command_line = job_wrapper.get_command_line() + + try: + job_wrapper.prepare() + command_line = job_wrapper.get_command_line() + except: + job_wrapper.fail( "failure preparing job", exception=True ) + log.exception("failure running job %d" % job_wrapper.job_id) + return + runner_url = job_wrapper.tool.job_runner # This is silly, why would we queue a job with no command line?

1 0