December 2008 - galaxy-dev - lists.galaxyproject.org

[hg] galaxy 1638: Update GMAJ to latest beta.
by Nate Coraor 04 Dec '08

04 Dec '08

details: http://www.bx.psu.edu/hg/galaxy/rev/66ea0a1ac929 changeset: 1638:66ea0a1ac929 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Mon Nov 24 11:19:48 2008 -0500 description: Update GMAJ to latest beta. 2 file(s) affected in this change: static/gmaj/gmaj.jar tools/visualization/GMAJ.xml diffs (30 lines): diff -r c141cb89aa45 -r 66ea0a1ac929 static/gmaj/gmaj.jar Binary file static/gmaj/gmaj.jar has changed diff -r c141cb89aa45 -r 66ea0a1ac929 tools/visualization/GMAJ.xml --- a/tools/visualization/GMAJ.xml Thu Nov 20 16:15:39 2008 -0500 +++ b/tools/visualization/GMAJ.xml Mon Nov 24 11:19:48 2008 -0500 @@ -46,6 +46,7 @@ <option name="MAF File" value="maf"> <option name="Invalid MAF version (maf_version)" value="maf_version"/> <option name="Skipping unsupported paragraph (maf_paragraph)" value="maf_paragraph"/> + <option name="Unrecognized character found in alignment (bad_char_all)" value="bad_char_all"/> <option name="Skipping all reconstruction scores: no species specified (recon_noseq)" value="recon_noseq"/> <option name="Skipping reconstruction scores in blocks with missing row (recon_missing)" value="recon_missing"/> <option name="The first row in some blocks is not the specified reference sequence (refseq_not_first)" value="refseq_not_first"/> @@ -75,7 +76,7 @@ <option name="Score shading is not yet supported (score_shading)" value="score_shading"/> </option> <option name="Red Flags" value="red"> - <option name="Sequence name in annotation file does not match name in MAF (seqname_mismatch)" value="seqname_mismatch"/> + <option name="Assuming that annotations in file ___ are for species ___ (seqname_fix_all)" value="seqname_fix_all"/> <option name="BED start or end < 0 (bed_coord)" value="bed_coord"/> <option name="GFF start or end < 1 (gff_coord)" value="gff_coord"/> <option name="Missing item name for URL substitution (url_subst)" value="url_subst"/> @@ -87,6 +88,7 @@ <option name="Skipping blocks for export where reference sequence is hidden or all gaps (export_skip)" value="export_skip"/> <option name="Possible parse error: token ends with an escaped quote (escaped_quote)" value="escaped_quote"/> <option name="Draggable panel dividers will not be sticky (no_sticky)" value="no_sticky"/> + <option name="Selecting a large block may be very slow (big_block)" value="big_block"/> </option> </option> </options>

1 0

[hg] galaxy 1639: Fix for setting metadata on a history item des...
by Nate Coraor 04 Dec '08

04 Dec '08

details: http://www.bx.psu.edu/hg/galaxy/rev/bcd158679985 changeset: 1639:bcd158679985 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Mon Nov 24 11:46:45 2008 -0500 description: Fix for setting metadata on a history item designated as a MAF file, but is not a valid MAF. 1 file(s) affected in this change: lib/galaxy/datatypes/sequence.py diffs (14 lines): diff -r 66ea0a1ac929 -r bcd158679985 lib/galaxy/datatypes/sequence.py --- a/lib/galaxy/datatypes/sequence.py Mon Nov 24 11:19:48 2008 -0500 +++ b/lib/galaxy/datatypes/sequence.py Mon Nov 24 11:46:45 2008 -0500 @@ -208,9 +208,9 @@ species = [] species_chromosomes = {} - maf_reader = bx.align.maf.Reader( open( dataset.file_name ) ) indexes = bx.interval_index_file.Indexes() try: + maf_reader = bx.align.maf.Reader( open( dataset.file_name ) ) while True: pos = maf_reader.file.tell() block = maf_reader.next()

1 0

[hg] galaxy 1641: Improve template for error reporting from users.
by Nate Coraor 04 Dec '08

04 Dec '08

details: http://www.bx.psu.edu/hg/galaxy/rev/e2e7c33f587d changeset: 1641:e2e7c33f587d user: Greg Von Kuster <greg(a)bx.psu.edu> date: Mon Dec 01 09:57:59 2008 -0500 description: Improve template for error reporting from users. 1 file(s) affected in this change: lib/galaxy/web/controllers/dataset.py diffs (63 lines): diff -r 20683fe3bfcb -r e2e7c33f587d lib/galaxy/web/controllers/dataset.py --- a/lib/galaxy/web/controllers/dataset.py Sun Nov 30 08:03:21 2008 -0500 +++ b/lib/galaxy/web/controllers/dataset.py Mon Dec 01 09:57:59 2008 -0500 @@ -21,25 +21,23 @@ GALAXY TOOL ERROR REPORT ------------------------ -This error report is in reference to dataset ${dataset_id}. The user -(${email}) provided the following information: - +This error report is in reference to output dataset ${dataset_id}. ----------------------------------------------------------------------------- +The user '${email}' provided the following information: ${message} ----------------------------------------------------------------------------- - -The tool error was: - +job id: ${job_id} +tool id: ${tool_id} ----------------------------------------------------------------------------- +job stderr: ${stderr} ----------------------------------------------------------------------------- - -And the tool output was: - ------------------------------------------------------------------------------ +job stdout: ${stdout} ----------------------------------------------------------------------------- - +job info: +${info} +----------------------------------------------------------------------------- (This is an automated message). """ @@ -62,7 +60,8 @@ smtp_server = trans.app.config.smtp_server if smtp_server is None: return trans.show_error_message( "Sorry, mail is not configured for this galaxy instance" ) - to_address = trans.app.config.error_email_to + #to_address = trans.app.config.error_email_to + to_address = 'greg(a)bx.psu.edu' if to_address is None: return trans.show_error_message( "Sorry, error reporting has been disabled for this galaxy instance" ) # Get the dataset and associated job @@ -72,9 +71,12 @@ msg = MIMEText( string.Template( error_report_template ) .safe_substitute( dataset_id=dataset.id, email=email, - message=message, + message=message, + job_id=job.id, + tool_id=job.tool_id, stderr=job.stderr, - stdout=job.stdout ) ) + stdout=job.stdout, + info=job.info ) ) frm = to_address # Check email a bit email = email.strip()

1 0

[hg] galaxy 1642: Refix cleanup_dataset.py script.
by Nate Coraor 04 Dec '08

04 Dec '08

details: http://www.bx.psu.edu/hg/galaxy/rev/499eed864ab3 changeset: 1642:499eed864ab3 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Mon Dec 01 14:07:36 2008 -0500 description: Refix cleanup_dataset.py script. 1 file(s) affected in this change: scripts/cleanup_datasets/cleanup_datasets.py diffs (16 lines): diff -r e2e7c33f587d -r 499eed864ab3 scripts/cleanup_datasets/cleanup_datasets.py --- a/scripts/cleanup_datasets/cleanup_datasets.py Mon Dec 01 09:57:59 2008 -0500 +++ b/scripts/cleanup_datasets/cleanup_datasets.py Mon Dec 01 14:07:36 2008 -0500 @@ -8,6 +8,12 @@ new_path = [ os.path.join( os.getcwd(), "lib" ) ] new_path.extend( sys.path[1:] ) # remove scripts/ from the path sys.path = new_path + +from galaxy import eggs +import galaxy.model.mapping +import pkg_resources + +pkg_resources.require( "SQLAlchemy >= 0.4" ) from galaxy.model.orm import *

1 0

[hg] galaxy 1640: Fixes, code cleanup in cleanup_datasets.py.
by Nate Coraor 04 Dec '08

04 Dec '08

details: http://www.bx.psu.edu/hg/galaxy/rev/20683fe3bfcb changeset: 1640:20683fe3bfcb user: Greg Von Kuster <greg(a)bx.psu.edu> date: Sun Nov 30 08:03:21 2008 -0500 description: Fixes, code cleanup in cleanup_datasets.py. 1 file(s) affected in this change: scripts/cleanup_datasets/cleanup_datasets.py diffs (141 lines): diff -r bcd158679985 -r 20683fe3bfcb scripts/cleanup_datasets/cleanup_datasets.py --- a/scripts/cleanup_datasets/cleanup_datasets.py Mon Nov 24 11:46:45 2008 -0500 +++ b/scripts/cleanup_datasets/cleanup_datasets.py Sun Nov 30 08:03:21 2008 -0500 @@ -9,12 +9,7 @@ new_path.extend( sys.path[1:] ) # remove scripts/ from the path sys.path = new_path -from galaxy import eggs -import galaxy.model.mapping -import pkg_resources - -pkg_resources.require( "SQLAlchemy >= 0.4" ) -from sqlalchemy.orm import eagerload +from galaxy.model.orm import * assert sys.version_info[:2] >= ( 2, 4 ) @@ -79,8 +74,10 @@ # Provide info about the histories and datasets that will be affected if the delete_userless_histories function is executed. history_count = 0 dataset_count = 0 - where = ( h.table.c.user_id==None ) & ( h.table.c.deleted==False ) & ( h.table.c.update_time < cutoff_time ) - histories = h.query().filter( where ).options( eagerload( 'active_datasets' ) ).all() + histories = h.filter( and_( h.table.c.user_id==None, + h.table.c.deleted==False, + h.table.c.update_time < cutoff_time ) ) \ + .options( eagerload( 'active_datasets' ) ).all() print '# The following datasets and associated userless histories will be deleted' start = time.clock() @@ -102,17 +99,18 @@ # The datasets associated with each history are also deleted. Nothing is removed from disk. history_count = 0 dataset_count = 0 - h_where = ( h.table.c.user_id==None ) & ( h.table.c.deleted==False ) & ( h.table.c.update_time < cutoff_time ) print '# The following datasets and associated userless histories have been deleted' start = time.clock() - histories = h.query().filter( h_where ).options( eagerload( 'active_datasets' ) ).all() + histories = h.filter( and_( h.table.c.user_id==None, + h.table.c.deleted==False, + h.table.c.update_time < cutoff_time ) ) \ + .options( eagerload( 'active_datasets' ) ).all() for history in histories: for dataset_assoc in history.active_datasets: if not dataset_assoc.deleted: # Mark all datasets as deleted - d_where = ( d.table.c.id==dataset_assoc.dataset_id ) - datasets = d.query().filter( d_where ).all() + datasets = d.filter( d.table.c.id==dataset_assoc.dataset_id ).all() for dataset in datasets: if not dataset.deleted: dataset.deleted = True @@ -136,17 +134,17 @@ history_count = 0 dataset_count = 0 disk_space = 0 - h_where = ( h.table.c.deleted==True ) & ( h.table.c.purged==False ) & ( h.table.c.update_time < cutoff_time ) - print '# The following datasets and associated deleted histories will be purged' start = time.clock() - histories = h.query().filter( h_where ).options( eagerload( 'datasets' ) ).all() + histories = h.filter( and_( h.table.c.deleted==True, + h.table.c.purged==False, + h.table.c.update_time < cutoff_time ) ) \ + .options( eagerload( 'datasets' ) ).all() for history in histories: for dataset_assoc in history.datasets: # Datasets can only be purged if their HistoryDatasetAssociation has been deleted. if dataset_assoc.deleted: - d_where = ( d.table.c.id==dataset_assoc.dataset_id ) - datasets = d.query().filter( d_where ).all() + datasets = d.filter( d.table.c.id==dataset_assoc.dataset_id ).all() for dataset in datasets: if dataset.purgable and not dataset.purged: print "%s" % dataset.file_name @@ -169,17 +167,17 @@ disk_space = 0 file_size = 0 errors = False - h_where = ( h.table.c.deleted==True ) & ( h.table.c.purged==False ) & ( h.table.c.update_time < cutoff_time ) - print '# The following datasets and associated deleted histories have been purged' start = time.clock() - histories = h.query().filter( h_where ).options( eagerload( 'datasets' ) ).all() + histories = h.filter( and_( h.table.c.deleted==True, + h.table.c.purged==False, + h.table.c.update_time < cutoff_time ) ) \ + .options( eagerload( 'datasets' ) ).all() for history in histories: errors = False for dataset_assoc in history.datasets: if dataset_assoc.deleted: - d_where = ( d.table.c.id==dataset_assoc.dataset_id ) - datasets = d.query().filter( d_where ).all() + datasets = d.filter( d.table.c.id==dataset_assoc.dataset_id ).all() for dataset in datasets: if dataset.purgable and not dataset.purged: file_size = dataset.file_size @@ -221,11 +219,12 @@ # Provide info about the datasets that will be affected if the purge_datasets function is executed. dataset_count = 0 disk_space = 0 - where = ( d.table.c.deleted==True ) & ( d.table.c.purgable==True ) & ( d.table.c.purged==False ) & ( d.table.c.update_time < cutoff_time ) - print '# The following deleted datasets will be purged' start = time.clock() - datasets = d.query().filter( where ).all() + datasets = d.filter( and_( d.table.c.deleted==True, + d.table.c.purgable==True, + d.table.c.purged==False, + d.table.c.update_time < cutoff_time ) ).all() for dataset in datasets: print "%s" % dataset.file_name dataset_count += 1 @@ -243,11 +242,12 @@ dataset_count = 0 disk_space = 0 file_size = 0 - where = ( d.table.c.deleted==True ) & ( d.table.c.purgable==True ) & ( d.table.c.purged==False ) & ( d.table.c.update_time < cutoff_time ) - print '# The following deleted datasets have been purged' start = time.clock() - datasets = d.query().filter( where ).all() + datasets = d.filter( and_( d.table.c.deleted==True, + d.table.c.purgable==True, + d.table.c.purged==False, + d.table.c.update_time < cutoff_time ) ).all() for dataset in datasets: file_size = dataset.file_size if remove_from_disk: @@ -289,7 +289,8 @@ # See if the dataset has been shared if dataset.external_filename: # This check handles the pre-history_dataset_association approach to sharing. - shared_data = d.filter( and_( d.table.c.external_filename==dataset.external_filename, d.table.c.deleted==False ) ).all() + shared_data = d.filter( and_( d.table.c.external_filename==dataset.external_filename, + d.table.c.deleted==False ) ).all() if shared_data: purgable = False if purgable:

1 0