February 2010 - galaxy-dev - lists.galaxyproject.org

Sqlite to MYSQL conversion in galaxy
by Roy Weckiewicz 19 Feb '10

19 Feb '10

---------- Forwarded message ---------- From: Roy Weckiewicz <roystefan(a)gmail.com> Date: Fri, Feb 19, 2010 at 1:31 PM Subject: Re: [galaxy-dev] Sqlite to MYSQL conversion in galaxy To: Nate Coraor <nate(a)bx.psu.edu> Thanks again Nate, I just tried the one you sent me - it seems I just have a database revision issue: Traceback (most recent call last): File "/Users/rsw3284/Downloads/galaxy-0c07212afc00/lib/galaxy/web/buildapp.py", line 61, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/Users/rsw3284/Downloads/galaxy-0c07212afc00/lib/galaxy/app.py", line 28, in __init__ create_or_verify_database( db_url, self.config.database_engine_options ) File "/Users/rsw3284/Downloads/galaxy-0c07212afc00/lib/galaxy/model/migrate/check.py", line 72, in create_or_verify_database raise Exception( "Your database is older than hg revision 1464:c7acaa1bb88f and will need to be updated manually" ) Exception: Your database is older than hg revision 1464:c7acaa1bb88f and will need to be updated manually Rgds, Roy Weckiewicz On Fri, Feb 19, 2010 at 1:28 PM, Nate Coraor <nate(a)bx.psu.edu> wrote: > Roy Weckiewicz wrote: > > What's strange is that eventhough the file name on the egg is >> MySQL*fat-ucs2.egg the error still reports the file in question as >> MySQL*fat.egg -- I've tried renaming the egg to match the original file, but >> it didn't seem to make a difference. Furthermore, when I run.sh, the program >> will create a directory in home called .python-eggs/ containing the >> _MySQL.so file, among other things - Will I have to add a different URL to >> the eggs.ini to keep it from re-downloading the old file? >> > > Ah, sorry, that egg relies on some uncommitted patches. I've rebuilt it > using a clean checkout, could you try this one instead?: > > > http://www.bx.psu.edu/~nate/galaxy/MySQL_python-1.2.2_5.0.67_static-py2.5-m… > > Thanks, > --nate > -- Roy Weckiewicz Texas A&M University -- Roy Weckiewicz Texas A&M University

1 0

Sqlite to MYSQL conversion in galaxy
by Roy Weckiewicz 19 Feb '10

19 Feb '10

Hello, I am trying to configure a local galaxy server using a MYSQL database instead of the standard sqlite3. I'm using OSX and have been attempting (unsuccessfully) to convert the sqlite database (universe.sql) to mysql format. Basically, I am in need of the database table schema for mysql. I noticed on your web page that you host a mysql server at: genome-mysql.cse.ucsc.edu -- would it be possible to do some sort of mysqldump or mysqlhotcopy in order to download the table schema, so that I could upload them into my local mysql installation? I've tried using Sqlite3 to dump the contents of universe.sql, but there are numerous syntax differences that need to be addressed before it will load successfully into my mysql database. Do you have any suggestions on how to either migrate the database from sqlite3 to MYSQL (via script or program), or create the mysql database from scratch and just import the correct table schema? Any help would be appreciated. MYSQL: Ver 14.14 Distrib 5.1.37, for apple-darwin9.5.0 (i386) using readline 5.1 Python 2.5.1 Rgds, Roy Weckiewicz. -- Roy Weckiewicz Texas A&M University

3 7

boolean testing problems
by Assaf Gordon 19 Feb '10

19 Feb '10

Hi, I really appreciate your help with this, and I'm sorry to keep bothering you about the boolean thing, but: It seems the recent successive changes to the boolean parameter are conflicting with one another (some changes made to the testing framework and others to the rest of galaxy). At changeset 3402:bb50ac713feb (before the whole boolean issue start), testing "gpos_concat_1" works OK - single tests the succeeds (and only tests the "true" case). At changeset 3410:4a416d872806 (which fixed the boolean in the testing framework), testing "gpos_concat_1" works OK - with two tests, checking both "true" and "false" cases. But changeset 3411:0c07212afc00 (which fixed the boolean bug not related to testing framework) broke the testing framework entirely: testing "gpos_concat_1" fails on both tests (together with all my tools which have a boolean parameter, even if those tests worked before). -gordon

2 2

[hg] galaxy 3413: lims: added feature to to transfer folders fro...
by Greg Von Kuster 19 Feb '10

19 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/bbff5050cfd2 changeset: 3413:bbff5050cfd2 user: rc date: Thu Feb 18 11:50:19 2010 -0500 description: lims: added feature to to transfer folders from a sequencer diffstat: lib/galaxy/web/controllers/requests_admin.py | 63 ++++++++++++++------------- 1 files changed, 32 insertions(+), 31 deletions(-) diffs (94 lines): diff -r e05cdf1205bc -r bbff5050cfd2 lib/galaxy/web/controllers/requests_admin.py --- a/lib/galaxy/web/controllers/requests_admin.py Thu Feb 18 11:26:39 2010 -0500 +++ b/lib/galaxy/web/controllers/requests_admin.py Thu Feb 18 11:50:19 2010 -0500 @@ -1146,7 +1146,7 @@ for index in range(len(sample.request.samples)): # check for empty bar code if not barcode.strip(): - msg = 'Please fill the barcode for sample <b>%s</b>.' % samples[index].name + msg = 'Please fill the barcode for sample <b>%s</b>.' % sample.name break # check all the saved bar codes all_samples = trans.sa_session.query( trans.app.model.Sample ) @@ -1401,9 +1401,19 @@ sample_id=trans.security.encode_id(sample.id), msg=msg, messagetype='error', folder_path=folder_path )) - files = output.split()[2:] - return files - + return output.split()[2:] + + def __get_files_in_dir(self, trans, sample, folder_path): + tmpfiles = self.__get_files(trans, sample, folder_path) + for tf in tmpfiles: + if tf[-1] == os.sep: + self.__get_files_in_dir(trans, sample, os.path.join(folder_path, tf)) + else: + sample.dataset_files.append([os.path.join(folder_path, tf), + sample.transfer_status.NOT_STARTED]) + trans.sa_session.add( sample ) + trans.sa_session.flush() + return @web.expose @web.require_admin def get_data(self, trans, **kwd): @@ -1460,19 +1470,24 @@ sample=sample, dataset_files=sample.dataset_files) elif params.get( 'start_transfer_button', False ): - for f in files_list: - if f[-1] == os.sep: - # the selected item is a folder so transfer all the - # folder contents - sample.dataset_files.append([os.path.join(folder_path, f), - sample.transfer_status.NOT_STARTED]) - else: - sample.dataset_files.append([os.path.join(folder_path, f), - sample.transfer_status.NOT_STARTED]) - trans.sa_session.add( sample ) - trans.sa_session.flush() - return self.__start_datatx(trans, sample) - + folder_files = [] + if len(files_list): + for f in files_list: + if f[-1] == os.sep: + # the selected item is a folder so transfer all the + # folder contents + self.__get_files_in_dir(trans, sample, os.path.join(folder_path, f)) + else: + sample.dataset_files.append([os.path.join(folder_path, f), + sample.transfer_status.NOT_STARTED]) + trans.sa_session.add( sample ) + trans.sa_session.flush() + return self.__start_datatx(trans, sample) + return trans.response.send_redirect( web.url_for( controller='requests_admin', + action='show_datatx_page', + sample_id=trans.security.encode_id(sample.id), + folder_path=folder_path)) + def __setup_datatx_user(self, trans, library, folder): ''' This method sets up the datatx user: @@ -1562,20 +1577,6 @@ dataset_index=index, cmd=cmd) dtt.start() -# # set the transfer status -# sample.dataset_files[index][1] = sample.transfer_status.IN_PROGRESS -# trans.sa_session.add( sample ) -# trans.sa_session.flush() -# try: -# retcode = subprocess.call(cmd) -# except Exception, e: -# error_msg = dfile.split('/')[-1] + ": Data transfer failed. " + str(e) + "<br/>" -# return trans.response.send_redirect( web.url_for( controller='requests_admin', -# action='show_datatx_page', -# sample_id=trans.security.encode_id(sample.id), -# folder_path=os.path.dirname(dfile), -# messagetype='error', -# msg=error_msg)) # set the sample state to the last state if sample.current_state().id != sample.request.type.states[-1].id: event = trans.app.model.SampleEvent(sample, sample.request.type.states[-1],

1 0

[hg] galaxy 3415: lims:
by Greg Von Kuster 19 Feb '10

19 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/52edc83ed81c changeset: 3415:52edc83ed81c user: rc date: Thu Feb 18 21:38:49 2010 -0500 description: lims: - added data directory field in sequencer info - UI changes on samples page diffstat: lib/galaxy/web/controllers/requests_admin.py | 9 +++++++-- templates/admin/requests/create_request_type.mako | 4 ++++ templates/admin/requests/show_request.mako | 12 ++++++++---- templates/admin/requests/view_request_type.mako | 4 ++++ templates/requests/show_request.mako | 11 +++++++---- 5 files changed, 30 insertions(+), 10 deletions(-) diffs (164 lines): diff -r c280939fd48d -r 52edc83ed81c lib/galaxy/web/controllers/requests_admin.py --- a/lib/galaxy/web/controllers/requests_admin.py Thu Feb 18 17:04:43 2010 -0500 +++ b/lib/galaxy/web/controllers/requests_admin.py Thu Feb 18 21:38:49 2010 -0500 @@ -1370,10 +1370,14 @@ status='error', message="Invalid sample ID", **kwd) ) + if sample.request.type.datatx_info.get('data_dir', ''): + folder_path = util.restore_text( sample.request.type.datatx_info.get('data_dir', '') ) + else: + folder_path = util.restore_text( params.get( 'folder_path', '' ) ) return trans.fill_template( '/admin/requests/get_data.mako', sample=sample, dataset_files=sample.dataset_files, msg=msg, messagetype=messagetype, files=[], - folder_path=util.restore_text( params.get( 'folder_path', '' ) )) + folder_path=folder_path ) def __get_files(self, trans, sample, folder_path): ''' This method retrieves the filenames to be transfer from the remote host. @@ -1687,7 +1691,8 @@ # data transfer info rt.datatx_info = dict(host=util.restore_text( params.get( 'host', '' ) ), username=util.restore_text( params.get( 'username', '' ) ), - password=util.restore_text( params.get( 'password', '' ) )) + password=util.restore_text( params.get( 'password', '' ) ), + data_dir=util.restore_text( params.get( 'data_dir', '' ) )) trans.sa_session.add( rt ) trans.sa_session.flush() return trans.response.send_redirect( web.url_for( controller='requests_admin', diff -r c280939fd48d -r 52edc83ed81c templates/admin/requests/create_request_type.mako --- a/templates/admin/requests/create_request_type.mako Thu Feb 18 17:04:43 2010 -0500 +++ b/templates/admin/requests/create_request_type.mako Thu Feb 18 21:38:49 2010 -0500 @@ -66,6 +66,10 @@ <input type="password" name="password" value="" size="40"/> </div> <div class="form-row"> + <label>Data directory:</label> + <input type="text" name="data_dir" value="" size="40"/> + </div> + <div class="form-row"> <div style="float: left; width: 250px; margin-right: 10px;"> <input type="hidden" name="new" value="submitted" size="40"/> </div> diff -r c280939fd48d -r 52edc83ed81c templates/admin/requests/show_request.mako --- a/templates/admin/requests/show_request.mako Thu Feb 18 17:04:43 2010 -0500 +++ b/templates/admin/requests/show_request.mako Thu Feb 18 21:38:49 2010 -0500 @@ -43,7 +43,7 @@ </script> <style type="text/css"> .msg_head { - padding: 10px 0px; + padding: 0px 0px; cursor: pointer; } @@ -134,7 +134,7 @@ <th>Data Library</th> <th>Folder</th> %if request.submitted() or request.complete(): - <th>Dataset(s)</th> + <th>Dataset(s) Transfered</th> %endif <th></th> </tr> @@ -252,7 +252,7 @@ <div class="toolForm"> <div class="form-row"> <div class="msg_list"> - <h4 class="msg_head">Request Information</h4> + <h4 class="msg_head"><u>Request Information</u></h4> <div class="msg_body"> %for index, rd in enumerate(request_details): <div class="form-row"> @@ -282,11 +282,14 @@ </div> </div> +<br/> + <%def name="render_grid( grid_index, grid_name, fields_dict )"> + <br/> <div class="msg_list"> %if grid_name: - <h4 class="msg_head">${grid_name}</h4> + <h4 class="msg_head"><u>${grid_name}</u></h4> %else: <h4>Grid ${grid_index}</h4> %endif @@ -346,6 +349,7 @@ <% trans.sa_session.refresh( request.type.sample_form ) %> %for grid_index, grid_name in enumerate(request.type.sample_form.layout): ${render_grid( grid_index, grid_name, request.type.sample_form.fields_of_grid( grid_index ) )} + <br/> %endfor %else: <label>There are no samples.</label> diff -r c280939fd48d -r 52edc83ed81c templates/admin/requests/view_request_type.mako --- a/templates/admin/requests/view_request_type.mako Thu Feb 18 17:04:43 2010 -0500 +++ b/templates/admin/requests/view_request_type.mako Thu Feb 18 21:38:49 2010 -0500 @@ -58,6 +58,10 @@ <input type="password" name="password" value="${request_type.datatx_info['password']}" size="40"/> </div> <div class="form-row"> + <label>Data directory:</label> + <input type="text" name="data_dir" value="${request_type.datatx_info.get('data_dir', '')}" size="40"/> + </div> + <div class="form-row"> <input type="submit" name="save_changes" value="Save changes"/> </div> </form> diff -r c280939fd48d -r 52edc83ed81c templates/requests/show_request.mako --- a/templates/requests/show_request.mako Thu Feb 18 17:04:43 2010 -0500 +++ b/templates/requests/show_request.mako Thu Feb 18 21:38:49 2010 -0500 @@ -44,7 +44,7 @@ </script> <style type="text/css"> .msg_head { - padding: 15px 0px; + padding: 0px 0px; cursor: pointer; } @@ -124,7 +124,7 @@ <th>Data Library</th> <th>Folder</th> %if request.submitted() or request.complete(): - <th>Dataset(s)</th> + <th>Dataset(s) Transfered</th> %endif <th></th> </tr> @@ -240,7 +240,7 @@ <div class="toolForm"> <div class="form-row"> <div class="msg_list"> - <h4 class="msg_head">Request Information</h4> + <h4 class="msg_head"><u>Request Information</u></h4> <div class="msg_body"> %for index, rd in enumerate(request_details): <div class="form-row"> @@ -269,11 +269,13 @@ </div> </div> </div> +<br/> <%def name="render_grid( grid_index, grid_name, fields_dict )"> + <br/> <div class="msg_list"> %if grid_name: - <h4 class="msg_head">${grid_name}</h4> + <h4 class="msg_head"><u>${grid_name}</u></h4> %else: <h4>Grid ${grid_index}</h4> %endif @@ -333,6 +335,7 @@ <% trans.sa_session.refresh( request.type.sample_form ) %> %for grid_index, grid_name in enumerate(request.type.sample_form.layout): ${render_grid( grid_index, grid_name, request.type.sample_form.fields_of_grid( grid_index ) )} + <br/> %endfor %else: <label>There are no samples.</label>

1 0

[hg] galaxy 3414: Make page 'View' option use same URL as shared...
by Greg Von Kuster 19 Feb '10

19 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/c280939fd48d changeset: 3414:c280939fd48d user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Thu Feb 18 17:04:43 2010 -0500 description: Make page 'View' option use same URL as shared/accessible/published page does so that relative URLs behave the same in both contexts. This change addresses the issue raised by the temporarily broken hyperlink on the Bushman page. diffstat: lib/galaxy/web/controllers/page.py | 2 +- lib/galaxy/web/framework/helpers/grids.py | 5 +++++ 2 files changed, 6 insertions(+), 1 deletions(-) diffs (27 lines): diff -r bbff5050cfd2 -r c280939fd48d lib/galaxy/web/controllers/page.py --- a/lib/galaxy/web/controllers/page.py Thu Feb 18 11:50:19 2010 -0500 +++ b/lib/galaxy/web/controllers/page.py Thu Feb 18 17:04:43 2010 -0500 @@ -44,7 +44,7 @@ grids.GridAction( "Add new page", dict( action='create' ) ) ] operations = [ - grids.GridOperation( "View", allow_multiple=False, url_args=dict( action='display') ), + grids.DisplayByUsernameAndSlugGridOperation( "View", allow_multiple=False ), grids.GridOperation( "Edit name/id", allow_multiple=False, url_args=dict( action='edit') ), grids.GridOperation( "Edit content", allow_multiple=False, url_args=dict( action='edit_content') ), grids.GridOperation( "Share or Publish", allow_multiple=False, condition=( lambda item: not item.deleted ), async_compatible=False ), diff -r bbff5050cfd2 -r c280939fd48d lib/galaxy/web/framework/helpers/grids.py --- a/lib/galaxy/web/framework/helpers/grids.py Thu Feb 18 11:50:19 2010 -0500 +++ b/lib/galaxy/web/framework/helpers/grids.py Thu Feb 18 17:04:43 2010 -0500 @@ -541,6 +541,11 @@ return self.condition( item ) else: return True + +class DisplayByUsernameAndSlugGridOperation( GridOperation ): + """ Operation to display an item by username and slug. """ + def get_url_args( self, item ): + return { 'action' : 'display_by_username_and_slug', 'username' : item.user.username, 'slug' : item.slug } class GridAction( object ): def __init__( self, label=None, url_args=None ):

1 0

[hg] galaxy 3416: Re-fix new functional test support for setting...
by Greg Von Kuster 19 Feb '10

19 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/d995557d383b changeset: 3416:d995557d383b user: Greg Von Kuster <greg(a)bx.psu.edu> date: Fri Feb 19 10:01:33 2010 -0500 description: Re-fix new functional test support for setting check boxes and boolean tool params to false, and fix a broken functional test in the security and libraries suite. diffstat: lib/galaxy/web/controllers/library_common.py | 24 ++++++++++---------- templates/library/common/browse_library.mako | 4 +- templates/library/common/common.mako | 2 +- test-data/gops_concat_out2.bed | 10 ++++---- test/base/twilltestcase.py | 31 ++++++++++++++++--------- test/functional/test_security_and_libraries.py | 13 ++++------ 6 files changed, 45 insertions(+), 39 deletions(-) diffs (228 lines): diff -r 52edc83ed81c -r d995557d383b lib/galaxy/web/controllers/library_common.py --- a/lib/galaxy/web/controllers/library_common.py Thu Feb 18 21:38:49 2010 -0500 +++ b/lib/galaxy/web/controllers/library_common.py Fri Feb 19 10:01:33 2010 -0500 @@ -1067,7 +1067,6 @@ # use act_on_multiple_datasets( self, trans, cntrller, library_id, ldda_ids='', **kwd ) since it does what we need kwd['do_action'] = 'zip' return self.act_on_multiple_datasets( trans, cntrller, library_id, ldda_ids=id, **kwd ) - else: mime = trans.app.datatypes_registry.get_mimetype_by_extension( ldda.extension.lower() ) trans.response.set_content_type( mime ) @@ -1198,15 +1197,16 @@ msg = util.restore_text( params.get( 'msg', '' ) ) messagetype = params.get( 'messagetype', 'done' ) show_deleted = util.string_as_bool( params.get( 'show_deleted', False ) ) + action = params.get( 'do_action', None ) if not ldda_ids: msg = "You must select at least one dataset" messagetype = 'error' - elif not params.do_action: + elif not action: msg = "You must select an action to perform on selected datasets" messagetype = 'error' else: ldda_ids = util.listify( ldda_ids ) - if params.do_action == 'add': + if action == 'add': history = trans.get_history() total_imported_lddas = 0 msg = '' @@ -1223,7 +1223,7 @@ trans.sa_session.add( history ) trans.sa_session.flush() msg += "%i dataset(s) have been imported into your history. " % total_imported_lddas - elif params.do_action == 'manage_permissions': + elif action == 'manage_permissions': # We need the folder containing the LibraryDatasetDatasetAssociation(s) ldda = trans.sa_session.query( trans.app.model.LibraryDatasetDatasetAssociation ).get( trans.security.decode_id( ldda_ids[0] ) ) trans.response.send_redirect( web.url_for( controller='library_common', @@ -1235,7 +1235,7 @@ show_deleted=show_deleted, msg=util.sanitize_text( msg ), messagetype=messagetype ) ) - elif params.do_action == 'delete': + elif action == 'delete': for ldda_id in ldda_ids: ldda = trans.sa_session.query( trans.app.model.LibraryDatasetDatasetAssociation ).get( trans.security.decode_id( ldda_id ) ) ldda.deleted = True @@ -1245,18 +1245,18 @@ else: error = False try: - if params.do_action == 'zip': + if action == 'zip': # Can't use mkstemp - the file must not exist first tmpd = tempfile.mkdtemp() - tmpf = os.path.join( tmpd, 'library_download.' + params.do_action ) + tmpf = os.path.join( tmpd, 'library_download.' + action ) if ziptype == '64': archive = zipfile.ZipFile( tmpf, 'w', zipfile.ZIP_DEFLATED, True ) else: archive = zipfile.ZipFile( tmpf, 'w', zipfile.ZIP_DEFLATED ) archive.add = lambda x, y: archive.write( x, y.encode('CP437') ) - elif params.do_action == 'tgz': + elif action == 'tgz': archive = util.streamball.StreamBall( 'w|gz' ) - elif params.do_action == 'tbz': + elif action == 'tbz': archive = util.streamball.StreamBall( 'w|bz2' ) except (OSError, zipfile.BadZipFile): error = True @@ -1321,7 +1321,7 @@ msg = "Unable to create archive for download, please report this error" messagetype = 'error' if not error: - if params.do_action == 'zip': + if action == 'zip': archive.close() tmpfh = open( tmpf ) # clean up now @@ -1335,11 +1335,11 @@ messagetype = 'error' if not error: trans.response.set_content_type( "application/x-zip-compressed" ) - trans.response.headers[ "Content-Disposition" ] = "attachment; filename=GalaxyLibraryFiles.%s" % params.do_action + trans.response.headers[ "Content-Disposition" ] = "attachment; filename=GalaxyLibraryFiles.%s" % action return tmpfh else: trans.response.set_content_type( "application/x-tar" ) - trans.response.headers[ "Content-Disposition" ] = "attachment; filename=GalaxyLibraryFiles.%s" % params.do_action + trans.response.headers[ "Content-Disposition" ] = "attachment; filename=GalaxyLibraryFiles.%s" % action archive.wsgi_status = trans.response.wsgi_status() archive.wsgi_headeritems = trans.response.wsgi_headeritems() return archive.stream diff -r 52edc83ed81c -r d995557d383b templates/library/common/browse_library.mako --- a/templates/library/common/browse_library.mako Thu Feb 18 21:38:49 2010 -0500 +++ b/templates/library/common/browse_library.mako Fri Feb 19 10:01:33 2010 -0500 @@ -472,12 +472,12 @@ %if cntrller in [ 'library', 'requests' ]: ${self.render_folder( 'library', library.root_folder, 0, created_ldda_ids, trans.security.encode_id( library.id ), hidden_folder_ids, tracked_datasets, show_deleted=show_deleted, parent=None, row_counter=row_counter, root_folder=True )} %if not library.deleted: - ${render_actions_on_multiple_items( 'library', default_action=default_action )} + ${render_actions_on_multiple_items()} %endif %elif cntrller in [ 'library_admin', 'requests_admin' ]: ${self.render_folder( 'library_admin', library.root_folder, 0, created_ldda_ids, trans.security.encode_id( library.id ), [], tracked_datasets, show_deleted=show_deleted, parent=None, row_counter=row_counter, root_folder=True )} %if not library.deleted and not show_deleted: - ${render_actions_on_multiple_items( 'library_admin' )} + ${render_actions_on_multiple_items()} %endif %endif </table> diff -r 52edc83ed81c -r d995557d383b templates/library/common/common.mako --- a/templates/library/common/common.mako Thu Feb 18 21:38:49 2010 -0500 +++ b/templates/library/common/common.mako Fri Feb 19 10:01:33 2010 -0500 @@ -356,7 +356,7 @@ %endif </%def> -<%def name="render_actions_on_multiple_items( cntrller, default_action=None )"> +<%def name="render_actions_on_multiple_items()"> <tfoot> <tr> <td colspan="4" style="padding-left: 42px;"> diff -r 52edc83ed81c -r d995557d383b test-data/gops_concat_out2.bed --- a/test-data/gops_concat_out2.bed Thu Feb 18 21:38:49 2010 -0500 +++ b/test-data/gops_concat_out2.bed Fri Feb 19 10:01:33 2010 -0500 @@ -63,8 +63,8 @@ chrX 152648964 152649196 CCDS14733.1_cds_0_0_chrX_152648965_r 0 - chrX 152691446 152691471 CCDS14735.1_cds_0_0_chrX_152691447_f 0 + chrX 152694029 152694263 CCDS14736.1_cds_0_0_chrX_152694030_r 0 - -chr1 4348187 4348589 3.70 4.90 2.55 -chr1 4488177 4488442 4.03 5.77 1.92 -chr1 4774091 4774440 8.07 8.33 7.82 -chr1 4800122 4800409 6.40 7.35 5.44 -chr1 4878925 4879277 2.18 0.28 4.93 +chr1 4348187 4348589 . . + +chr1 4488177 4488442 . . + +chr1 4774091 4774440 . . + +chr1 4800122 4800409 . . + +chr1 4878925 4879277 . . + diff -r 52edc83ed81c -r d995557d383b test/base/twilltestcase.py --- a/test/base/twilltestcase.py Thu Feb 18 21:38:49 2010 -0500 +++ b/test/base/twilltestcase.py Fri Feb 19 10:01:33 2010 -0500 @@ -862,17 +862,14 @@ # Copied from form_builder.CheckboxField if value == True: return True - if isinstance( value, basestring ) and value.lower() in ( "yes", "true", "on" ): - return True - # This may look strange upon initial inspection, but see the comments in the get_html() method - # above for clarification. Basically, if value is not True, then it will always be a list with - # 2 input fields ( a checkbox and a hidden field ) if the checkbox is checked. If it is not - # checked, then value will be only the hidden field. - return isinstance( value, list ) and len( value ) == 2 + if isinstance( value, list ): + value = value[0] + return isinstance( value, basestring ) and value.lower() in ( "yes", "true", "on" ) try: checkbox = control.get() checkbox.selected = is_checked( control_value ) except Exception, e1: + print "Attempting to set checkbox selected value threw exception: ", e1 # if there's more than one checkbox, probably should use the behaviour for # ClientForm.ListControl ( see twill code ), but this works for now... for elem in control_value: @@ -885,6 +882,7 @@ try: tc.fv( f.name, control.name, str( elem ) ) except Exception, e2: + print "Attempting to set control '", control.name, "' to value '", elem, "' threw exception: ", e2 # Galaxy truncates long file names in the dataset_collector in ~/parameters/basic.py if len( elem ) > 30: elem_name = '%s..%s' % ( elem[:17], elem[-11:] ) @@ -1654,11 +1652,22 @@ self.home() def download_archive_of_library_files( self, cntrller, library_id, ldda_ids, format ): self.home() - self.visit_url( "%s/library_common/browse_library?cntrller=%s&id=%s" % ( self.url, cntrller, library_id ) ) + # Here it would be ideal to have twill set form values and submit the form, but + # twill barfs on that due to the recently introduced page wrappers around the contents + # of the browse_library.mako template which enable panel layout when visiting the + # page from an external URL. By "barfs", I mean that twill somehow loses hod on the + # cntrller param. We'll just simulate the form submission by building the URL manually. + # Here's the old, better approach... + #self.visit_url( "%s/library_common/browse_library?cntrller=%s&id=%s" % ( self.url, cntrller, library_id ) ) + #for ldda_id in ldda_ids: + # tc.fv( "1", "ldda_ids", ldda_id ) + #tc.fv( "1", "do_action", format ) + #tc.submit( "action_on_datasets_button" ) + # Here's the new approach... + url = "%s/library_common/act_on_multiple_datasets?cntrller=%s&library_id=%s&do_action=%s" % ( self.url, cntrller, library_id, format ) for ldda_id in ldda_ids: - tc.fv( "1", "ldda_ids", ldda_id ) - tc.fv( "1", "do_action", format ) - tc.submit( "action_on_datasets_button" ) + url += "&ldda_ids=%s" % ldda_id + self.visit_url( url ) tc.code( 200 ) archive = self.write_temp_file( self.last_page(), suffix=format ) self.home() diff -r 52edc83ed81c -r d995557d383b test/functional/test_security_and_libraries.py --- a/test/functional/test_security_and_libraries.py Thu Feb 18 21:38:49 2010 -0500 +++ b/test/functional/test_security_and_libraries.py Fri Feb 19 10:01:33 2010 -0500 @@ -1596,18 +1596,15 @@ self.home() self.logout() self.login( email=admin_user.email ) - """ - TODO: debug this, somebody recently broke it def test_167_download_archive_of_library_files( self ): - Testing downloading an archive of files from the library + """Testing downloading an archive of files from the library""" for format in ( 'tbz', 'tgz', 'zip' ): - archive = self.download_archive_of_library_files( 'library', - self.security.encode_id( library_one.id ), - ( self.security.encode_id( ldda_one.id ), self.security.encode_id( ldda_two.id ) ), - format ) + archive = self.download_archive_of_library_files( cntrller='library', + library_id=self.security.encode_id( library_one.id ), + ldda_ids=[ self.security.encode_id( ldda_one.id ), self.security.encode_id( ldda_two.id ) ], + format=format ) self.check_archive_contents( archive, ( ldda_one, ldda_two ) ) os.remove( archive ) - """ def test_170_mark_group_deleted( self ): """Testing marking a group as deleted""" # Logged in as admin_user

1 0

[hg] galaxy 3412: Added mm5 to the list of manual builds
by Greg Von Kuster 19 Feb '10

19 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/e05cdf1205bc changeset: 3412:e05cdf1205bc user: gua110 date: Thu Feb 18 11:26:39 2010 -0500 description: Added mm5 to the list of manual builds diffstat: tool-data/shared/ucsc/manual_builds.txt | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (9 lines): diff -r 0c07212afc00 -r e05cdf1205bc tool-data/shared/ucsc/manual_builds.txt --- a/tool-data/shared/ucsc/manual_builds.txt Thu Feb 18 09:47:21 2010 -0500 +++ b/tool-data/shared/ucsc/manual_builds.txt Thu Feb 18 11:26:39 2010 -0500 @@ -668,3 +668,4 @@ arabidopsis Arabidopsis thaliana TAIR9 arabidopsis_tair8 Arabidopsis thaliana TAIR8 araTha1 Arabidopsis thaliana TAIR7 +mm5 Mouse May 2004 (mm5) \ No newline at end of file

1 0

Fwd: QueuePool error
by SHAUN WEBB 18 Feb '10

18 Feb '10

Hi Galaxy Team, I sent this query last week and was wondering if anyone new anything about this error. I closed and re-started Galaxy to get it going again, however I have had to do this twice this week whenever we have multiple people using the server (3) and one is running a large workflow (15 steps). Also, we are having an additional problem when trying to convert large tabular files to interval. The file is over 12 million lines, we are getting a server time out error as I assume Galaxy tries to inspect the entire file before changing format. Thanks Shaun Webb Hi, our Galaxy server is showing a timeout error: TimeoutError: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 30 I assume this is due to overuse as it is consistent with the database settings in the universe.wsgi file. First of all, is there anyway to view and cancel queued jobs? I usually use the admin page but this is not accessible at the moment. Secondly, we plan for Galaxy to be used on a large scale, is there anyway of preventing this type of error in future? I've included the full traceback below. Thanks in advance for any help Shaun Webb URL: http://bifx3.bio.ed.ac.uk:8080/admin Module paste.exceptions.errormiddleware:143 in __call__ << try: __traceback_supplement__ = Supplement, self, environ app_iter = self.application(environ, start_response) return self.make_catching_iter(app_iter, environ) except:>> app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ << try: status, headers, body = wsgilib.intercept_output( environ, self.app) if status is None: # Some error occurred >> environ, self.app) Module paste.wsgilib:539 in intercept_output << data.append(headers) return output.write app_iter = application(environ, replacement_start_response) if data[0] is None: return (None, None, app_iter)>> app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ << environ['paste.recursive.script_name'] = my_script_name try: return self.application(environ, start_response) except ForwardRequestException, e: middleware = CheckForRecursionMiddleware(>> return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ << []).append(HTTPException) try: return self.application(environ, start_response) except HTTPException, exc: return exc(environ, start_response)>> return self.application(environ, start_response) Module galaxy.web.framework.base:99 in __call__ << rc.environ = environ # Setup the transaction trans = self.transaction_factory( environ ) rc.redirect = trans.response.send_redirect # Get the controller class >> trans = self.transaction_factory( environ ) Module galaxy.web.framework:110 in <lambda> << def __init__( self, galaxy_app, session_cookie='galaxysession' ): base.WebApplication.__init__( self ) self.set_transaction_factory( lambda e: UniverseWebTransaction( e, galaxy_app, self, session_cookie ) ) # Mako support self.mako_template_lookup = mako.lookup.TemplateLookup(>> self.set_transaction_factory( lambda e: UniverseWebTransaction( e, galaxy_app, self, session_cookie ) ) Module galaxy.web.framework:150 in __init__ << self.workflow_building_mode = False # Always have a valid galaxy session self.__ensure_valid_session( session_cookie ) # Prevent deleted users from accessing Galaxy if self.app.config.use_remote_user and self.galaxy_session.user.deleted:>> self.__ensure_valid_session( session_cookie ) Module galaxy.web.framework:279 in __ensure_valid_session << galaxy_session = self.sa_session.query( self.app.model.GalaxySession ) \ .filter( and_( self.app.model.GalaxySession.table.c.session_key==session_key, self.app.model.GalaxySession.table.c.is_valid==True ) ) \ .first() # If remote user is in use it can invalidate the session, so we need to to check some things now. >> self.app.model.GalaxySession.table.c.is_valid==True ) ) Module sqlalchemy.orm.query:1300 in first << ret = list(self)[0:1] else: ret = list(self[0:1]) if len(ret) > 0: return ret[0]>> ret = list(self[0:1]) Module sqlalchemy.orm.query:1221 in __getitem__ << return list(res)[None:None:item.step] else: return list(res) else: return list(self[item:item+1])[0]>> return list(res) Module sqlalchemy.orm.query:1361 in __iter__ << if self._autoflush and not self._populate_existing: self.session._autoflush() return self._execute_and_instances(context) def _execute_and_instances(self, querycontext):>> return self._execute_and_instances(context) Module sqlalchemy.orm.query:1364 in _execute_and_instances << def _execute_and_instances(self, querycontext): result = self.session.execute(querycontext.statement, params=self._params, mapper=self._mapper_zero_or_none()) return self.instances(result, querycontext)>> result = self.session.execute(querycontext.statement, params=self._params, mapper=self._mapper_zero_or_none()) Module sqlalchemy.orm.session:754 in execute << engine = self.get_bind(mapper, clause=clause, **kw) return self.__connection(engine, close_with_result=True).execute( clause, params or {})>> return self.__connection(engine, close_with_result=True).execute( Module sqlalchemy.orm.session:721 in __connection << return self.transaction._connection_for_bind(engine) else: return engine.contextual_connect(**kwargs) def execute(self, clause, params=None, mapper=None, **kw):>> return engine.contextual_connect(**kwargs) Module sqlalchemy.engine.base:1229 in contextual_connect << """ return self.Connection(self, self.pool.connect(), close_with_result=close_with_result, **kwargs) def table_names(self, schema=None, connection=None):>> return self.Connection(self, self.pool.connect(), close_with_result=close_with_result, **kwargs) Module sqlalchemy.pool:142 in connect << def connect(self): if not self._use_threadlocal: return _ConnectionFairy(self).checkout() try:>> return _ConnectionFairy(self).checkout() Module sqlalchemy.pool:304 in __init__ << self.__counter = 0 try: rec = self._connection_record = pool.get() conn = self.connection = self._connection_record.get_connection() self._connection_record.backref = weakref.ref(self, lambda ref:_finalize_fairy(conn, rec, pool, ref))>> rec = self._connection_record = pool.get() Module sqlalchemy.pool:161 in get << def get(self): return self.do_get() def do_get(self):>> return self.do_get() Module sqlalchemy.pool:628 in do_get << return self.do_get() else: raise exc.TimeoutError("QueuePool limit of size %d overflow %d reached, connection timed out, timeout %d" % (self.size(), self.overflow(), self._timeout)) if self._overflow_lock is not None:>> raise exc.TimeoutError("QueuePool limit of size %d overflow %d reached, connection timed out, timeout %d" % (self.size(), self.overflow(), self._timeout)) TimeoutError: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 30 -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ galaxy-dev mailing list galaxy-dev(a)lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev ----- End forwarded message ----- -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 1

Issue with boolean parameters in Galaxy
by Tam, Oliver 18 Feb '10

18 Feb '10

Hi, I have recently noticed a strange behavior with boolean parameters. The XML files are treating all boolean values as true when sending instructions to tools. This means that regardless of whether the boolean box is checked, the value of the parameter is always true. Tools with boolean parameters works fine with changeset 3372:4a6bb3b64012 But fails with changeset 3373:a3e6350ffc57 I have attached an XML that I¹ve been using to test this behavior (boolean_value_test.xml), which outputs yes (for a true value) and no (for a false value). Cheers, Oliver

2 1