October 2009 - galaxy-dev - lists.galaxyproject.org

Feature request: counting job queue time
by Gordon, Assaf 07 Oct '09

07 Oct '09

Hello, I have a small feature request: I'd like to be able to measure the time jobs wait in "queued" state - that is - how much time passed between the job's creation and the start of execution. The reason is that I'm testing a move from the local runner to an SGE runner. I'm sure the local runner is much faster in executing (assuming there are free job threads), but the SGE runner will eventually scale better. My main concern is that because my Galaxy will compete for the SGE cluster with other users, the advantages of using the cluster will be outweighed by the wasted time the SGE jobs will be queued and not executed. Thanks, -gordon

2 1

tool_conf.xml question
by Assaf Gordon 07 Oct '09

07 Oct '09

Hello, a small question regarding 'tool_conf.xml' : Is it possible to see the tool_conf.xml that you use for the main galaxy ? it is not "tool_conf.xml.main" (which is missing the NGS tools), and not exactly "tool_conf.xml.sample" (which has different ordering, and also contains references to RGenetics tools). Since it is not monitored in the mercurial repository, Cloning a fresh galaxy copy and running "sh setup.sh" does not reproduce the same tools and categories as your main web site. Also, The "tool_conf.xml.sample" contains "CutWrapper.xml" twice, and, Is it possible to have the RGenetics XMLs (and tools) ? They are referenced in the tool_conf.xml.sample, but do not exist in the repository. My goal is to have my local galaxy look as closely as possible to the public one, so people would be comfortable going from one to the other (and vice versa). Thanks! -gordon.

2 1

[hg] galaxy 2830: merge
by Greg Von Kuster 06 Oct '09

06 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/50dbd8907d88 changeset: 2830:50dbd8907d88 user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Mon Oct 05 17:39:29 2009 -0400 description: merge 0 file(s) affected in this change: diffs (35 lines): diff -r e6324f395d26 -r 50dbd8907d88 tools/filters/compare.xml --- a/tools/filters/compare.xml Mon Oct 05 17:38:54 2009 -0400 +++ b/tools/filters/compare.xml Mon Oct 05 17:39:29 2009 -0400 @@ -1,4 +1,4 @@ -<tool id="comp1" name="Compare two Queries"> +<tool id="comp1" name="Compare two Queries" version="1.0.1"> <description>to find common or distinct rows</description> <command interpreter="python">joinWrapper.py $input1 $input2 $field1 $field2 $mode $out_file1</command> <inputs> diff -r e6324f395d26 -r 50dbd8907d88 tools/filters/joinWrapper.py --- a/tools/filters/joinWrapper.py Mon Oct 05 17:38:54 2009 -0400 +++ b/tools/filters/joinWrapper.py Mon Oct 05 17:39:29 2009 -0400 @@ -22,8 +22,8 @@ try: #Sort the two files based on specified fields - os.system("sort -k %d -o %s %s" %(field1, tmpfile1.name, infile1)) - os.system("sort -k %d -o %s %s" %(field2, tmpfile2.name, infile2)) + os.system("sort -t $'\t' -k %d -o %s %s" %(field1, tmpfile1.name, infile1)) + os.system("sort -t $'\t' -k %d -o %s %s" %(field2, tmpfile2.name, infile2)) except Exception, exc: stop_err( 'Initialization error -> %s' %str(exc) ) @@ -40,9 +40,9 @@ break if mode == "V": - cmdline = 'join -v 1 -o %s -1 %d -2 %d %s %s | tr " " "\t" > %s' %(option, field1, field2, tmpfile1.name, tmpfile2.name, outfile) + cmdline = "join -t $'\t' -v 1 -o %s -1 %d -2 %d %s %s > %s" %(option, field1, field2, tmpfile1.name, tmpfile2.name, outfile) else: - cmdline = 'join -o %s -1 %d -2 %d %s %s | tr " " "\t" > %s' %(option, field1, field2, tmpfile1.name, tmpfile2.name, outfile) + cmdline = "join -t $'\t' -o %s -1 %d -2 %d %s %s > %s" %(option, field1, field2, tmpfile1.name, tmpfile2.name, outfile) try: os.system(cmdline)

1 0

[hg] galaxy 2829: Added method get_display_name() to History and...
by Greg Von Kuster 06 Oct '09

06 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/e6324f395d26 changeset: 2829:e6324f395d26 user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Mon Oct 05 17:38:54 2009 -0400 description: Added method get_display_name() to History and HistoryDatasetAssociation objects; these methods return unicode objects that can be used to display non-ascii characters. Fixes #181 4 file(s) affected in this change: lib/galaxy/model/__init__.py lib/galaxy/web/controllers/history.py templates/dataset/edit_attributes.mako templates/root/history.mako diffs (100 lines): diff -r f648d6dec544 -r e6324f395d26 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py Mon Oct 05 15:07:20 2009 -0400 +++ b/lib/galaxy/model/__init__.py Mon Oct 05 17:38:54 2009 -0400 @@ -245,6 +245,12 @@ def activatable_datasets( self ): # This needs to be a list return [ hda for hda in self.datasets if not hda.dataset.deleted ] + def get_display_name( self ): + ## History name can be either a string or a unicode object. If string, convert to unicode object assuming 'utf-8' format. + history_name = self.name + if isinstance(history_name, str): + history_name = unicode(history_name, 'utf-8') + return history_name class HistoryUserShareAssociation( object ): def __init__( self ): @@ -682,6 +688,13 @@ for assoc in self.implicitly_converted_datasets: if not metadata_safe or not assoc.metadata_safe: assoc.clear( purge = purge ) + def get_display_name( self ): + ## Name can be either a string or a unicode object. If string, convert to unicode object assuming 'utf-8' format. + hda_name = self.name + if isinstance(hda_name, str): + hda_name = unicode(hda_name, 'utf-8') + return hda_name + class HistoryDatasetAssociationDisplayAtAuthorization( object ): def __init__( self, hda=None, user=None, site=None ): diff -r f648d6dec544 -r e6324f395d26 lib/galaxy/web/controllers/history.py --- a/lib/galaxy/web/controllers/history.py Mon Oct 05 15:07:20 2009 -0400 +++ b/lib/galaxy/web/controllers/history.py Mon Oct 05 17:38:54 2009 -0400 @@ -18,6 +18,13 @@ class HistoryListGrid( grids.Grid ): # Custom column types + class NameColumn( grids.GridColumn ): + def __init( self, key, link, attach_popup ): + grids.GridColumn.__init__(self, key, link, attach_popup) + + def get_value( self, trans, grid, history ): + return history.get_display_name() + class DatasetsByStateColumn( grids.GridColumn ): def get_value( self, trans, grid, history ): rval = [] @@ -42,7 +49,7 @@ return dict( operation="sharing" ) return None class TagsColumn( grids.GridColumn ): - def __init__(self, col_name, key, filterable): + def __init__( self, col_name, key, filterable ): grids.GridColumn.__init__(self, col_name, key=key, filterable=filterable) # Tags cannot be sorted. self.sortable = False @@ -98,8 +105,8 @@ template='/history/grid.mako' default_sort_key = "-create_time" columns = [ - grids.GridColumn( "Name", key="name", - link=( lambda item: iff( item.deleted, None, dict( operation="switch", id=item.id ) ) ), + NameColumn( "Name", key="name", + link=( lambda history: iff( history.deleted, None, dict( operation="switch", id=history.id ) ) ), attach_popup=True ), DatasetsByStateColumn( "Datasets (by state)", ncells=4 ), TagsColumn( "Tags", key="tags", filterable=True), diff -r f648d6dec544 -r e6324f395d26 templates/dataset/edit_attributes.mako --- a/templates/dataset/edit_attributes.mako Mon Oct 05 15:07:20 2009 -0400 +++ b/templates/dataset/edit_attributes.mako Mon Oct 05 17:38:54 2009 -0400 @@ -35,7 +35,7 @@ Name: </label> <div style="float: left; width: 250px; margin-right: 10px;"> - <input type="text" name="name" value="${data.name}" size="40"/> + <input type="text" name="name" value="${data.get_display_name()}" size="40"/> </div> <div style="clear: both"></div> </div> @@ -155,7 +155,7 @@ %if trans.app.security_agent.can_manage_dataset( user_roles, data.dataset ): <%namespace file="/dataset/security_common.mako" import="render_permission_form" /> - ${render_permission_form( data.dataset, data.name, h.url_for( controller='root', action='edit', id=data.id ), user_roles )} + ${render_permission_form( data.dataset, data.get_display_name(), h.url_for( controller='root', action='edit', id=data.id ), user_roles )} %elif trans.user: <div class="toolForm"> <div class="toolFormTitle">View Permissions</div> diff -r f648d6dec544 -r e6324f395d26 templates/root/history.mako --- a/templates/root/history.mako Mon Oct 05 15:07:20 2009 -0400 +++ b/templates/root/history.mako Mon Oct 05 17:38:54 2009 -0400 @@ -290,7 +290,7 @@ <div id="history-name-area" class="historyLinks" style="color: gray; font-weight: bold;"> <div style="float: right"><a id="history-rename" title="Rename" class="icon-button edit" target="galaxy_main" href="${h.url_for( controller='history', action='rename' )}"></a></div> - <div id="history-name">${history.name}</div> + <div id="history-name">${history.get_display_name()}</div> </div> %if history.deleted:

1 0

[hg] galaxy 2832: small tweaks to bowtie & bwa interfaces
by Greg Von Kuster 06 Oct '09

06 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/a7a7376cf8e9 changeset: 2832:a7a7376cf8e9 user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Tue Oct 06 13:20:24 2009 -0400 description: small tweaks to bowtie & bwa interfaces 2 file(s) affected in this change: tools/sr_mapping/bowtie_wrapper.xml tools/sr_mapping/bwa_wrapper.xml diffs (28 lines): diff -r 8f87205219ef -r a7a7376cf8e9 tools/sr_mapping/bowtie_wrapper.xml --- a/tools/sr_mapping/bowtie_wrapper.xml Tue Oct 06 09:44:34 2009 -0400 +++ b/tools/sr_mapping/bowtie_wrapper.xml Tue Oct 06 13:20:24 2009 -0400 @@ -158,9 +158,9 @@ <when value="history"> <param name="ownFile" type="data" format="fasta" metadata_name="dbkey" label="Select a reference genome" /> <conditional name="indexParams"> - <param name="index_settings" type="select" label="Choose whether to use default options or to set your own"> - <option value="index_pre_set">Commonly Used</option> - <option value="index_full">Full Parameter List</option> + <param name="index_settings" type="select" label="Choose whether to use default options for building indices or to set your own"> + <option value="index_pre_set">Default</option> + <option value="index_full">Set your own</option> </param> <when value="index_pre_set" /> <when value="index_full"> diff -r 8f87205219ef -r a7a7376cf8e9 tools/sr_mapping/bwa_wrapper.xml --- a/tools/sr_mapping/bwa_wrapper.xml Tue Oct 06 09:44:34 2009 -0400 +++ b/tools/sr_mapping/bwa_wrapper.xml Tue Oct 06 13:20:24 2009 -0400 @@ -103,7 +103,7 @@ </param> </when> <when value="history"> - <param name="ownFile" type="data" format="fasta" metadata_name="dbkey" label="Select a reference genome" /> + <param name="ownFile" type="data" format="fasta" metadata_name="dbkey" label="Select a reference from history" /> </when> </conditional> </when>

1 0

[hg] galaxy 2831: bug fix for pilup_parser reported by user
by Greg Von Kuster 06 Oct '09

06 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/8f87205219ef changeset: 2831:8f87205219ef user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Tue Oct 06 09:44:34 2009 -0400 description: bug fix for pilup_parser reported by user 1 file(s) affected in this change: tools/samtools/pileup_parser.pl diffs (35 lines): diff -r 50dbd8907d88 -r 8f87205219ef tools/samtools/pileup_parser.pl --- a/tools/samtools/pileup_parser.pl Mon Oct 05 17:39:29 2009 -0400 +++ b/tools/samtools/pileup_parser.pl Tue Oct 06 09:44:34 2009 -0400 @@ -37,17 +37,19 @@ die "Coverage column" . ($cvrg_column+1) . " contains non-numeric values. Check your input parameters as well as format of input dataset." if ( not isdigit $fields[ $cvrg_column ] ); next if $fields[ $cvrg_column ] < $cvrg_cutoff; my $base_quality = $fields[ $base_quality_column ]; - if ($read_bases =~ m/[\$\^\+-]/) { + $read_bases =~ s/\^.//g; #removing the start of the read segement mark $read_bases =~ s/\$//g; #removing end of the read segment mark - $read_bases =~ s/\^.//g; #removing the start of the read segement mark while ($read_bases =~ m/[\+-]{1}(\d+)/g) { my $indel_len = $1; $read_bases =~ s/[\+-]{1}$indel_len.{$indel_len}//; # remove indel info from read base field } } - die "Error parsing read bases and qualities in line $.. Last processed line conatined these values: " . join("\t", @fields) . "\n" if ( length($read_bases) != length($base_quality) ); - + if ( length($read_bases) != length($base_quality) ) { + $first_skipped_line = $. if $first_skipped_line eq ""; + ++$invalid_line_counter; + next; + } # after removing read block and indel data the length of read_base # field should identical to the length of base_quality field @@ -94,6 +96,6 @@ } -print STDERR "Could not parse $invalid_line_counter line(s) beginning with: $first_skipped_line\n" if $invalid_line_counter > 0; +print "Skipped $invalid_line_counter invalid line(s) beginning with line $first_skipped_line\n" if $invalid_line_counter > 0; close IN; close OUT;

1 0

[hg] galaxy 2833: Bug fix for adding a new folder template ( fix...
by Greg Von Kuster 06 Oct '09

06 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/40ab85150694 changeset: 2833:40ab85150694 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Tue Oct 06 14:13:31 2009 -0400 description: Bug fix for adding a new folder template ( fixes # 193 ), more functioanl test coverage for folder templates, more code mering and cleanup in the library controllers, shortened the table buttons on the brose library grid and template. 8 file(s) affected in this change: lib/galaxy/web/controllers/library.py lib/galaxy/web/controllers/library_common.py templates/admin/library/browse_library.mako templates/admin/library/select_info_template.mako templates/library/browse_library.mako templates/library/select_info_template.mako test/base/twilltestcase.py test/functional/test_security_and_libraries.py diffs (389 lines): diff -r a7a7376cf8e9 -r 40ab85150694 lib/galaxy/web/controllers/library.py --- a/lib/galaxy/web/controllers/library.py Tue Oct 06 13:20:24 2009 -0400 +++ b/lib/galaxy/web/controllers/library.py Tue Oct 06 14:13:31 2009 -0400 @@ -70,7 +70,7 @@ trans.app.security_agent.permitted_actions.LIBRARY_MANAGE ] # The authorized_libraries dictionary looks like: { library : '1,2' }, library : '3' } # Its keys are the libraries that should be displayed for the current user and whose values are a - # string of comma-separated folder ids, of the associated folders the should NOT be displayed. + # string of comma-separated folder ids of the associated folders that should NOT be displayed. # The folders that should not be displayed may not be a complete list, but it is ultimately passed # to the browse_library() method and the browse_library.mako template to keep from re-checking the # same folders when the library is rendered. diff -r a7a7376cf8e9 -r 40ab85150694 lib/galaxy/web/controllers/library_common.py --- a/lib/galaxy/web/controllers/library_common.py Tue Oct 06 13:20:24 2009 -0400 +++ b/lib/galaxy/web/controllers/library_common.py Tue Oct 06 14:13:31 2009 -0400 @@ -211,7 +211,7 @@ messagetype='error' ) ) @web.expose def info_template( self, trans, cntrller, library_id, response_action='library', obj_id=None, folder_id=None, ldda_id=None, **kwd ): - # Only adding a new templAte to a library or folder is currently allowed. Editing an existing template is + # Only adding a new template to a library or folder is currently allowed. Editing an existing template is # a future enhancement. The response_action param is the name of the method to which this method will redirect # if a new template is being added to a library or folder. params = util.Params( kwd ) @@ -241,7 +241,6 @@ msg += "try again to add the information template to the %s." % library_item_desc trans.response.send_redirect( web.url_for( controller='forms', action='new', - new=True, msg=msg, messagetype='done' ) ) if params.get( 'add_info_template_button', False ): @@ -260,19 +259,17 @@ trans.response.send_redirect( web.url_for( controller=cntrller, action=response_action, obj_id=response_id, + library_id=library_id, msg=msg, message_type='done' ) ) - # TODO: handle this better - if cntrller == 'library_admin': - tmplt = '/admin/library/select_info_template.mako' - else: - tmplt = '/library/select_info_template.mako' - return trans.fill_template( tmplt, + return trans.fill_template( '/library/select_info_template.mako', + controller=cntrller, library_item_name=library_item.name, library_item_desc=library_item_desc, library_id=library_id, folder_id=folder_id, ldda_id=ldda_id, + obj_id=response_id, forms=forms, msg=msg, messagetype=messagetype ) diff -r a7a7376cf8e9 -r 40ab85150694 templates/admin/library/browse_library.mako --- a/templates/admin/library/browse_library.mako Tue Oct 06 13:20:24 2009 -0400 +++ b/templates/admin/library/browse_library.mako Tue Oct 06 14:13:31 2009 -0400 @@ -244,10 +244,10 @@ <ul class="manage-table-actions"> %if not deleted: <li> - <a class="action-button" href="${h.url_for( controller='library_admin', action='upload_library_dataset', library_id=library.id, folder_id=library.root_folder.id )}"><span>Add datasets to this data library</span></a> + <a class="action-button" href="${h.url_for( controller='library_admin', action='upload_library_dataset', library_id=library.id, folder_id=library.root_folder.id )}"><span>Add datasets</span></a> </li> <li> - <a class="action-button" href="${h.url_for( controller='library_admin', action='folder', new=True, obj_id=library.root_folder.id, library_id=library.id )}">Add a folder to this data library</a> + <a class="action-button" href="${h.url_for( controller='library_admin', action='folder', new=True, obj_id=library.root_folder.id, library_id=library.id )}">Add folder</a> </li> %endif </ul> diff -r a7a7376cf8e9 -r 40ab85150694 templates/admin/library/select_info_template.mako --- a/templates/admin/library/select_info_template.mako Tue Oct 06 13:20:24 2009 -0400 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,44 +0,0 @@ -<%inherit file="/base.mako"/> -<%namespace file="/message.mako" import="render_msg" /> - -<br/><br/> -<ul class="manage-table-actions"> - <li> - <a class="action-button" href="${h.url_for( controller='library_admin', action='browse_library', obj_id=library_id )}"><span>Browse this data library</span></a> - </li> -</ul> - -%if msg: - ${render_msg( msg, messagetype )} -%endif - -<div class="toolForm"> - <div class="toolFormTitle">Select a form on which to base the template for the ${library_item_desc} '${library_item_name}'</div> - <form name="new_info_template" action="${h.url_for( controller='library_common', action='info_template', cntrller='library_admin' )}" method="post" > - <div class="toolFormBody"> - <div class="form-row"> - <input type="hidden" name="library_id" value="${library_id}"/> - %if library_dataset_id: - <input type="hidden" name="library_dataset_id" value="${library_dataset_id}"/> - <input type="hidden" name="response_action" value="ldda_edit_info"/> - %elif folder_id: - <input type="hidden" name="folder_id" value="${folder_id}"/> - <input type="hidden" name="response_action" value="folder"/> - %elif ldda_id: - <input type="hidden" name="ldda_id" value="${ldda_id}"/> - <input type="hidden" name="response_action" value="ldda_edit_info"/> - %endif - <label>Template:</label> - <select name="form_id"> - %for form in forms: - <option value="${form.id}">${form.name}</option> - %endfor - </select> - </div> - <div style="clear: both"></div> - <div class="form-row"> - <input type="submit" name="add_info_template_button" value="Add template to ${library_item_desc}"/> - </div> - </div> - </form> -</div> diff -r a7a7376cf8e9 -r 40ab85150694 templates/library/browse_library.mako --- a/templates/library/browse_library.mako Tue Oct 06 13:20:24 2009 -0400 +++ b/templates/library/browse_library.mako Tue Oct 06 14:13:31 2009 -0400 @@ -215,6 +215,8 @@ if created_ldda_ids: created_ldda_ids = [ int( ldda_id ) for ldda_id in util.listify( created_ldda_ids ) ] can_add = trans.app.security_agent.can_add_library_item( user, roles, folder ) + if can_add: + info_association, inherited = folder.get_info_association( restrict=True ) can_modify = trans.app.security_agent.can_modify_library_item( user, roles, folder ) can_manage = trans.app.security_agent.can_manage_library_item( user, roles, folder ) my_row = None @@ -245,7 +247,7 @@ %else: <a class="action-button" href="${h.url_for( controller='library', action='folder', information=True, obj_id=folder.id, library_id=library_id )}">View this folder's information</a> %endif - %if can_add and forms and not folder.info_association: + %if can_add and not info_association: <a class="action-button" href="${h.url_for( controller='library_common', action='info_template', cntrller='library', library_id=library.id, response_action='folder', folder_id=folder.id )}">Add an information template to this folder</a> %endif %if can_manage: @@ -280,6 +282,8 @@ <% can_add = trans.app.security_agent.can_add_library_item( user, roles, library ) + if can_add: + info_association, inherited = library.get_info_association() can_modify = trans.app.security_agent.can_modify_library_item( user, roles, library ) can_manage = trans.app.security_agent.can_manage_library_item( user, roles, library ) %> @@ -287,22 +291,22 @@ <ul class="manage-table-actions"> %if can_add: <li> - <a class="action-button" href="${h.url_for( controller='library', action='upload_library_dataset', library_id=library.id, folder_id=library.root_folder.id )}"><span>Add datasets to this data library</span></a> + <a class="action-button" href="${h.url_for( controller='library', action='upload_library_dataset', library_id=library.id, folder_id=library.root_folder.id )}"><span>Add datasets</span></a> </li> <li> - <a class="action-button" href="${h.url_for( controller='library', action='folder', new=True, obj_id=library.root_folder.id, library_id=library.id )}">Add a folder to this data library</a> + <a class="action-button" href="${h.url_for( controller='library', action='folder', new=True, obj_id=library.root_folder.id, library_id=library.id )}">Add folder</a> </li> %endif %if can_modify: - <li><a class="action-button" href="${h.url_for( controller='library', action='library', information=True, obj_id=library.id )}">Edit this data library's information</a></li> + <li><a class="action-button" href="${h.url_for( controller='library', action='library', information=True, obj_id=library.id )}">Edit information</a></li> %else: - <li><a class="action-button" href="${h.url_for( controller='library', action='library', information=True, obj_id=library.id )}">View this data library's information</a></li> + <li><a class="action-button" href="${h.url_for( controller='library', action='library', information=True, obj_id=library.id )}">View information</a></li> %endif - %if can_add and forms and not library.info_association: - <a class="action-button" href="${h.url_for( controller='library_common', action='info_template', cntrller='library', library_id=library.id, response_action='browse_library' )}">Add an information template to this data library</a> + %if can_add and not info_association: + <a class="action-button" href="${h.url_for( controller='library_common', action='info_template', cntrller='library', library_id=library.id, response_action='browse_library' )}">Add template</a> %endif %if can_manage: - <li><a class="action-button" href="${h.url_for( controller='library', action='library', permissions=True, obj_id=library.id )}">Edit this data library's permissions</a></li> + <li><a class="action-button" href="${h.url_for( controller='library', action='library', permissions=True, obj_id=library.id )}">Edit permissions</a></li> %endif </ul> diff -r a7a7376cf8e9 -r 40ab85150694 templates/library/select_info_template.mako --- a/templates/library/select_info_template.mako Tue Oct 06 13:20:24 2009 -0400 +++ b/templates/library/select_info_template.mako Tue Oct 06 14:13:31 2009 -0400 @@ -4,7 +4,7 @@ <br/><br/> <ul class="manage-table-actions"> <li> - <a class="action-button" href="${h.url_for( controller='library', action='browse_library', obj_id=library_id )}"><span>Browse this data library</span></a> + <a class="action-button" href="${h.url_for( controller=controller, action='browse_library', obj_id=library_id )}"><span>Browse this data library</span></a> </li> </ul> @@ -14,7 +14,7 @@ <div class="toolForm"> <div class="toolFormTitle">Select a form on which to base the template for the ${library_item_desc} '${library_item_name}'</div> - <form name="new_info_template" action="${h.url_for( controller='library_common', action='info_template', cntrller='library' )}" method="post" > + <form name="new_info_template" action="${h.url_for( controller='library_common', action='info_template', cntrller=controller )}" method="post" > <div class="toolFormBody"> <div class="form-row"> <input type="hidden" name="library_id" value="${library_id}"/> diff -r a7a7376cf8e9 -r 40ab85150694 test/base/twilltestcase.py --- a/test/base/twilltestcase.py Tue Oct 06 13:20:24 2009 -0400 +++ b/test/base/twilltestcase.py Tue Oct 06 14:13:31 2009 -0400 @@ -1171,8 +1171,8 @@ check_str = "Library '%s' has been renamed to '%s'" % ( old_name, name ) self.check_page_for_string( check_str ) self.home() - def add_library_info_template( self, cntrller, library_id, library_name, form_id, form_name ): - """Add a new info template to a library""" + def add_library_info_template( self, cntrller, library_id, form_id, form_name ): + """Add a new info template to a library item""" self.home() url = "%s/library_common/info_template?cntrller=%s&library_id=%s&response_action='browse_library'" % ( self.url, cntrller, library_id ) self.visit_url( url ) @@ -1209,25 +1209,16 @@ check_str = "Information template '%s' has been updated" % name self.check_page_for_string( check_str ) self.home() - def add_folder_info_template( self, controller, cntrller, library_id, library_name, folder_id, folder_name, num_fields='2', - name='Folder Template 1', ele_name_0='Fu', ele_help_0='', ele_name_1='Bar', ele_help_1='' ): + def add_folder_info_template( self, cntrller, library_id, folder_id, form_id, form_name ): """Add a new info template to a folder""" self.home() - url = "%s/%s/info_template?cntrller=%s&library_id=%s&response_action='folder'&create_info_template_button=Go" % \ - ( self.url, controller, cntrller, library_id, folder_id ) - self.home() + url = "%s/library_common/info_template?cntrller=%s&library_id=%s&folder_id=%s&response_action='folder'" % \ + ( self.url, cntrller, library_id, folder_id ) self.visit_url( url ) - check_str = "Create a new information template for folder '%s'" % folder_name - self.check_page_for_string ( check_str ) + self.check_page_for_string ( "Select a form on which to base the template" ) tc.fv( '1', 'library_id', library_id ) - tc.fv( '1', 'folder_id', folder_id ) - tc.fv( '1', 'set_num_fields', num_fields ) - tc.fv( '1', 'name', name ) - tc.fv( '1', 'new_element_name_0', ele_name_0 ) - tc.fv( '1', 'new_element_description_0', ele_help_0.replace( '+', ' ' ) ) - tc.fv( '1', 'new_element_name_1', ele_name_1 ) - tc.fv( '1', 'new_element_description_1', ele_help_1.replace( '+', ' ' ) ) - tc.submit( 'new_info_template_button' ) + tc.submit( 'add_info_template_button' ) + self.check_page_for_string = 'An information template based on the form "%s" has been added to this folder.' % form_name self.home() def add_folder( self, controller, library_id, folder_id, name='Folder One', description='This is Folder One' ): """Create a new folder""" @@ -1239,17 +1230,26 @@ tc.fv( "1", "description", description ) # form field 2 is the field named description... tc.submit( "new_folder_button" ) self.home() - def edit_folder_info( self, controller, folder_id, library_id, name, new_name, description ): + def edit_folder_info( self, controller, folder_id, library_id, name, new_name, description, contents='', field_name='' ): """Add information to a library using an existing template with 2 elements""" self.home() self.visit_url( "%s/%s/folder?obj_id=%s&library_id=%s&information=True" % \ ( self.url, controller, folder_id, library_id) ) - self.check_page_for_string( "Edit folder name and description" ) + # Twill cannot handle the following call for some reason - it's buggy + # self.check_page_for_string( "Edit folder name and description" ) tc.fv( '1', "name", new_name ) tc.fv( '1', "description", description ) tc.submit( 'rename_folder_button' ) - check_str = "Folder '%s' has been renamed to '%s'" % ( name, new_name ) - self.check_page_for_string( check_str ) + # Twill cannot handle the following call for some reason - it's buggy + # check_str = "Folder '%s' has been renamed to '%s'" % ( name, new_name ) + # self.check_page_for_string( check_str ) + if contents and field_name: + # We have an information template associated with the folder, so + # there are 2 forms on this page and the template is the 2nd form + tc.fv( '2', field_name, contents ) + tc.submit( 'edit_info_button' ) + # Twill cannot handle the following call for some reason - it's buggy + # self.check_page_for_string( 'The information has been updated.' ) self.home() def rename_folder( self, library_id, folder_id, old_name, name='Folder One Renamed', description='This is Folder One Re-described' ): """Rename a Folder""" diff -r a7a7376cf8e9 -r 40ab85150694 test/functional/test_security_and_libraries.py --- a/test/functional/test_security_and_libraries.py Tue Oct 06 13:20:24 2009 -0400 +++ b/test/functional/test_security_and_libraries.py Tue Oct 06 14:13:31 2009 -0400 @@ -497,7 +497,6 @@ template_name = 'Library Template 1' self.add_library_info_template( 'library_admin', str( library_one.id ), - library_one.name, str( form_one.id ), form_one.name ) # Make sure the template fields are displayed on the library information page @@ -1622,6 +1621,7 @@ self.visit_page( 'library_admin/browse_libraries' ) self.check_page_for_string( name ) self.check_page_for_string( description ) + global library_three library_three = galaxy.model.Library.filter( and_( galaxy.model.Library.table.c.name==name, galaxy.model.Library.table.c.description==description, galaxy.model.Library.table.c.deleted==False ) ).first() @@ -1658,20 +1658,41 @@ description = "This is the root folder's Folder X" self.add_folder( 'library', str( library_three.id ), - str( library_three.root_folder.id ), + str( library_three.root_folder.id ), name=name, description=description ) + global folder_x folder_x = galaxy.model.LibraryFolder.filter( and_( galaxy.model.LibraryFolder.table.c.parent_id==library_three.root_folder.id, galaxy.model.LibraryFolder.table.c.name==name, galaxy.model.LibraryFolder.table.c.description==description ) ).first() + # Add an information template to the folder + template_name = 'Folder Template 1' + self.add_folder_info_template( 'library', + str( library_one.id ), + str( folder_x.id ), + str( form_one.id ), + form_one.name ) # Modify the folder's information + contents = '%s folder contents' % form_one_field_label new_name = "Root Folder's Folder Y" new_description = "This is the root folder's Folder Y" - self.edit_folder_info( 'library', str( folder_x.id ), str( library_three.id ), name, new_name, new_description ) + self.edit_folder_info( 'library', + str( folder_x.id ), + str( library_three.id ), + name, + new_name, + new_description, + contents=contents, + field_name=form_one_field_name ) + # Twill barfs when self.check_page_for_string() is called after dealing with an information template, + # the exception is: TypeError: 'str' object is not callable + # the work-around it to end this method so any calls are in the next method. + def test_265_template_features_and_permissions( self ): + """Test library template and more permissions behavior from the Data Libraries view""" + # Logged in as regular_user2 folder_x.refresh() # Add a dataset to the folder - name2 = "Folder Y subfolder" - description2 = "Folder Y subfolder description" + message = 'Testing adding 2.bed to Library Three root folder' self.add_library_dataset( 'library', '2.bed', str( library_three.id ), @@ -1681,21 +1702,41 @@ dbkey='hg18', message=message.replace( ' ', '+' ), root=False ) + global ldda_x ldda_x = galaxy.model.LibraryDatasetDatasetAssociation.query() \ .order_by( desc( galaxy.model.LibraryDatasetDatasetAssociation.table.c.create_time ) ).first() assert ldda_x is not None, 'Problem retrieving ldda_x from the database' - # Log in as regular_user1 + # Add an information template to the library + template_name = 'Library Template 3' + self.add_library_info_template( 'library', + str( library_three.id ), + str( form_one.id ), + form_one.name ) + # Add information to the library using the template + contents = '%s library contents' % form_one_field_label + self.visit_url( '%s/library/library?obj_id=%s&information=True' % ( self.url, str( library_three.id ) ) ) + # There are 2 forms on this page and the template is the 2nd form + tc.fv( '2', form_one_field_name, contents ) + tc.submit( 'edit_info_button' ) + # For some reason, the following check: + # self.check_page_for_string ( 'The information has been updated.' ) + # ...throws the following exception - I have not idea why! + # TypeError: 'str' object is not callable + # The work-around is to not make ANY self.check_page_for_string() calls until the next method + def test_270_permissions_as_different_regular_user( self ): + """Test library template and more permissions behavior from the Data Libraries view as a different user""" + # Log in as regular_user2 self.logout() self.login( email=regular_user1.email ) self.visit_url( '%s/library/browse_library?obj_id=%s' % ( self.url, str( library_three.id ) ) ) self.check_page_for_string( ldda_x.name ) + def test_275_reset_data_for_later_test_runs( self ): + """Reseting data to enable later test runs to pass""" + # Logged in as regular_user2 self.logout() self.login( email=admin_user.email ) self.delete_library_item( str( library_three.id ), str( library_three.id ), library_three.name, library_item_type='library' ) self.purge_library( str( library_three.id ), library_three.name ) - def test_265_reset_data_for_later_test_runs( self ): - """Reseting data to enable later test runs to pass""" - # Logged in as admin_user ################## # Eliminate all non-private roles ##################

1 0

[hg] galaxy 2826: Several small changes to Samtools pileup tool ...
by Greg Von Kuster 06 Oct '09

06 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/e93da066b847 changeset: 2826:e93da066b847 user: Kelly Vincent <kpvincent(a)bx.psu.edu> date: Mon Oct 05 10:35:39 2009 -0400 description: Several small changes to Samtools pileup tool to get it to work with user-supplied reference genome 1 file(s) affected in this change: tools/samtools/sam_pileup.py diffs (39 lines): diff -r a60c55fb0d76 -r e93da066b847 tools/samtools/sam_pileup.py --- a/tools/samtools/sam_pileup.py Sun Oct 04 10:55:43 2009 -0400 +++ b/tools/samtools/sam_pileup.py Mon Oct 05 10:35:39 2009 -0400 @@ -68,14 +68,16 @@ stop_err( "No sequences are available for '%s', request them by reporting this error." % options.dbkey ) cmd3 = cmd3 % (opts, seq_path, tmpf0bam, options.output1) elif options.ref == 'history': - cmd1 = 'cp %s %s; cp %s.fai %s' % (options.ownFile, tmpf1.name, options.ownFile, tmpf1fai) + cmd1 = 'cp %s %s; samtools faidx %s' % (options.ownFile, tmpf1.name, tmpf1.name) cmd3 = cmd3 % (opts, tmpf1.name, tmpf0bam, options.output1) # index reference if necessary if cmd1: try: os.system(cmd1) + if options.ref == 'history' and not os.path.exists( tmpf1fai ): + stop_err( "Problem creating index file from history item." ) except Exception, eq: - stop_err('Error moving reference sequence\n' + str(eq)) + stop_err('Error handling reference sequence\n' + str(eq)) # copy bam index to working directory try: os.system(cmd2) @@ -85,7 +87,7 @@ try: os.system(cmd3) except Exception, eq: - stop_err('Error running SAMtools merge tool\n' + str(eq)) + stop_err('Error running SAMtools pileup tool\n' + str(eq)) # clean up temp files tmpf1.close() tmpf0.close() @@ -94,6 +96,6 @@ if os.path.exists(tmpf0bambai): os.remove(tmpf0bambai) if os.path.exists(tmpf1fai): - os.remove(tmpf0bambai) + os.remove(tmpf1fai) if __name__ == "__main__" : __main__()

1 0

[hg] galaxy 2827: Improved paging UI for history grid and added ...
by Greg Von Kuster 06 Oct '09

06 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/f648d6dec544 changeset: 2827:f648d6dec544 user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Mon Oct 05 15:07:20 2009 -0400 description: Improved paging UI for history grid and added keyword 'all' for page varible. Specifying 'page=all' in URL string shows all items on single page. 3 file(s) affected in this change: lib/galaxy/web/controllers/history.py lib/galaxy/web/framework/helpers/grids.py templates/history/grid.mako diffs (148 lines): diff -r e93da066b847 -r f648d6dec544 lib/galaxy/web/controllers/history.py --- a/lib/galaxy/web/controllers/history.py Mon Oct 05 10:35:39 2009 -0400 +++ b/lib/galaxy/web/controllers/history.py Mon Oct 05 15:07:20 2009 -0400 @@ -125,6 +125,7 @@ grids.GridColumnFilter( "All", args=dict( deleted='All' ) ), ] default_filter = dict( deleted="False", tags="All" ) + num_rows_per_page = 50 preserve_state = False use_paging = True def get_current_item( self, trans ): diff -r e93da066b847 -r f648d6dec544 lib/galaxy/web/framework/helpers/grids.py --- a/lib/galaxy/web/framework/helpers/grids.py Mon Oct 05 10:35:39 2009 -0400 +++ b/lib/galaxy/web/framework/helpers/grids.py Mon Oct 05 15:07:20 2009 -0400 @@ -25,7 +25,7 @@ preserve_state = False use_paging = False - num_rows_per_page = 10 + num_rows_per_page = 25 # Set preference names. cur_filter_pref_name = ".filter" @@ -123,15 +123,23 @@ # Process page number. if self.use_paging: if 'page' in kwargs: - page_num = int( kwargs['page'] ) + if kwargs['page'] == 'all': + page_num = 0 + else: + page_num = int( kwargs['page'] ) else: page_num = 1 - # Before modifying query, get the total number of rows that query returns so that the total number of pages can - # be computed. - total_num_rows = query.count() - query = query.limit( self.num_rows_per_page ).offset( ( page_num-1 ) * self.num_rows_per_page ) - num_pages = int ( math.ceil( float( total_num_rows ) / self.num_rows_per_page ) ) + if page_num == 0: + # Show all rows in page. + total_num_rows = query.count() + num_pages = 1 + else: + # Show a limited number of rows. Before modifying query, get the total number of rows that query + # returns so that the total number of pages can be computed. + total_num_rows = query.count() + query = query.limit( self.num_rows_per_page ).offset( ( page_num-1 ) * self.num_rows_per_page ) + num_pages = int ( math.ceil( float( total_num_rows ) / self.num_rows_per_page ) ) else: # Defaults. page_num = 1 diff -r e93da066b847 -r f648d6dec544 templates/history/grid.mako --- a/templates/history/grid.mako Mon Oct 05 10:35:39 2009 -0400 +++ b/templates/history/grid.mako Mon Oct 05 15:07:20 2009 -0400 @@ -92,9 +92,8 @@ t.autocomplete("${h.url_for( controller='tag', action='tag_autocomplete_data', item_class='History' )}", autocomplete_options); - //t.addClass("tag-input"); - - return t; + + $("#page-select").change(navigate_to_page); }); ## Can this be moved into base.mako? %if refresh_frames: @@ -141,6 +140,18 @@ %> var url_base = "${url( url_args )}"; var url = url_base.replace("TAGNAME", tag_name); + self.location = url; + } + + // + // Initiate navigation when user selects a page to view. + // + function navigate_to_page() + { + page_num = $(this).val(); + <% url_args = {"page" : "PAGE"} %> + var url_base = "${url( url_args )}"; + var url = url_base.replace("PAGE", page_num); self.location = url; } @@ -207,6 +218,7 @@ </form> </div> <form name="history_actions" action="${url()}" method="post" > + <input type="hidden" name="page" value="${cur_page_num}"> <table class="grid"> <thead> <tr> @@ -300,19 +312,42 @@ %endfor </tbody> <tfoot> + ## Row for navigating among pages. %if num_pages > 1: <tr> <td></td> - <td colspan="100" style="text-align: right"> - Page: - %for page_index in range(1, num_pages + 1): - %if page_index == cur_page_num: - <span style="font-style: italic">${page_index}</span> - %else: - <% args = { "page" : page_index } %> - <span><a href="${url( args )}">${page_index}</a></span> - %endif - %endfor + <td colspan="100"> + Page ${cur_page_num} of ${num_pages} +     Go to: + ## Next page link. + %if cur_page_num != num_pages: + <% args = { "page" : cur_page_num+1 } %> + <span><a href="${url( args )}">Next</a></span> + %endif + ## Previous page link. + %if cur_page_num != 1: + <span>|</span> + <% args = { "page" : cur_page_num-1 } %> + <span><a href="${url( args )}">Previous</a></span> + %endif + ## Go to page select box. + <span>| Select:</span> + <select id="page-select" onchange="navigate_to_page()"> + <option value=""></option> + %for page_index in range(1, num_pages + 1): + %if page_index == cur_page_num: + continue + %else: + <% args = { "page" : page_index } %> + <option value='${page_index}'>Page ${page_index}</option> + %endif + %endfor + </select> + ## Show all link. + <% args = { "page" : "all" } %> + <span>| <a href="${url( args )}">Show all histories on one page</a></span> + + </td> </tr> %endif

1 0

[hg] galaxy 2828: Bug fixed in \"compare two queries\" tool. Spe...
by Greg Von Kuster 06 Oct '09

06 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/ca273cc16b60 changeset: 2828:ca273cc16b60 user: guru date: Mon Oct 05 16:33:32 2009 -0400 description: Bug fixed in \"compare two queries\" tool. Specified tab as delimiter for sorting and joining. 2 file(s) affected in this change: tools/filters/compare.xml tools/filters/joinWrapper.py diffs (35 lines): diff -r f648d6dec544 -r ca273cc16b60 tools/filters/compare.xml --- a/tools/filters/compare.xml Mon Oct 05 15:07:20 2009 -0400 +++ b/tools/filters/compare.xml Mon Oct 05 16:33:32 2009 -0400 @@ -1,4 +1,4 @@ -<tool id="comp1" name="Compare two Queries"> +<tool id="comp1" name="Compare two Queries" version="1.0.1"> <description>to find common or distinct rows</description> <command interpreter="python">joinWrapper.py $input1 $input2 $field1 $field2 $mode $out_file1</command> <inputs> diff -r f648d6dec544 -r ca273cc16b60 tools/filters/joinWrapper.py --- a/tools/filters/joinWrapper.py Mon Oct 05 15:07:20 2009 -0400 +++ b/tools/filters/joinWrapper.py Mon Oct 05 16:33:32 2009 -0400 @@ -22,8 +22,8 @@ try: #Sort the two files based on specified fields - os.system("sort -k %d -o %s %s" %(field1, tmpfile1.name, infile1)) - os.system("sort -k %d -o %s %s" %(field2, tmpfile2.name, infile2)) + os.system("sort -t $'\t' -k %d -o %s %s" %(field1, tmpfile1.name, infile1)) + os.system("sort -t $'\t' -k %d -o %s %s" %(field2, tmpfile2.name, infile2)) except Exception, exc: stop_err( 'Initialization error -> %s' %str(exc) ) @@ -40,9 +40,9 @@ break if mode == "V": - cmdline = 'join -v 1 -o %s -1 %d -2 %d %s %s | tr " " "\t" > %s' %(option, field1, field2, tmpfile1.name, tmpfile2.name, outfile) + cmdline = "join -t $'\t' -v 1 -o %s -1 %d -2 %d %s %s > %s" %(option, field1, field2, tmpfile1.name, tmpfile2.name, outfile) else: - cmdline = 'join -o %s -1 %d -2 %d %s %s | tr " " "\t" > %s' %(option, field1, field2, tmpfile1.name, tmpfile2.name, outfile) + cmdline = "join -t $'\t' -o %s -1 %d -2 %d %s %s > %s" %(option, field1, field2, tmpfile1.name, tmpfile2.name, outfile) try: os.system(cmdline)

1 0