- galaxy-dev - lists.galaxyproject.org

[hg] galaxy 2810: Fixed trimmer test
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/6eb4f3017fbe changeset: 2810:6eb4f3017fbe user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Thu Oct 01 09:09:40 2009 -0400 description: Fixed trimmer test 1 file(s) affected in this change: tools/filters/trimmer.xml diffs (12 lines): diff -r 3bd7acfe34cc -r 6eb4f3017fbe tools/filters/trimmer.xml --- a/tools/filters/trimmer.xml Wed Sep 30 15:35:05 2009 -0400 +++ b/tools/filters/trimmer.xml Thu Oct 01 09:09:40 2009 -0400 @@ -1,7 +1,7 @@ <tool id="trimmer" name="Trim" version="0.0.1"> <description>leading or trailing characters</description> <command interpreter="python"> - chopper.py -a -f $input1 -c $col -s $start -e $end -i $ignore $fastq > $out_file1 + trimmer.py -a -f $input1 -c $col -s $start -e $end -i $ignore $fastq > $out_file1 </command> <inputs> <param format="tabular,text" name="input1" type="data" label="this dataset"/>

1 0

[hg] galaxy 2812: Add a new option, 'allow_library_path_paste' t...
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/d31ab50dc8e0 changeset: 2812:d31ab50dc8e0 user: Nate Coraor <nate(a)bx.psu.edu> date: Thu Oct 01 12:35:41 2009 -0400 description: Add a new option, 'allow_library_path_paste' that adds a new upload page ("Upload files from filesystem paths") to the admin-side library upload pages. This form contains a textarea that allows Galaxy admins to paste any number of filesystem paths (files or directories) from which Galaxy will import library datasets, saving the directory structure (if desired). Since such ability allows admins access to any file on the Galaxy server which is readable by Galaxy's system user, this option is disabled by default, and system administrators should take care in assigning Galaxy administrators when this feature is enabled. Controls on what files are accessible to this tool based on ownership or other properties can be added at a later date if there is sufficient interest for such features. This commit also includes a checkbox on the "Upload directory of files" page (as well as the new "Upload files from filesystem paths" page above) that will prevent Galaxy from copying data to its files directory (by default, 'database/files/'). This is useful for large library datasets that live in their own managed locations on the filesystem, this will prevent the existence of duplicate copies of datasets (but means administrators must take care to manage data - moving or removing the data from its Galaxy-external location will render these datasets invalid within Galaxy). One unique feature to be aware of: when using the "Copy data into Galaxy?" checkbox on the "Upload directory of files" page, any symbolic links encountered in the chosen import directory will be made absolute and dereferenced ONCE. This allows administrators to link large datasets to the import directory, rather than having to make full copies, while being able to delete such links after importing. Only the first symlink (the one in the import directory itself) is dereferenced; all others remain. See the following for an example: library_import_dir = /galaxy/import % ls -lR /galaxy/import /galaxy/import: total 6 drwxr-xr-x 2 nate nate 512 Oct 1 11:31 link/ /galaxy/import/link: total 10 lrwxrwxrwx 1 nate nate 71 Oct 1 10:38 1.bed -> ../../../home/nate/galaxy/test-data/1.bed lrwxrwxrwx 1 nate nate 60 Oct 1 10:38 2.bed -> /home/nate/galaxy/test-data/2.bed lrwxrwxrwx 1 nate nate 11 Oct 1 10:38 3.bed -> ../../3.bed lrwxrwxrwx 1 nate nate 35 Oct 1 11:30 4.bed -> ../../galaxy_symlink/test-data/4.bed lrwxrwxrwx 1 nate nate 41 Oct 1 11:31 5.bed -> /galaxy/galaxy_symlink/test-data/5.bed % ls -l /galaxy/3.bed lrwxrwxrwx 1 nate nate 60 Oct 1 10:39 /galaxy/3.bed -> /home/nate/galaxy/test-data/3.bed % ls -l /galaxy/galaxy_symlink lrwxrwxrwx 1 nate nate 44 Oct 1 11:30 /galaxy/galaxy_symlink -> /home/nate/galaxy/ In this example, 1.bed is a relative symbolic link to the real 1.bed. 2.bed is an absolute symlink to the real 2.bed. 3.bed is a relative symlink to ../../3.bed, aka /galaxy/3.bed, which itself is a symlink to the real 3.bed. 4.bed is a relative symlink which follows another symlink (/galaxy/galaxy_symlink) to the real 4.bed. 5.bed is an absolute symlink in the same fashion as 4.bed If the 'link' server directory is chosen on the "Upload directory of files" page, and "Copy data into Galaxy?" is checked "No", the following files will be referenced by Galaxy: /home/nate/galaxy/test-data/1.bed /home/nate/galaxy/test-data/2.bed /galaxy/3.bed /galaxy/galaxy_symlink/test-data/4.bed /galaxy/galaxy_symlink/test-data/5.bed The Galaxy administrator may now safely delete /galaxy/import/link, but should take care not to remove the referenced symbolic links (/galaxy/3.bed, /galaxy/galaxy_symlink). Not all symbolic links are dereferenced because it is assumed that if an administrator links to a path in the import directory which itself is (or contains) links, that is the preferred path for accessing the data. 10 file(s) affected in this change: lib/galaxy/config.py lib/galaxy/tools/actions/upload_common.py lib/galaxy/util/__init__.py lib/galaxy/web/controllers/library_common.py templates/admin/library/browse_library.mako templates/admin/library/upload.mako templates/library/browse_library.mako templates/library/library_dataset_common.mako tools/data_source/upload.py universe_wsgi.ini.sample diffs (377 lines): diff -r 2364764e3604 -r d31ab50dc8e0 lib/galaxy/config.py --- a/lib/galaxy/config.py Thu Oct 01 11:39:55 2009 -0400 +++ b/lib/galaxy/config.py Thu Oct 01 12:35:41 2009 -0400 @@ -87,6 +87,7 @@ self.user_library_import_dir = kwargs.get( 'user_library_import_dir', None ) if self.user_library_import_dir is not None and not os.path.exists( self.user_library_import_dir ): raise ConfigurationError( "user_library_import_dir specified in config (%s) does not exist" % self.user_library_import_dir ) + self.allow_library_path_paste = kwargs.get( 'allow_library_path_paste', False ) # Configuration options for taking advantage of nginx features self.nginx_x_accel_redirect_base = kwargs.get( 'nginx_x_accel_redirect_base', False ) self.nginx_upload_store = kwargs.get( 'nginx_upload_store', False ) diff -r 2364764e3604 -r d31ab50dc8e0 lib/galaxy/tools/actions/upload_common.py --- a/lib/galaxy/tools/actions/upload_common.py Thu Oct 01 11:39:55 2009 -0400 +++ b/lib/galaxy/tools/actions/upload_common.py Thu Oct 01 12:35:41 2009 -0400 @@ -3,6 +3,7 @@ from galaxy import datatypes, util from galaxy.datatypes import sniff from galaxy.util.json import to_json_string +from galaxy.model.orm import eagerload_all import logging log = logging.getLogger( __name__ ) @@ -127,12 +128,29 @@ or trans.user.email in trans.app.config.get( "admin_users", "" ).split( "," ) ): # This doesn't have to be pretty - the only time this should happen is if someone's being malicious. raise Exception( "User is not authorized to add datasets to this library." ) + folder = library_bunch.folder + if uploaded_dataset.get( 'in_folder', False ): + # Create subfolders if desired + for name in uploaded_dataset.in_folder.split( os.path.sep ): + folder.refresh() + matches = filter( lambda x: x.name == name, active_folders( trans, folder ) ) + if matches: + log.debug( 'DEBUGDEBUG: In %s, found a folder name match: %s:%s' % ( folder.name, matches[0].id, matches[0].name ) ) + folder = matches[0] + else: + new_folder = trans.app.model.LibraryFolder( name=name, description='Automatically created by upload tool' ) + new_folder.genome_build = util.dbnames.default_value + folder.add_folder( new_folder ) + new_folder.flush() + trans.app.security_agent.copy_library_permissions( folder, new_folder ) + log.debug( 'DEBUGDEBUG: In %s, created a new folder: %s:%s' % ( folder.name, new_folder.id, new_folder.name ) ) + folder = new_folder if library_bunch.replace_dataset: ld = library_bunch.replace_dataset else: - ld = trans.app.model.LibraryDataset( folder=library_bunch.folder, name=uploaded_dataset.name ) + ld = trans.app.model.LibraryDataset( folder=folder, name=uploaded_dataset.name ) ld.flush() - trans.app.security_agent.copy_library_permissions( library_bunch.folder, ld ) + trans.app.security_agent.copy_library_permissions( folder, ld ) ldda = trans.app.model.LibraryDatasetDatasetAssociation( name = uploaded_dataset.name, extension = uploaded_dataset.file_type, dbkey = uploaded_dataset.dbkey, @@ -153,8 +171,8 @@ else: # Copy the current user's DefaultUserPermissions to the new LibraryDatasetDatasetAssociation.dataset trans.app.security_agent.set_all_dataset_permissions( ldda.dataset, trans.app.security_agent.user_get_default_permissions( trans.user ) ) - library_bunch.folder.add_library_dataset( ld, genome_build=uploaded_dataset.dbkey ) - library_bunch.folder.flush() + folder.add_library_dataset( ld, genome_build=uploaded_dataset.dbkey ) + folder.flush() ld.library_dataset_dataset_association_id = ldda.id ld.flush() # Handle template included in the upload form, if any @@ -230,6 +248,10 @@ is_binary = uploaded_dataset.datatype.is_binary except: is_binary = None + try: + link_data_only = uploaded_dataset.link_data_only + except: + link_data_only = False json = dict( file_type = uploaded_dataset.file_type, ext = uploaded_dataset.ext, name = uploaded_dataset.name, @@ -237,6 +259,7 @@ dbkey = uploaded_dataset.dbkey, type = uploaded_dataset.type, is_binary = is_binary, + link_data_only = link_data_only, space_to_tab = uploaded_dataset.space_to_tab, path = uploaded_dataset.path ) json_file.write( to_json_string( json ) + '\n' ) @@ -276,3 +299,13 @@ trans.app.job_queue.put( job.id, tool ) trans.log_event( "Added job to the job queue, id: %s" % str(job.id), tool_id=job.tool_id ) return dict( [ ( 'output%i' % i, v ) for i, v in enumerate( data_list ) ] ) + +def active_folders( trans, folder ): + # Stolen from galaxy.web.controllers.library_common (importing from which causes a circular issues). + # Much faster way of retrieving all active sub-folders within a given folder than the + # performance of the mapper. This query also eagerloads the permissions on each folder. + return trans.sa_session.query( trans.app.model.LibraryFolder ) \ + .filter_by( parent=folder, deleted=False ) \ + .options( eagerload_all( "actions" ) ) \ + .order_by( trans.app.model.LibraryFolder.table.c.name ) \ + .all() diff -r 2364764e3604 -r d31ab50dc8e0 lib/galaxy/util/__init__.py --- a/lib/galaxy/util/__init__.py Thu Oct 01 11:39:55 2009 -0400 +++ b/lib/galaxy/util/__init__.py Thu Oct 01 12:35:41 2009 -0400 @@ -178,7 +178,7 @@ # better solution I think is to more responsibility for # sanitizing into the tool parameters themselves so that # different parameters can be sanitized in different ways. - NEVER_SANITIZE = ['file_data', 'url_paste', 'URL'] + NEVER_SANITIZE = ['file_data', 'url_paste', 'URL', 'filesystem_paths'] def __init__( self, params, safe=True, sanitize=True, tool=None ): if safe: diff -r 2364764e3604 -r d31ab50dc8e0 lib/galaxy/web/controllers/library_common.py --- a/lib/galaxy/web/controllers/library_common.py Thu Oct 01 11:39:55 2009 -0400 +++ b/lib/galaxy/web/controllers/library_common.py Thu Oct 01 12:35:41 2009 -0400 @@ -81,7 +81,9 @@ tool_params = upload_common.persist_uploads( tool_params ) uploaded_datasets = upload_common.get_uploaded_datasets( trans, tool_params, precreated_datasets, dataset_upload_inputs, library_bunch=library_bunch ) elif upload_option == 'upload_directory': - uploaded_datasets = self.get_server_dir_uploaded_datasets( trans, params, full_dir, import_dir_desc, library_bunch, err_redirect, msg ) + uploaded_datasets, err_redirect, msg = self.get_server_dir_uploaded_datasets( trans, params, full_dir, import_dir_desc, library_bunch, err_redirect, msg ) + elif upload_option == 'upload_paths': + uploaded_datasets, err_redirect, msg = self.get_path_paste_uploaded_datasets( trans, params, library_bunch, err_redirect, msg ) upload_common.cleanup_unused_precreated_datasets( precreated_datasets ) if upload_option == 'upload_file' and not uploaded_datasets: msg = 'Select a file, enter a URL or enter text' @@ -98,37 +100,86 @@ json_file_path = upload_common.create_paramfile( uploaded_datasets ) data_list = [ ud.data for ud in uploaded_datasets ] return upload_common.create_job( trans, tool_params, tool, json_file_path, data_list, folder=library_bunch.folder ) + def make_library_uploaded_dataset( self, trans, params, name, path, type, library_bunch, in_folder=None ): + library_bunch.replace_dataset = None # not valid for these types of upload + uploaded_dataset = util.bunch.Bunch() + uploaded_dataset.name = name + uploaded_dataset.path = path + uploaded_dataset.type = type + uploaded_dataset.ext = None + uploaded_dataset.file_type = params.file_type + uploaded_dataset.dbkey = params.dbkey + uploaded_dataset.space_to_tab = params.space_to_tab + if in_folder: + uploaded_dataset.in_folder = in_folder + uploaded_dataset.data = upload_common.new_upload( trans, uploaded_dataset, library_bunch ) + if params.get( 'link_data_only', False ): + uploaded_dataset.link_data_only = True + uploaded_dataset.data.file_name = os.path.abspath( path ) + uploaded_dataset.data.flush() + return uploaded_dataset def get_server_dir_uploaded_datasets( self, trans, params, full_dir, import_dir_desc, library_bunch, err_redirect, msg ): files = [] try: for entry in os.listdir( full_dir ): # Only import regular files - if os.path.isfile( os.path.join( full_dir, entry ) ): - files.append( entry ) + path = os.path.join( full_dir, entry ) + if os.path.islink( path ) and os.path.isfile( path ) and params.get( 'link_data_only', False ): + # If we're linking instead of copying, link the file the link points to, not the link itself. + link_path = os.readlink( path ) + if os.path.isabs( link_path ): + path = link_path + else: + path = os.path.abspath( os.path.join( os.path.dirname( path ), link_path ) ) + if os.path.isfile( path ): + files.append( path ) except Exception, e: msg = "Unable to get file list for configured %s, error: %s" % ( import_dir_desc, str( e ) ) err_redirect = True - return None + return None, err_redirect, msg if not files: msg = "The directory '%s' contains no valid files" % full_dir err_redirect = True - return None + return None, err_redirect, msg uploaded_datasets = [] for file in files: - library_bunch.replace_dataset = None - uploaded_dataset = util.bunch.Bunch() - uploaded_dataset.path = os.path.join( full_dir, file ) - if not os.path.isfile( uploaded_dataset.path ): + name = os.path.basename( file ) + uploaded_datasets.append( self.make_library_uploaded_dataset( trans, params, name, file, 'server_dir', library_bunch ) ) + return uploaded_datasets, None, None + def get_path_paste_uploaded_datasets( self, trans, params, library_bunch, err_redirect, msg ): + if params.get( 'filesystem_paths', '' ) == '': + msg = "No paths entered in the upload form" + err_redirect = True + return None, err_redirect, msg + preserve_dirs = True + if params.get( 'dont_preserve_dirs', False ): + preserve_dirs = False + # locate files + bad_paths = [] + uploaded_datasets = [] + for line in [ l.strip() for l in params.filesystem_paths.splitlines() if l.strip() ]: + path = os.path.abspath( line ) + if not os.path.exists( path ): + bad_paths.append( path ) continue - uploaded_dataset.type = 'server_dir' - uploaded_dataset.name = file - uploaded_dataset.ext = None - uploaded_dataset.file_type = params.file_type - uploaded_dataset.dbkey = params.dbkey - uploaded_dataset.space_to_tab = params.space_to_tab - uploaded_dataset.data = upload_common.new_upload( trans, uploaded_dataset, library_bunch ) - uploaded_datasets.append( uploaded_dataset ) - return uploaded_datasets + # don't bother processing if we're just going to return an error + if not bad_paths: + if os.path.isfile( path ): + name = os.path.basename( path ) + uploaded_datasets.append( self.make_library_uploaded_dataset( trans, params, name, path, 'path_paste', library_bunch ) ) + for basedir, dirs, files in os.walk( line ): + for file in files: + file_path = os.path.abspath( os.path.join( basedir, file ) ) + if preserve_dirs: + in_folder = os.path.dirname( file_path.replace( path, '', 1 ).lstrip( '/' ) ) + else: + in_folder = None + uploaded_datasets.append( self.make_library_uploaded_dataset( trans, params, file, file_path, 'path_paste', library_bunch, in_folder ) ) + if bad_paths: + msg = "Invalid paths:<br><ul><li>%s</li></ul>" % "</li><li>".join( bad_paths ) + err_redirect = True + return None, err_redirect, msg + return uploaded_datasets, None, None @web.expose def info_template( self, trans, cntrller, library_id, response_action='library', obj_id=None, folder_id=None, ldda_id=None, **kwd ): # Only adding a new templAte to a library or folder is currently allowed. Editing an existing template is diff -r 2364764e3604 -r d31ab50dc8e0 templates/admin/library/browse_library.mako --- a/templates/admin/library/browse_library.mako Thu Oct 01 11:39:55 2009 -0400 +++ b/templates/admin/library/browse_library.mako Thu Oct 01 12:35:41 2009 -0400 @@ -73,7 +73,7 @@ // Make ajax call $.ajax( { type: "POST", - url: "${h.url_for( controller='library_dataset', action='library_item_updates' )}", + url: "${h.url_for( controller='library_common', action='library_item_updates' )}", dataType: "json", data: { ids: ids.join( "," ), states: states.join( "," ) }, success : function ( data ) { diff -r 2364764e3604 -r d31ab50dc8e0 templates/admin/library/upload.mako --- a/templates/admin/library/upload.mako Thu Oct 01 11:39:55 2009 -0400 +++ b/templates/admin/library/upload.mako Thu Oct 01 12:35:41 2009 -0400 @@ -18,6 +18,9 @@ %if trans.app.config.library_import_dir and os.path.exists( trans.app.config.library_import_dir ): <a class="action-button" href="${h.url_for( controller='library_admin', action='upload_library_dataset', library_id=library_id, folder_id=folder_id, replace_id=replace_id, upload_option='upload_directory' )}">Upload directory of files</a> %endif + %if trans.app.config.allow_library_path_paste: + <a class="action-button" href="${h.url_for( controller='library_admin', action='upload_library_dataset', library_id=library_id, folder_id=folder_id, replace_id=replace_id, upload_option='upload_paths' )}">Upload files from filesystem paths</a> + %endif <a class="action-button" href="${h.url_for( controller='library_admin', action='upload_library_dataset', library_id=library_id, folder_id=folder_id, replace_id=replace_id, upload_option='import_from_history' )}">Import datasets from your current history</a> </div> <br/><br/> diff -r 2364764e3604 -r d31ab50dc8e0 templates/library/browse_library.mako --- a/templates/library/browse_library.mako Thu Oct 01 11:39:55 2009 -0400 +++ b/templates/library/browse_library.mako Thu Oct 01 12:35:41 2009 -0400 @@ -105,7 +105,7 @@ // Make ajax call $.ajax( { type: "POST", - url: "${h.url_for( controller='library_dataset', action='library_item_updates' )}", + url: "${h.url_for( controller='library_common', action='library_item_updates' )}", dataType: "json", data: { ids: ids.join( "," ), states: states.join( "," ) }, success : function ( data ) { diff -r 2364764e3604 -r d31ab50dc8e0 templates/library/library_dataset_common.mako --- a/templates/library/library_dataset_common.mako Thu Oct 01 11:39:55 2009 -0400 +++ b/templates/library/library_dataset_common.mako Thu Oct 01 12:35:41 2009 -0400 @@ -1,11 +1,13 @@ <%def name="render_upload_form( controller, upload_option, action, library_id, folder_id, replace_dataset, file_formats, dbkeys, roles, history )"> <% import os, os.path %> - %if upload_option in [ 'upload_file', 'upload_directory' ]: + %if upload_option in [ 'upload_file', 'upload_directory', 'upload_paths' ]: <div class="toolForm" id="upload_library_dataset"> - %if upload_option == 'upload_file': + %if upload_option == 'upload_directory': + <div class="toolFormTitle">Upload a directory of files</div> + %elif upload_option == 'upload_paths': + <div class="toolFormTitle">Upload files from filesystem paths</div> + %else: <div class="toolFormTitle">Upload files</div> - %else: - <div class="toolFormTitle">Upload a directory of files</div> %endif <div class="toolFormBody"> <form name="upload_library_dataset" action="${action}" enctype="multipart/form-data" method="post"> @@ -103,6 +105,44 @@ %endif </div> <div style="clear: both"></div> + </div> + %elif upload_option == 'upload_paths': + <div class="form-row"> + <label>Paths to upload</label> + <div class="form-row-input"> + <textarea name="filesystem_paths" rows="10" cols="35"></textarea> + </div> + <div class="toolParamHelp" style="clear: both;"> + Upload all files pasted in the box. The (recursive) contents of any pasted directories will be added as well. + </div> + </div> + <div class="form-row"> + <label>Preserve directory structure?</label> + <div class="form-row-input"> + <input type="checkbox" name="dont_preserve_dirs" value="No"/>No + </div> + <div class="toolParamHelp" style="clear: both;"> + If checked, all files in subdirectories on the filesystem will be placed at the top level of the folder, instead of into subfolders. + </div> + </div> + %endif + %if upload_option in ( 'upload_directory', 'upload_paths' ): + <div class="form-row"> + <label>Copy data into Galaxy?</label> + <div class="form-row-input"> + <input type="checkbox" name="link_data_only" value="No"/>No + </div> + <div class="toolParamHelp" style="clear: both;"> + Normally data uploaded with this tool is copied into Galaxy's "files" directory + so any later changes to the data will not affect Galaxy. However, this may not + be desired (especially for large NGS datasets), so use of this option will + force Galaxy to always read the data from its original path. + %if upload_option == 'upload_directory': + Any symlinks encountered in the upload directory will be dereferenced once - + that is, Galaxy will point directly to the file that is linked, but no other + symlinks further down the line will be dereferenced. + %endif + </div> </div> %endif <div class="form-row"> diff -r 2364764e3604 -r d31ab50dc8e0 tools/data_source/upload.py --- a/tools/data_source/upload.py Thu Oct 01 11:39:55 2009 -0400 +++ b/tools/data_source/upload.py Thu Oct 01 12:35:41 2009 -0400 @@ -238,7 +238,9 @@ if ext == 'auto': ext = 'data' # Move the dataset to its "real" path - if dataset.type == 'server_dir': + if dataset.get( 'link_data_only', False ): + pass # data will remain in place + elif dataset.type in ( 'server_dir', 'path_paste' ): shutil.copy( dataset.path, output_path ) else: shutil.move( dataset.path, output_path ) diff -r 2364764e3604 -r d31ab50dc8e0 universe_wsgi.ini.sample --- a/universe_wsgi.ini.sample Thu Oct 01 11:39:55 2009 -0400 +++ b/universe_wsgi.ini.sample Thu Oct 01 12:35:41 2009 -0400 @@ -60,13 +60,22 @@ # Galaxy session security id_secret = changethisinproductiontoo -# Directories of files contained in the following directory can be uploaded to a library from the Admin view +# Directories of files contained in the following directory can be uploaded to +# a library from the Admin view #library_import_dir = /var/opt/galaxy/import -# The following can be configured to allow non-admin users to upload a directory of files. The -# configured directory must contain sub-directories named the same as the non-admin user's Galaxy -# login ( email ). The non-admin user is restricted to uploading files or sub-directories of files -# contained in their directory. -# user_library_import_dir = /var/opt/galaxy/import/users + +# The following can be configured to allow non-admin users to upload a +# directory of files. The configured directory must contain sub-directories +# named the same as the non-admin user's Galaxy login ( email ). The non-admin +# user is restricted to uploading files or sub-directories of files contained +# in their directory. +#user_library_import_dir = /var/opt/galaxy/import/users + +# The admin library upload tool may contain a box allowing admins to paste +# filesystem paths to files and directories to add to a library. Set to True +# to enable. Please note the security implication that this will give Galaxy +# Admins access to anything your Galaxy user has access to. +#allow_library_path_paste = False # path to sendmail sendmail_path = /usr/sbin/sendmail

1 0

[hg] galaxy 2811: Fix spelling of Successfully when running a wo...
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/2364764e3604 changeset: 2811:2364764e3604 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Thu Oct 01 11:39:55 2009 -0400 description: Fix spelling of Successfully when running a workflow. 1 file(s) affected in this change: templates/workflow/run_complete.mako diffs (19 lines): diff -r 6eb4f3017fbe -r 2364764e3604 templates/workflow/run_complete.mako --- a/templates/workflow/run_complete.mako Thu Oct 01 09:09:40 2009 -0400 +++ b/templates/workflow/run_complete.mako Thu Oct 01 11:39:55 2009 -0400 @@ -14,7 +14,7 @@ <body> <div class="donemessage"> <p> - Sucesfully ran workflow "${workflow.name}", the following datasets have + Successfully ran workflow "${workflow.name}", the following datasets have been added to the queue. </p> @@ -28,4 +28,4 @@ </div> </body> -</html> \ No newline at end of file +</html>

1 0

[hg] galaxy 2809: Allow for a specified target for display links...
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/3bd7acfe34cc changeset: 2809:3bd7acfe34cc user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Sep 30 15:35:05 2009 -0400 description: Allow for a specified target for display links, rahter than forcing it to be _blank - this will allow external tools like GeneTrack to run inside the Galaxy frame set. 3 file(s) affected in this change: lib/galaxy/datatypes/data.py lib/galaxy/datatypes/tracks.py templates/root/history_common.mako diffs (64 lines): diff -r 9cc7a17e0ac7 -r 3bd7acfe34cc lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py Wed Sep 30 14:47:17 2009 -0400 +++ b/lib/galaxy/datatypes/data.py Wed Sep 30 15:35:05 2009 -0400 @@ -171,11 +171,11 @@ def get_mime(self): """Returns the mime type of the datatype""" return 'application/octet-stream' - def add_display_app (self, app_id, label, file_function, links_function ): + def add_display_app ( self, app_id, label, file_function, links_function ): """ Adds a display app to the datatype. app_id is a unique id - label is the primary display label, ie display at 'UCSC' + label is the primary display label, e.g., display at 'UCSC' file_function is a string containing the name of the function that returns a properly formatted display links_function is a string containing the name of the function that returns a list of (link_name,link) """ @@ -205,7 +205,7 @@ except: log.exception('Function %s is referred to in datatype %s for displaying as type %s, but is not accessible' % (self.supported_display_apps[type]['file_function'], self.__class__.__name__, type) ) return "This display type (%s) is not implemented for this datatype (%s)." % ( type, dataset.ext) - def get_display_links(self, dataset, type, app, base_url, **kwd): + def get_display_links( self, dataset, type, app, base_url, target_frame='_blank', **kwd ): """ Returns a list of tuples of (name, link) for a particular display type. No check on 'access' permissions is done here - if you can view the dataset, you can also save it @@ -214,7 +214,7 @@ """ try: if type in self.get_display_types(): - return getattr (self, self.supported_display_apps[type]['links_function']) (dataset, type, app, base_url, **kwd) + return target_frame, getattr ( self, self.supported_display_apps[type]['links_function'] ) ( dataset, type, app, base_url, **kwd ) except: log.exception( 'Function %s is referred to in datatype %s for generating links for type %s, but is not accessible' \ % ( self.supported_display_apps[type]['links_function'], self.__class__.__name__, type ) ) diff -r 9cc7a17e0ac7 -r 3bd7acfe34cc lib/galaxy/datatypes/tracks.py --- a/lib/galaxy/datatypes/tracks.py Wed Sep 30 14:47:17 2009 -0400 +++ b/lib/galaxy/datatypes/tracks.py Wed Sep 30 15:35:05 2009 -0400 @@ -27,6 +27,8 @@ def __init__(self, **kwargs): super( GeneTrack, self ).__init__( **kwargs ) self.add_display_app( 'genetrack', 'View in GeneTrack', '', 'genetrack_link' ) + def get_display_links( self, dataset, type, app, base_url, target_frame='galaxy_main', **kwd ): + return data.Binary.get_display_links( self, dataset, type, app, base_url, target_frame=target_frame, **kwd ) def genetrack_link( self, hda, type, app, base_url ): ret_val = [] if hda.has_data: diff -r 9cc7a17e0ac7 -r 3bd7acfe34cc templates/root/history_common.mako --- a/templates/root/history_common.mako Wed Sep 30 14:47:17 2009 -0400 +++ b/templates/root/history_common.mako Wed Sep 30 15:35:05 2009 -0400 @@ -83,11 +83,11 @@ <a href="${h.url_for( action='display', id=data.id, tofile='yes', toext=data.ext )}" target="_blank">save</a> | <a href="${h.url_for( controller='tool_runner', action='rerun', id=data.id )}" target="galaxy_main">rerun</a> %for display_app in data.datatype.get_display_types(): - <% display_links = data.datatype.get_display_links( data, display_app, app, request.base ) %> + <% target_frame, display_links = data.datatype.get_display_links( data, display_app, app, request.base ) %> %if len( display_links ) > 0: | ${data.datatype.get_display_label(display_app)} %for display_name, display_link in display_links: - <a target="_blank" href="${display_link}">${_(display_name)}</a> + <a target="${target_frame}" href="${display_link}">${_(display_name)}</a> %endfor %endif %endfor

1 0

[hg] galaxy 2808: If you populate a history before logging in, t...
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/9cc7a17e0ac7 changeset: 2808:9cc7a17e0ac7 user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Sep 30 14:47:17 2009 -0400 description: If you populate a history before logging in, then login, your current history will be the one you populated before logging in - fixes ticket number 163. Also added additional functional tests to cover scenario. 2 file(s) affected in this change: lib/galaxy/web/framework/__init__.py test/functional/test_history_functions.py diffs (65 lines): diff -r aafecc059ba6 -r 9cc7a17e0ac7 lib/galaxy/web/framework/__init__.py --- a/lib/galaxy/web/framework/__init__.py Wed Sep 30 14:11:01 2009 -0400 +++ b/lib/galaxy/web/framework/__init__.py Wed Sep 30 14:47:17 2009 -0400 @@ -404,18 +404,17 @@ except: users_last_session = None last_accessed = False - if users_last_session and users_last_session.current_history: + if prev_galaxy_session.current_history and prev_galaxy_session.current_history.datasets: + if prev_galaxy_session.current_history.user is None or prev_galaxy_session.current_history.user == user: + # If the previous galaxy session had a history, associate it with the new + # session, but only if it didn't belong to a different user. + history = prev_galaxy_session.current_history + elif self.galaxy_session.current_history: + history = self.galaxy_session.current_history + if not history and users_last_session and users_last_session.current_history: history = users_last_session.current_history - if not history: - if prev_galaxy_session.current_history: - if prev_galaxy_session.current_history.user is None or prev_galaxy_session.current_history.user == user: - # If the previous galaxy session had a history, associate it with the new - # session, but only if it didn't belong to a different user. - history = prev_galaxy_session.current_history - elif self.galaxy_session.current_history: - history = self.galaxy_session.current_history - else: - history = self.get_history( create=True ) + elif not history: + history = self.get_history( create=True ) if history not in self.galaxy_session.histories: self.galaxy_session.add_history( history ) if history.user is None: diff -r aafecc059ba6 -r 9cc7a17e0ac7 test/functional/test_history_functions.py --- a/test/functional/test_history_functions.py Wed Sep 30 14:11:01 2009 -0400 +++ b/test/functional/test_history_functions.py Wed Sep 30 14:47:17 2009 -0400 @@ -9,11 +9,28 @@ """Testing history behavior between logout and login""" self.logout() self.history_options() - # Make sure we have created the following 4 accounts + # Create a new, empty history named anonymous + name = 'anonymous' + self.new_history( name=name ) + global anonymous_history + anonymous_history = galaxy.model.History \ + .filter( and_( galaxy.model.History.table.c.deleted==False, + galaxy.model.History.table.c.name==name ) ) \ + .order_by( desc( galaxy.model.History.table.c.create_time ) ) \ + .first() + assert anonymous_history is not None, "Problem retrieving anonymous_history from database" + # Upload a dataset to anonymous_history so it will be set as the current history after login + self.upload_file( '1.bed', dbkey='hg18' ) self.login( email='test1(a)bx.psu.edu' ) global regular_user1 regular_user1 = galaxy.model.User.filter( galaxy.model.User.table.c.email=='test1(a)bx.psu.edu' ).first() assert regular_user1 is not None, 'Problem retrieving user with email "test1(a)bx.psu.edu" from the database' + # Current history should be anonymous_history + self.check_history_for_string( name ) + self.logout() + # Login as the same user again to ensure anonymous_history is still the current history + self.login( email=regular_user1.email ) + self.check_history_for_string( name ) self.logout() self.login( email='test2(a)bx.psu.edu' ) global regular_user2

1 0

[hg] galaxy 2806: Fix reporting number of regions processed in M...
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/268ab95fc8a7 changeset: 2806:268ab95fc8a7 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Wed Sep 30 14:06:06 2009 -0400 description: Fix reporting number of regions processed in MAF Stats tool. 1 file(s) affected in this change: tools/maf/maf_stats.py diffs (22 lines): diff -r c63a444cd7ba -r 268ab95fc8a7 tools/maf/maf_stats.py --- a/tools/maf/maf_stats.py Wed Sep 30 11:40:09 2009 -0400 +++ b/tools/maf/maf_stats.py Wed Sep 30 14:06:06 2009 -0400 @@ -54,7 +54,7 @@ out = open(output_filename, 'w') - num_region = 0 + num_region = None species_summary = {} total_length = 0 #loop through interval file @@ -97,7 +97,8 @@ for spec in species_summary: out.write( "%s\t%s\t%.4f\n" % ( spec, species_summary[spec], float( species_summary[spec] ) / total_length ) ) out.close() - print "%i regions were processed with a total length of %i." % ( num_region, total_length ) + if num_region is not None: + print "%i regions were processed with a total length of %i." % ( num_region + 1, total_length ) maf_utilities.remove_temp_index_file( index_filename ) if __name__ == "__main__": __main__()

1 0

[hg] galaxy 2807: Added text trimmer that can be used to groom f...
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/aafecc059ba6 changeset: 2807:aafecc059ba6 user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Wed Sep 30 14:11:01 2009 -0400 description: Added text trimmer that can be used to groom fastq data 6 file(s) affected in this change: test-data/trimmer_a_f_c0_s1_e13_i62.dat test-data/trimmer_a_f_c2_s1_e2_i62.dat test-data/trimmer_tab_delimited.dat tool_conf.xml.sample tools/filters/trimmer.py tools/filters/trimmer.xml diffs (273 lines): diff -r 268ab95fc8a7 -r aafecc059ba6 test-data/trimmer_a_f_c0_s1_e13_i62.dat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/trimmer_a_f_c0_s1_e13_i62.dat Wed Sep 30 14:11:01 2009 -0400 @@ -0,0 +1,5 @@ +12345 abcdef +67890 ghjkl g +>assa lljlj ljlj +sasas hghg hg +@dgf gfgf gfg diff -r 268ab95fc8a7 -r aafecc059ba6 test-data/trimmer_a_f_c2_s1_e2_i62.dat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/trimmer_a_f_c2_s1_e2_i62.dat Wed Sep 30 14:11:01 2009 -0400 @@ -0,0 +1,5 @@ +12345 ab xyz +67890 gh ghjt +>assa lljlj ljlj +sasas hg hghg +@dgf gf gfgf diff -r 268ab95fc8a7 -r aafecc059ba6 test-data/trimmer_tab_delimited.dat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/trimmer_tab_delimited.dat Wed Sep 30 14:11:01 2009 -0400 @@ -0,0 +1,5 @@ +12345 abcdef xyz +67890 ghjkl ghjt +>assa lljlj ljlj +sasas hghg hghg +@dgf gfgf gfgf diff -r 268ab95fc8a7 -r aafecc059ba6 tool_conf.xml.sample --- a/tool_conf.xml.sample Wed Sep 30 14:06:06 2009 -0400 +++ b/tool_conf.xml.sample Wed Sep 30 14:11:01 2009 -0400 @@ -45,6 +45,7 @@ <tool file="filters/remove_beginning.xml" /> <tool file="filters/headWrapper.xml" /> <tool file="filters/tailWrapper.xml" /> + <tool file="filters/trimmer.xml" /> </section> <section name="Filter and Sort" id="filter"> <tool file="stats/filtering.xml" /> diff -r 268ab95fc8a7 -r aafecc059ba6 tools/filters/trimmer.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/filters/trimmer.py Wed Sep 30 14:11:01 2009 -0400 @@ -0,0 +1,108 @@ +#!/usr/bin/env python + +import sys +import optparse + +def stop_err( msg ): + sys.stderr.write( msg ) + sys.exit() + +def main(): + usage = """%prog [options] + +options (listed below) default to 'None' if omitted + """ + parser = optparse.OptionParser(usage=usage) + + parser.add_option( + '-a','--ascii', + dest='ascii', + action='store_true', + default = False, + help='Use ascii codes to defined ignored beginnings instead of raw characters') + + parser.add_option( + '-q','--fastq', + dest='fastq', + action='store_true', + default = False, + help='The input data in fastq format. It selected the script skips every even line since they contain sequence ids') + + parser.add_option( + '-i','--ignore', + dest='ignore', + help='A comma separated list on ignored beginnings (e.g., ">,@"), or its ascii codes (e.g., "60,42") if option -a is enabled') + + parser.add_option( + '-s','--start', + dest='start', + default = '0', + help='Trim from beginning to here (1-based)') + + parser.add_option( + '-e','--end', + dest='end', + default = '0', + help='Trim from here to the ned (1-based)') + + parser.add_option( + '-f','--file', + dest='input_txt', + default = False, + help='Name of file to be chopped. STDIN is default') + + parser.add_option( + '-c','--column', + dest='col', + default = '0', + help='Column to chop. If 0 = chop the whole line') + + + options, args = parser.parse_args() + invalid_starts = [] + + if options.input_txt: + infile = open ( options.input_txt, 'r') + else: + infile = sys.stdin + + if options.ignore and options.ignore != "None": + invalid_starts = options.ignore.split(',') + + if options.ascii and options.ignore and options.ignore != "None": + for i, item in enumerate( invalid_starts ): + invalid_starts[i] = chr( int( item ) ) + + col = int( options.col ) + + for i, line in enumerate( infile ): + line = line.rstrip( '\r\n' ) + if line: + + if options.fastq and i % 2 == 0: + print line + continue + + if options.fastq and line.startswith('@'): + stop_err('Malformed fastq file: even numbered line starts with @') + + if line[0] not in invalid_starts: + if col == 0: + if int( options.end ) > 0: + line = line[ int( options.start )-1 : int( options.end ) ] + else: + line = line[ int( options.start )-1 : ] + else: + fields = line.split( '\t' ) + if col-1 > len( fields ): + stop_err('Column %d does not exist. Check input parameters\n' % col) + + if int( options.end ) > 0: + fields[col - 1] = fields[col - 1][ int( options.start )-1 : int( options.end ) ] + else: + fields[col - 1] = fields[col - 1][ int( options.start )-1 : ] + line = '\t'.join(fields) + print line + +if __name__ == "__main__": main() + diff -r 268ab95fc8a7 -r aafecc059ba6 tools/filters/trimmer.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/filters/trimmer.xml Wed Sep 30 14:11:01 2009 -0400 @@ -0,0 +1,119 @@ +<tool id="trimmer" name="Trim" version="0.0.1"> + <description>leading or trailing characters</description> + <command interpreter="python"> + chopper.py -a -f $input1 -c $col -s $start -e $end -i $ignore $fastq > $out_file1 + </command> + <inputs> + <param format="tabular,text" name="input1" type="data" label="this dataset"/> + <param name="col" type="integer" value="0" label="Trim this column only" help="0 = process entire line" /> + <param name="start" type="integer" size="10" value="1" label="Trim from the beginning to this position" help="1 = do not trim the beginning"/> + <param name="end" type="integer" size="10" value="0" label="Remove everything from this position to the end" help="0 = do not trim the end"/> + <param name="fastq" type="select" label="Is input dataset in fastq format?" help="If set to YES, the tool will not trim evenly numbered lines (0, 2, 4, etc...)"> + <option value="-q">Yes</option> + <option value="">No</option> + </param> + <param name="ignore" type="select" display="checkboxes" multiple="True" label="Ignore lines beginning with these characters" help="lines beginning with these are not trimmed"> + <option value="62">></option> + <option value="64">@</option> + <option value="43">+</option> + <option value="60"><</option> + <option value="42">*</option> + <option value="45">-</option> + <option value="61">=</option> + <option value="124">|</option> + <option value="63">?</option> + <option value="36">$</option> + <option value="46">.</option> + <option value="58">:</option> + <option value="38">&</option> + <option value="37">%</option> + <option value="94">^</option> + </param> + </inputs> + <outputs> + <data name="out_file1" format="input" metadata_source="input1"/> + </outputs> + <tests> + <test> + <param name="input1" value="trimmer_tab_delimited.dat"/> + <param name="col" value="0"/> + <param name="start" value="1"/> + <param name="end" value="13"/> + <param name="ignore" value="62"/> + <param name="fastq" value="No"/> + <output name="out_file1" file="trimmer_a_f_c0_s1_e13_i62.dat"/> + </test> + <test> + <param name="input1" value="trimmer_tab_delimited.dat"/> + <param name="col" value="2"/> + <param name="start" value="1"/> + <param name="end" value="2"/> + <param name="ignore" value="62"/> + <param name="fastq" value="No"/> + <output name="out_file1" file="trimmer_a_f_c2_s1_e2_i62.dat"/> + </test> + + </tests> + + <help> + + +**What it does** + +Trims specified number of characters from a dataset or its field (if dataset is tab-delimited). + +----- + +**Example 1** + +Trimming this dataset:: + + 1234567890 + abcdefghijk + +by setting **Trim from the beginning to this position** to *2* and **Remove everything from this position to the end** to *6* will produce:: + + 23456 + bcdef + +----- + +**Eaxmple 2** + +Trimming column 2 of this dataset:: + + bcde 12345 fghij 67890 + fghij 67890 abcde 12345 + +by setting **Trim content of this column only** to *2*, **Trim from the beginning to this position** to *2*, and **Remove everything from this position to the end** to *4* will produce:: + + abcde 234 fghij 67890 + fghij 789 abcde 12345 + +----- + +**Trimming FASTQ datasets** + +This tool can be used to trim sequences and quality strings in fastq datasets. This is done by selected *Yes* from the **Is input dataset in fastq format?** dropdown. If set to *Yes*, the tool will skip all even numbered lines (see warning below). For example, trimming last 5 bases of this dataset:: + + @081017-and-081020:1:1:1715:1759 + GGACTCAGATAGTAATCCACGCTCCTTTAAAATATC + + + II#IIIIIII$5+.(9IIIIIII$%*$G$A31I&&B + +cab done by setting **Remove everything from this position to the end** to 31:: + + @081017-and-081020:1:1:1715:1759 + GGACTCAGATAGTAATCCACGCTCCTTTAAA + + + II#IIIIIII$5+.(9IIIIIII$%*$G$A3 + +**Note** that headers are skipped. + +.. class:: warningmark + +**WARNING:** This tool will only work on properly formatted fastq datasets where (1) each read and quality string occupy one line and (2) '@' (read header) and "+" (quality header) lines are evenly numbered like in the above example. + + + </help> +</tool>

1 0

[hg] galaxy 2805: Requests functional tests: fixed a hack in use...
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/c63a444cd7ba changeset: 2805:c63a444cd7ba user: rc date: Wed Sep 30 11:40:09 2009 -0400 description: Requests functional tests: fixed a hack in user address creation 1 file(s) affected in this change: test/functional/test_forms_and_requests.py diffs (60 lines): diff -r 3e5428b298b1 -r c63a444cd7ba test/functional/test_forms_and_requests.py --- a/test/functional/test_forms_and_requests.py Wed Sep 30 10:40:08 2009 -0400 +++ b/test/functional/test_forms_and_requests.py Wed Sep 30 11:40:09 2009 -0400 @@ -12,14 +12,14 @@ ( 'Received', 'Sample tube received' ), ( 'Done', 'Sequence run complete' ) ] address1 = dict( short_desc="Office", - name="James Bond", + name="James+Bond", institution="MI6" , - address1="MI6 Headquaters", + address1="MI6+Headquarters", address2="", city="London", state="London", postal_code="007", - country="United Kingdom", + country="United+Kingdom", phone="007-007-0007" ) @@ -165,26 +165,21 @@ self.visit_url( '%s/library_admin/browse_library?obj_id=%s' % ( self.url, str( library_one.id ) ) ) self.check_page_for_string( name ) # create address - #self.create_address( user_address1 ) - #self.check_page_for_string( 'Address <b>%s</b> has been added' % user_address1[ 'short_desc' ] ) - ## TODO: FIX HACK - ## the user address creation should be done as a test. + self.logout() + self.login( email='test1(a)bx.psu.edu' ) + self.home() + url_str = '%s/user/new_address?short_desc=%s&name=%s&institution=%s&address1=%s&address2=%s&city=%s&state=%s&postal_code=%s&country=%s&phone=%s' \ + % ( self.url, address1[ 'short_desc' ], address1[ 'name' ], address1[ 'institution' ], + address1[ 'address1' ], address1[ 'address2' ], address1[ 'city' ], address1[ 'state' ], + address1[ 'postal_code' ], address1[ 'country' ], address1[ 'phone' ] ) + print url_str + self.visit_url( url_str ) + self.check_page_for_string( 'Address <b>%s</b> has been added' % address1[ 'short_desc' ] ) global regular_user regular_user = galaxy.model.User.filter( galaxy.model.User.table.c.email=='test1(a)bx.psu.edu' ).first() global user_address - user_address = galaxy.model.UserAddress() - user_address.user = regular_user - user_address.desc = address1[ 'short_desc' ] - user_address.name = address1[ 'name' ] - user_address.institution = address1[ 'institution' ] - user_address.address = address1[ 'address1' ]+' '+address1[ 'address2' ] - user_address.city = address1[ 'city' ] - user_address.state = address1[ 'state' ] - user_address.postal_code = address1[ 'postal_code' ] - user_address.country = address1[ 'country' ] - user_address.phone = address1[ 'phone' ] - user_address.flush() - user_address.user.refresh() + user_address = galaxy.model.UserAddress.filter( and_( galaxy.model.UserAddress.table.c.desc==address1[ 'short_desc' ], + galaxy.model.UserAddress.table.c.deleted==False ) ).first() def test_030_create_request( self ): """Testing creating, editing and submitting a request as a regular user""" # login as a regular user

1 0

[hg] galaxy 2803: Prevent script tags in uploads as reported by ...
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/88fdb7f40b0e changeset: 2803:88fdb7f40b0e user: Nate Coraor <nate(a)bx.psu.edu> date: Wed Sep 30 09:22:02 2009 -0400 description: Prevent script tags in uploads as reported by Mitsuteru Nakao 1 file(s) affected in this change: tools/data_source/upload.py diffs (16 lines): diff -r feec758a8774 -r 88fdb7f40b0e tools/data_source/upload.py --- a/tools/data_source/upload.py Wed Sep 30 08:28:31 2009 -0400 +++ b/tools/data_source/upload.py Wed Sep 30 09:22:02 2009 -0400 @@ -49,10 +49,11 @@ regexp2 = re.compile( "<IFRAME[^>]*>", re.I ) regexp3 = re.compile( "<FRAMESET[^>]*>", re.I ) regexp4 = re.compile( "<META[^>]*>", re.I ) + regexp5 = re.compile( "<SCRIPT[^>]*>", re.I ) lineno = 0 for line in temp: lineno += 1 - matches = regexp1.search( line ) or regexp2.search( line ) or regexp3.search( line ) or regexp4.search( line ) + matches = regexp1.search( line ) or regexp2.search( line ) or regexp3.search( line ) or regexp4.search( line ) or regexp5.search( line ) if matches: if chunk is None: temp.close()

1 0

[hg] galaxy 2804: More functional tests for Requests
by Nate Coraor 02 Oct '09

02 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/3e5428b298b1 changeset: 2804:3e5428b298b1 user: rc date: Wed Sep 30 10:40:08 2009 -0400 description: More functional tests for Requests 2 file(s) affected in this change: templates/admin/forms/show_form_read_only.mako test/functional/test_forms_and_requests.py diffs (141 lines): diff -r 88fdb7f40b0e -r 3e5428b298b1 templates/admin/forms/show_form_read_only.mako --- a/templates/admin/forms/show_form_read_only.mako Wed Sep 30 09:22:02 2009 -0400 +++ b/templates/admin/forms/show_form_read_only.mako Wed Sep 30 10:40:08 2009 -0400 @@ -62,7 +62,11 @@ </%def> <div class="toolForm"> - <div class="toolFormTitle">${form.name} - <i>${form.desc}</i></div> + %if form.desc: + <div class="toolFormTitle">${form.name} - <i>${form.desc}</i></div> + %else: + <div class="toolFormTitle">${form.name}</div> + %endif <form name="library" action="${h.url_for( controller='forms', action='manage' )}" method="post" > %if form.type == trans.app.model.FormDefinition.types.SAMPLE: %if not len(form.layout): diff -r 88fdb7f40b0e -r 3e5428b298b1 test/functional/test_forms_and_requests.py --- a/test/functional/test_forms_and_requests.py Wed Sep 30 09:22:02 2009 -0400 +++ b/test/functional/test_forms_and_requests.py Wed Sep 30 10:40:08 2009 -0400 @@ -169,9 +169,11 @@ #self.check_page_for_string( 'Address <b>%s</b> has been added' % user_address1[ 'short_desc' ] ) ## TODO: FIX HACK ## the user address creation should be done as a test. + global regular_user + regular_user = galaxy.model.User.filter( galaxy.model.User.table.c.email=='test1(a)bx.psu.edu' ).first() global user_address user_address = galaxy.model.UserAddress() - user_address.user = galaxy.model.User.filter( galaxy.model.User.table.c.email=='test1(a)bx.psu.edu' ).first() + user_address.user = regular_user user_address.desc = address1[ 'short_desc' ] user_address.name = address1[ 'name' ] user_address.institution = address1[ 'institution' ] @@ -211,14 +213,22 @@ for field_value in fields: self.check_page_for_string( field_value ) # edit this request - fields = ['field one value (editted)', 'field two value (editted)', str(user_address.id)] + fields = ['field one value (edited)', 'field two value (edited)', str(user_address.id)] self.edit_request(request_one.id, request_one.name, request_one.name+' (Renamed)', request_one.desc+' (Re-described)', library_one.id, folder_one.id, fields) request_one.refresh() self.check_page_for_string( request_name+' (Renamed)' ) self.check_page_for_string( request_desc+' (Re-described)' ) + # check if the request is showing in the 'unsubmitted' filter + self.home() + self.visit_url( '%s/requests/list?show_filter=Unsubmitted' % self.url ) + self.check_page_for_string( request_one.name ) # submit the request self.submit_request( request_one.id, request_one.name ) request_one.refresh() + # check if the request is showing in the 'submitted' filter + self.home() + self.visit_url( '%s/requests/list?show_filter=Submitted' % self.url ) + self.check_page_for_string( request_one.name ) # check if the request's state is now set to 'submitted' assert request_one.state is not request_one.states.SUBMITTED, "The state of the request '%s' should be set to '%s'" % ( request_one.name, request_one.states.SUBMITTED ) def test_035_request_lifecycle( self ): @@ -250,40 +260,50 @@ self.visit_url('%s/requests_admin/list?show_filter=Complete' % self.url) self.check_page_for_string( request_one.name ) assert request_one.state is not request_one.states.COMPLETE, "The state of the request '%s' should be set to '%s'" % ( request_one.name, request_one.states.COMPLETE ) -# def test_40_admin_create_request_on_behalf_of_regular_user( self ): -# """Testing creating and submitting a request as an admin on behalf of a regular user""" -# self.logout() -# self.login( email='test(a)bx.psu.edu' ) -## permissions_in = [ k for k, v in galaxy.model.Library.permitted_actions.items() ] -## permissions_out = [] -## self.set_library_permissions( str( library_one.id ), library_one.name, str( admin_user_private_role.id ), permissions_in, permissions_out ) -# # set field values -# fields = ['field one value', 'field two value', str(user_address.id)] -# # create the request -# request_name, request_desc = 'Request Two', 'Request Two Description' -# self.create_request_admin(request_type.id, regular_user1.id, request_name, request_desc, library_one.id, fields) -# self.check_page_for_string( request_name ) -# self.check_page_for_string( request_desc ) -# global request_two -# request_one = galaxy.model.Request.filter( and_( galaxy.model.Request.table.c.name==request_name, -# galaxy.model.Request.table.c.deleted==False ) ).first() -# # check if the request's state is now set to 'unsubmitted' -# assert request_two.state is not request_two.states.UNSUBMITTED, "The state of the request '%s' should be set to '%s'" % ( request_two.name, request_two.states.UNSUBMITTED ) -# # sample fields -# samples = [ ( 'Sample One', [ 'S1 Field 0 Value' ] ), -# ( 'Sample Two', [ 'S2 Field 0 Value' ] ) ] -# # add samples to this request -# self.add_samples( request_two.id, request_two.name, samples ) -# for sample_name, fields in samples: -# self.check_page_for_string( sample_name ) -# self.check_page_for_string( 'Unsubmitted' ) -# for field_value in fields: -# self.check_page_for_string( field_value ) -# # submit the request -# self.submit_request( request_two.id, request_two.name ) -# request_two.refresh() -# # check if the request's state is now set to 'submitted' -# assert request_two.state is not request_two.states.SUBMITTED, "The state of the request '%s' should be set to '%s'" % ( request_two.name, request_two.states.SUBMITTED ) + def test_40_admin_create_request_on_behalf_of_regular_user( self ): + """Testing creating and submitting a request as an admin on behalf of a regular user""" + self.logout() + self.login( email='test(a)bx.psu.edu' ) + request_name = "RequestTwo" + # simulate request creation + url_str = '%s/requests_admin/new?create=True&create_request_button=Save&select_request_type=%i&select_user=%i&name=%s&library_id=%i&folder_id=%i&refresh=True&field_1=%s&field_2=%i' \ + % ( self.url, request_type.id, regular_user.id, request_name, library_one.id, library_one.root_folder.id, "field_1_value", user_address.id ) + self.home() + self.visit_url( url_str ) + self.check_page_for_string( "The new request named %s has been created" % request_name ) + global request_two + request_two = galaxy.model.Request.filter( and_( galaxy.model.Request.table.c.name==request_name, + galaxy.model.Request.table.c.deleted==False ) ).first() + # check if the request is showing in the 'unsubmitted' filter + self.home() + self.visit_url( '%s/requests_admin/list?show_filter=Unsubmitted' % self.url ) + self.check_page_for_string( request_two.name ) + # check if the request's state is now set to 'unsubmitted' + assert request_two.state is not request_two.states.UNSUBMITTED, "The state of the request '%s' should be set to '%s'" % ( request_two.name, request_two.states.UNSUBMITTED ) + # sample fields + samples = [ ( 'Sample One', [ 'S1 Field 0 Value' ] ), + ( 'Sample Two', [ 'S2 Field 0 Value' ] ) ] + # add samples to this request + self.add_samples( request_two.id, request_two.name, samples ) + for sample_name, fields in samples: + self.check_page_for_string( sample_name ) + self.check_page_for_string( 'Unsubmitted' ) + for field_value in fields: + self.check_page_for_string( field_value ) + # submit the request + self.submit_request( request_two.id, request_two.name ) + request_two.refresh() + # check if the request is showing in the 'submitted' filter + self.home() + self.visit_url( '%s/requests_admin/list?show_filter=Submitted' % self.url ) + self.check_page_for_string( request_two.name ) + # check if the request's state is now set to 'submitted' + assert request_two.state is not request_two.states.SUBMITTED, "The state of the request '%s' should be set to '%s'" % ( request_two.name, request_two.states.SUBMITTED ) + # check if both the requests is showing in the 'All' filter + self.home() + self.visit_url( '%s/requests_admin/list?show_filter=All' % self.url ) + self.check_page_for_string( request_one.name ) + self.check_page_for_string( request_two.name )

1 0