3 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/eb7a31e38a4c/ changeset: eb7a31e38a4c branch: dynamic_options_memory_fix user: jmchilton date: 2012-02-07 17:35:17 summary: Patch dynamic_options.py to just read just the first megabyte of an input dataset (if it is larger than that). Otherwise Galaxy will attempt the read the entire file into memory, this can causes crashes and freezeups without the user even selecting a bad input (for instance, if a large FASTQ file is the top most history item and a txt input is sought, Galaxy will freezeup as soon as the tool is clicked because it will try to load the options for the top item immediately by default). Though the implementation is not as clean, I am restricting it to a megabyte instead of a number of lines/options in case the top most history item has no line breaks. I imagine in that case the same problem could occur even if the input is restricted to a reasonable number of lines (say 100). affected #: 1 file diff -r 74b6319b38b4b3876d0b81ef1296bb5afc729cc1 -r eb7a31e38a4cf9968d115984ab2ded908d3ddc9e lib/galaxy/tools/parameters/dynamic_options.py --- a/lib/galaxy/tools/parameters/dynamic_options.py +++ b/lib/galaxy/tools/parameters/dynamic_options.py @@ -500,7 +500,16 @@ dataset = other_values.get( self.dataset_ref_name, None ) assert dataset is not None, "Required dataset '%s' missing from input" % self.dataset_ref_name if not dataset: return [] #no valid dataset in history - options = self.parse_file_fields( open( dataset.file_name ) ) + # Ensure parsing dynamic options does not consume more than a megabyte worth memory. + file_size = os.path.getsize( path ) + if os.path.getsize( path ) < 1048576: + options = self.parse_file_fields( open( path ) ) + else: + # Pass just the first megabyte to parse_file_fields. + import StringIO + log.warn( "Attempting to load options from large file, reading just first megabyte" ) + contents = open( path, 'r' ).read( megabyte ) + options = self.parse_file_fields( StringIO.StringIO( contents ) ) elif self.tool_data_table: options = self.tool_data_table.get_fields() else: https://bitbucket.org/galaxy/galaxy-central/changeset/53e55b49a24a/ changeset: 53e55b49a24a branch: dynamic_options_memory_fix user: jmchilton date: 2012-02-07 17:38:00 summary: Fix a typo made with last commit. affected #: 1 file diff -r eb7a31e38a4cf9968d115984ab2ded908d3ddc9e -r 53e55b49a24a8e9a2f00b1c594c4e54c226e8865 lib/galaxy/tools/parameters/dynamic_options.py --- a/lib/galaxy/tools/parameters/dynamic_options.py +++ b/lib/galaxy/tools/parameters/dynamic_options.py @@ -501,6 +501,7 @@ assert dataset is not None, "Required dataset '%s' missing from input" % self.dataset_ref_name if not dataset: return [] #no valid dataset in history # Ensure parsing dynamic options does not consume more than a megabyte worth memory. + path = dataset.file_name file_size = os.path.getsize( path ) if os.path.getsize( path ) < 1048576: options = self.parse_file_fields( open( path ) ) https://bitbucket.org/galaxy/galaxy-central/changeset/046157d1ed8a/ changeset: 046157d1ed8a branch: dynamic_options_memory_fix user: jmchilton date: 2012-02-07 17:50:22 summary: One more fixed typo. affected #: 1 file diff -r 53e55b49a24a8e9a2f00b1c594c4e54c226e8865 -r 046157d1ed8af7ff178c6b1edee9512deccbc0ce lib/galaxy/tools/parameters/dynamic_options.py --- a/lib/galaxy/tools/parameters/dynamic_options.py +++ b/lib/galaxy/tools/parameters/dynamic_options.py @@ -509,7 +509,7 @@ # Pass just the first megabyte to parse_file_fields. import StringIO log.warn( "Attempting to load options from large file, reading just first megabyte" ) - contents = open( path, 'r' ).read( megabyte ) + contents = open( path, 'r' ).read( 1048576 ) options = self.parse_file_fields( StringIO.StringIO( contents ) ) elif self.tool_data_table: options = self.tool_data_table.get_fields() Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.