Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: c4a2d8c2189423e45124d847928431a726587eec https://github.com/galaxyproject/galaxy/commit/c4a2d8c2189423e45124d84792843... Author: Nicola Soranzo nicola.soranzo@earlham.ac.uk Date: 2017-11-28 (Tue, 28 Nov 2017)

Changed paths: M lib/galaxy/datatypes/text.py

Log Message: ----------- Fix html sniffer to not return True in case of exception

Commit: 1ba863d4d6af388aaf5911b1f57a1ce59eb20f09 https://github.com/galaxyproject/galaxy/commit/1ba863d4d6af388aaf5911b1f57a1... Author: Nicola Soranzo nicola.soranzo@earlham.ac.uk Date: 2017-11-28 (Tue, 28 Nov 2017)

Changed paths: M lib/tool_shed/util/tool_util.py

Log Message: ----------- Remove duplicated functions

Added in commits 777d6a155c369e49127084b63dee5701da7589f1 and 917d5a72e83741594912578ee14d1df19f5974c9

Commit: 0df99d89d7d749df8d3432a3506d76b2461e8bfe https://github.com/galaxyproject/galaxy/commit/0df99d89d7d749df8d3432a3506d7... Author: Nicola Soranzo nicola.soranzo@earlham.ac.uk Date: 2017-11-28 (Tue, 28 Nov 2017)

Changed paths: M lib/galaxy/datatypes/images.py M lib/galaxy/datatypes/sniff.py A lib/galaxy/datatypes/test/454Score.pdf

Log Message: ----------- Improve and test the PDF sniffer

Read just the first 4 bytes in binary mode.

Commit: 4a19bf62e35d52236a1fe5572a19e24ac20c3efe https://github.com/galaxyproject/galaxy/commit/4a19bf62e35d52236a1fe5572a19e... Author: Nicola Soranzo nicola.soranzo@earlham.ac.uk Date: 2017-11-28 (Tue, 28 Nov 2017)

Changed paths: M test-data/1.fastqsanger M test-data/1.fastqsolexa M test-data/simple_line_alternative.txt M test-data/simple_lines_interleaved.txt

Log Message: ----------- Add missing newline at end of some test files

It gets added anyway when the file is uploaded to Galaxy due to the default "Use POSIX standard" (to_posix_lines) option. Many tests which simply copy them with cat and compare them would fail a strict diff test.

Commit: 5a2cb8546948c2f57d780201c1252d55269fcfe3 https://github.com/galaxyproject/galaxy/commit/5a2cb8546948c2f57d780201c1252... Author: Nicola Soranzo nicola.soranzo@earlham.ac.uk Date: 2017-11-28 (Tue, 28 Nov 2017)

Changed paths: M lib/galaxy/datatypes/text.py

Log Message: ----------- Fix `SnpSiftDbNSFP.generate_primary_file()` to return a string

Also use a `with` statement to open/close a file in `SnpSiftDbNSFP.regenerate_primary_file()` .

Commit: 6b730ae961c12375bd37e870cc52e0f77e4c0e17 https://github.com/galaxyproject/galaxy/commit/6b730ae961c12375bd37e870cc52e... Author: Nicola Soranzo nicola.soranzo@earlham.ac.uk Date: 2017-11-28 (Tue, 28 Nov 2017)

Changed paths: M lib/galaxy/datatypes/registry.py

Log Message: ----------- Add a datatype to sniff_order only if it has a sniff() method

This reduces the number of default sniffers from 412 to 246.

Commit: 59f44cf2be48abd4bed0f7c6ddf7cc0aee5ad1f5 https://github.com/galaxyproject/galaxy/commit/59f44cf2be48abd4bed0f7c6ddf7c... Author: Nicola Soranzo nicola.soranzo@earlham.ac.uk Date: 2017-11-28 (Tue, 28 Nov 2017)

Changed paths: M lib/galaxy/datatypes/annotation.py M lib/galaxy/datatypes/binary.py M lib/galaxy/datatypes/blast.py M lib/galaxy/datatypes/constructive_solid_geometry.py M lib/galaxy/datatypes/data.py M lib/galaxy/datatypes/graph.py M lib/galaxy/datatypes/molecules.py M lib/galaxy/datatypes/msa.py M lib/galaxy/datatypes/plant_tribes.py M lib/galaxy/datatypes/proteomics.py M lib/galaxy/datatypes/sequence.py M lib/galaxy/datatypes/sniff.py M lib/galaxy/datatypes/tabular.py M lib/galaxy/datatypes/text.py M lib/galaxy/datatypes/triples.py M lib/galaxy/datatypes/xml.py M lib/galaxy/jobs/__init__.py M lib/galaxy/model/__init__.py M lib/galaxy/model/migrate/versions/0005_cleanup_datasets_fix.py M lib/galaxy/tools/verify/__init__.py M lib/galaxy/util/compression_utils.py M tools/data_source/data_source.py M tools/data_source/upload.py

Log Message: ----------- Update `get_fileobj()` to use utf-8 encoding in text mode

Also, merge its 3 parameters `gzip_only`, `bz2_only`, `zip_only` into `compressed_formats` (a list of allowed formats).

As a consequence of the changes in `get_fileobj()`, update: - `files_diff()` - `get_file_peek()`, which now determines that a file is binary when a `UnicodeDecodeError` exception is raised and doesn't need `is_multi_byte` any more - `iter_headers()` and `get_headers()`, which now return Unicode and don't need `is_multi_byte` parameter any more

As a consequence of the changes in `get_file_peek()`, update: - `set_peek()`, which now doesn't need `is_multi_byte` any more

As a consequence of the changes in `get_headers()`, update: - `guess_ext` and `is_column_based()`, which now determine that a file is binary when a `UnicodeDecodeError` exception is raised and don't need `is_multi_byte` any more

As a consequence of the changes to `guess_ext`, update: - `handle_uploaded_dataset_file() doesn't need `is_multi_byte` any more

Also, remove duplicated calls to `get_file_peek()` in lib/galaxy/datatypes/molecules.py and lib/galaxy/datatypes/msa.py

The `is_multi_byte` was not removed from the signature of `get_file_peek()` and `set_peek()` in order to preserve compatibility for ToolShed datatypes, thanks @jmchilton for the review.

Commit: 8d7f344140d77a6c50d945fa9b48f842e00e837f https://github.com/galaxyproject/galaxy/commit/8d7f344140d77a6c50d945fa9b48f... Author: John Chilton jmchilton@gmail.com Date: 2017-11-28 (Tue, 28 Nov 2017)

Changed paths: M lib/galaxy/datatypes/annotation.py M lib/galaxy/datatypes/binary.py M lib/galaxy/datatypes/blast.py M lib/galaxy/datatypes/constructive_solid_geometry.py M lib/galaxy/datatypes/data.py M lib/galaxy/datatypes/graph.py M lib/galaxy/datatypes/images.py M lib/galaxy/datatypes/molecules.py M lib/galaxy/datatypes/msa.py M lib/galaxy/datatypes/plant_tribes.py M lib/galaxy/datatypes/proteomics.py M lib/galaxy/datatypes/registry.py M lib/galaxy/datatypes/sequence.py M lib/galaxy/datatypes/sniff.py M lib/galaxy/datatypes/tabular.py A lib/galaxy/datatypes/test/454Score.pdf M lib/galaxy/datatypes/text.py M lib/galaxy/datatypes/triples.py M lib/galaxy/datatypes/xml.py M lib/galaxy/jobs/__init__.py M lib/galaxy/model/__init__.py M lib/galaxy/model/migrate/versions/0005_cleanup_datasets_fix.py M lib/galaxy/tools/verify/__init__.py M lib/galaxy/util/compression_utils.py M lib/tool_shed/util/tool_util.py M test-data/1.fastqsanger M test-data/1.fastqsolexa M test-data/simple_line_alternative.txt M test-data/simple_lines_interleaved.txt M tools/data_source/data_source.py M tools/data_source/upload.py

Log Message: ----------- Merge pull request #5062 from nsoranzo/remove_wchartype_dep

Remove wchartype dependency and is_multi_byte [1/2]

Compare: https://github.com/galaxyproject/galaxy/compare/68795e415c16...8d7f344140d7