Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 9e71e1d2f0e7c6586b3f5946edb033466a495258 https://github.com/galaxyproject/galaxy/commit/9e71e1d2f0e7c6586b3f5946edb03... Author: John Chilton jmchilton@gmail.com Date: 2018-03-29 (Thu, 29 Mar 2018)
Changed paths: M lib/galaxy/webapps/galaxy/api/jobs.py
Log Message: ----------- Allow anonymous users to view their jobs via the API.
Commit: 083bdcbdd84da29b1e2692058bbebd6426e8b2ae https://github.com/galaxyproject/galaxy/commit/083bdcbdd84da29b1e2692058bbeb... Author: John Chilton jmchilton@gmail.com Date: 2018-03-29 (Thu, 29 Mar 2018)
Changed paths: A client/galaxy/scripts/components/RuleCollectionBuilder.vue M client/galaxy/scripts/mvc/collection/list-collection-creator.js M client/galaxy/scripts/mvc/history/history-view-edit.js M client/galaxy/scripts/mvc/history/job-states-model.js A client/galaxy/scripts/mvc/upload/collection/rules-input-view.js M client/galaxy/scripts/mvc/upload/upload-ftp.js A client/galaxy/scripts/mvc/upload/upload-utils.js M client/galaxy/scripts/mvc/upload/upload-view.js M client/galaxy/style/less/upload.less M client/package.json A test-data/rules/PRJDA60709.tsv A test-data/rules/PRJDB3920.tsv A test-data/rules/PRJNA355367.tsv A test-data/rules/uniprot.json A test-data/rules/uniprot.tsv M test/functional/tools/sample_datatypes_conf.xml M test/galaxy_selenium/has_driver.py M test/galaxy_selenium/navigates_galaxy.py M test/galaxy_selenium/navigation.yml M test/galaxy_selenium/smart_components.py M test/selenium_tests/framework.py M test/selenium_tests/test_collection_builders.py M test/selenium_tests/test_uploads.py
Log Message: ----------- Rule-based Collection Builder
Vue-based component for defining collections by applying rules to a list of files or more general spreadsheet style information (e.g. sample sheets or tabular data from data sources containing URL or FTP file paths for files along with metadata). The widget is fairly complex but very broadly is broken into two panes - one to preview how rules are applied to build up tabular data defining collections (each row corresponding to a file with columns for metadata and such) and one that displays defined rules and allows for editing of these rules and creation of new ones.
The goal behind defining rules this way instead of allowing the user to interact with the spreadsheet display directly is to enable scaling up collection creation. If a user wishes to upload hundreds of datasets - interacting with a widget directly for each input doesn't scale well and would be error prone. If a user wishes to upload hundreds of thousands of datasets - even loading this information in the GUI may not scale (though I've been impressed with the performance so far of this approach) and so we can potentially just display a preview of some of the rows and process the final set of rules on the backend.
Since we can handle an arbitrary number of columns this way, we can define multiple list identifiers per file and so we can easily construct nested lists. Hence this allows creation of not just potentially larger collections but arbitrarily complex lists as well. Paired identifiers via indicator columns are also implemented.
In order to operate over lists of datasets directly - the multi-select history widget now has a new option "Build Collection from Rules" along side the other collection builders. This mode uses the well established dataset collection API to build collections from HDAs.
In order to operate on lists of FTP files or URLs - the upload widget has a new tab "Rule-based" tab that allows users to paste in tabular data or select a history dataset and then send this tabular data to the new builder widget. This will be extended to include FTP directories for instance over time. This mode uses the new data fetch API to build collections and handle uploads of arbitrary collections of files.
The preview of the tabular data generated via rules is done via [Handsontable](https://handsontable.com/) - a JavaScript spreadsheet widget with a VueJS [wrapper component](https://github.com/handsontable/vue-handsontable-official). This turns out to be a fairly nice application for reactive components - as rules are added or modified the spreadsheet just naturally updates. In my hands the widget scales very nicely - I've uploaded files with tens of thousands of rows and rules modifying the data and changing the spreadsheet do not seem to cause siignificant delays in the web browser.
Commit: ebf08c16493b8fd3e770bf3c33c1f3eea55dd374 https://github.com/galaxyproject/galaxy/commit/ebf08c16493b8fd3e770bf3c33c1f... Author: Dannon Baker dannon.baker@gmail.com Date: 2018-03-29 (Thu, 29 Mar 2018)
Changed paths: M client/webpack.config.js
Log Message: ----------- Fix production build of vue-handsontable
Commit: cfa1208a79dcd0bdda5c7632366c2bd2b6deea0d https://github.com/galaxyproject/galaxy/commit/cfa1208a79dcd0bdda5c7632366c2... Author: Dannon Baker dannon.baker@gmail.com Date: 2018-03-29 (Thu, 29 Mar 2018)
Changed paths: M client/yarn.lock
Log Message: ----------- Yarn lock w/ new deps.
Commit: b32e78f338fcbd586f77c64f0fa18aee6ce23d1b https://github.com/galaxyproject/galaxy/commit/b32e78f338fcbd586f77c64f0fa18... Author: Dannon Baker dannon.baker@gmail.com Date: 2018-03-29 (Thu, 29 Mar 2018)
Changed paths: M client/galaxy/scripts/mvc/upload/upload-view.js
Log Message: ----------- Fix upload view initialization.
Commit: 1c04bbe9f6a95c1c8050bc5a8cd69bd8030cf350 https://github.com/galaxyproject/galaxy/commit/1c04bbe9f6a95c1c8050bc5a8cd69... Author: Dannon Baker dannon.baker@gmail.com Date: 2018-03-29 (Thu, 29 Mar 2018)
Changed paths: M client/.babelrc M client/package.json M client/yarn.lock
Log Message: ----------- Add babel plugin for vue template preprocessing. This allows precompilation without a vue runtime for nested local components in vue SFCs, and other situations.
Commit: e552bc4cbb592eb2fcb4658e287026a742aab286 https://github.com/galaxyproject/galaxy/commit/e552bc4cbb592eb2fcb4658e28702... Author: Martin Cech marten@bx.psu.edu Date: 2018-03-30 (Fri, 30 Mar 2018)
Changed paths: M client/.babelrc A client/galaxy/scripts/components/RuleCollectionBuilder.vue M client/galaxy/scripts/mvc/collection/list-collection-creator.js M client/galaxy/scripts/mvc/history/history-view-edit.js M client/galaxy/scripts/mvc/history/job-states-model.js A client/galaxy/scripts/mvc/upload/collection/rules-input-view.js M client/galaxy/scripts/mvc/upload/upload-ftp.js A client/galaxy/scripts/mvc/upload/upload-utils.js M client/galaxy/scripts/mvc/upload/upload-view.js M client/galaxy/style/less/upload.less M client/package.json M client/webpack.config.js M client/yarn.lock M lib/galaxy/webapps/galaxy/api/jobs.py A test-data/rules/PRJDA60709.tsv A test-data/rules/PRJDB3920.tsv A test-data/rules/PRJNA355367.tsv A test-data/rules/uniprot.json A test-data/rules/uniprot.tsv M test/functional/tools/sample_datatypes_conf.xml M test/galaxy_selenium/has_driver.py M test/galaxy_selenium/navigates_galaxy.py M test/galaxy_selenium/navigation.yml M test/galaxy_selenium/smart_components.py M test/selenium_tests/framework.py M test/selenium_tests/test_collection_builders.py M test/selenium_tests/test_uploads.py
Log Message: ----------- Merge pull request #5365 from jmchilton/rule_based_uploader
Rule-based Uploader / Builder
Compare: https://github.com/galaxyproject/galaxy/compare/ac74b7ffb221...e552bc4cbb59