Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 8d687bc3b7e2e4496b464e28c0961ba9fe2ad691 https://github.com/galaxyproject/galaxy/commit/8d687bc3b7e2e4496b464e28c0961... Author: John Chilton <jmchilton@gmail.com> Date: 2018-07-18 (Wed, 18 Jul 2018) Changed paths: M lib/galaxy/tools/actions/__init__.py Log Message: ----------- Cache chrom_info generation when mapping over collections. Eliminates several SQL queries per job and 20ms off the time it takes to create a job (about 10% for simple jobs). Timings below based on a branch with other optimizations so this will probably represent a smaller percent of runtime in dev until it catches up but the idea is pretty self contained unlike some other work in local my branch. Before chrom_info cache: ``` galaxy.tools.actions INFO 2018-07-18 14:59:35,041 Collected inputs for tool execute (14.586 ms) galaxy.tools.actions INFO 2018-07-18 14:59:35,069 setup execute (28.450 ms) galaxy.tools.actions INFO 2018-07-18 14:59:35,071 Handled output named out_file1 for tool cat_data_and_sleep (1.082 ms) galaxy.tools.actions INFO 2018-07-18 14:59:35,077 Added output datasets to history (5.986 ms) galaxy.tools.actions INFO 2018-07-18 14:59:35,104 Flushed output datasets (27.165 ms) ... galaxy.tools.actions INFO 2018-07-18 14:59:35,183 Collected inputs for tool execute (9.299 ms) galaxy.tools.actions INFO 2018-07-18 14:59:35,210 setup execute (26.842 ms) galaxy.tools.actions INFO 2018-07-18 14:59:35,211 Handled output named out_file1 for tool cat_data_and_sleep (1.375 ms) galaxy.tools.actions INFO 2018-07-18 14:59:35,217 Added output datasets to history (5.168 ms) galaxy.tools.actions INFO 2018-07-18 14:59:35,244 Flushed output datasets (27.395 ms) ... galaxy.model.orm.engine_factory INFO 2018-07-18 14:59:36,033 Executed [417] SQL requests in for web request [/api/tools] (807.188749(s)ms) ``` ``` galaxy.tools.actions INFO 2018-07-18 14:58:20,505 Collected inputs for tool execute (8.643 ms) galaxy.tools.actions INFO 2018-07-18 14:58:20,533 setup execute (27.325 ms) galaxy.tools.actions INFO 2018-07-18 14:58:20,534 Handled output named out_file1 for tool cat_data_and_sleep (1.201 ms) galaxy.tools.actions INFO 2018-07-18 14:58:20,539 Added output datasets to history (4.913 ms) galaxy.tools.actions INFO 2018-07-18 14:58:20,565 Flushed output datasets (25.520 ms) ... galaxy.tools.actions INFO 2018-07-18 14:58:20,641 Collected inputs for tool execute (7.044 ms) galaxy.tools.actions INFO 2018-07-18 14:58:20,669 setup execute (27.744 ms) galaxy.tools.actions INFO 2018-07-18 14:58:20,670 Handled output named out_file1 for tool cat_data_and_sleep (0.990 ms) galaxy.tools.actions INFO 2018-07-18 14:58:20,674 Added output datasets to history (4.727 ms) galaxy.tools.actions INFO 2018-07-18 14:58:20,701 Flushed output datasets (26.265 ms) ... galaxy.model.orm.engine_factory INFO 2018-07-18 14:58:21,480 Executed [417] SQL requests in for web request [/api/tools] (796.465635(s)ms) ``` After chrom_info cache: ``` ... galaxy.tools.actions INFO 2018-07-18 14:56:21,486 Collected inputs for tool execute (9.792 ms) galaxy.tools.actions INFO 2018-07-18 14:56:21,487 setup execute (0.399 ms) galaxy.tools.actions INFO 2018-07-18 14:56:21,488 Handled output named out_file1 for tool cat_data_and_sleep (1.068 ms) galaxy.tools.actions INFO 2018-07-18 14:56:21,497 Added output datasets to history (9.218 ms) galaxy.tools.actions INFO 2018-07-18 14:56:21,522 Flushed output datasets (24.905 ms) ... galaxy.model.orm.engine_factory INFO 2018-07-18 14:56:22,352 Executed [381] SQL requests in for web request [/api/tools] (684.080839(s)ms) ``` ``` galaxy.tools.actions INFO 2018-07-18 14:55:27,979 Collected inputs for tool execute (8.400 ms) galaxy.tools.actions INFO 2018-07-18 14:55:27,979 setup execute (0.462 ms) galaxy.tools.actions INFO 2018-07-18 14:55:27,980 Handled output named out_file1 for tool cat_data_and_sleep (1.208 ms) galaxy.tools.actions INFO 2018-07-18 14:55:27,991 Added output datasets to history (10.558 ms) galaxy.tools.actions INFO 2018-07-18 14:55:28,016 Flushed output datasets (24.182 ms) ... galaxy.model.orm.engine_factory INFO 2018-07-18 14:55:28,838 Executed [381] SQL requests in for web request [/api/tools] (714.353323(s)ms) ``` Commit: b39bebb6a694af2cd50cee8c416443e7dcb501cf https://github.com/galaxyproject/galaxy/commit/b39bebb6a694af2cd50cee8c41644... Author: Nicola Soranzo <nicola.soranzo@gmail.com> Date: 2018-07-20 (Fri, 20 Jul 2018) Changed paths: M lib/galaxy/tools/actions/__init__.py Log Message: ----------- Merge pull request #6515 from jmchilton/chrom_info_cache Cache chrom_info generation when mapping over collections. Compare: https://github.com/galaxyproject/galaxy/compare/20f10fe6db8c...b39bebb6a694 **NOTE:** This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.