Job launching slowed down by conda auto-install
Hello all, I recently realized that setting conda_auto_install to true in galaxy.ini has a noticeable effect in slowing down the start of job execution. On the first job run after activating conda_auto_install the job takes a lot longer as conda is actually installing packages, which is expected. On the following runs, even if there are no packages to be installed jobs still take longer to start. What's more surprising is that after deactivating conda_auto_install jobs still take longer to run. My particular test case was a simple small (12MB) file upload. It consistently went from 15s to over 30s after activating conda_auto_install and remained at that even after deactivating it. We tested running a few tools which presented a noticeable and irreversible delay after activation of conda_auto_install, but I didn't measure these delays. I also tested this on fresh installs of galaxy 16.07 and 16.10 with very similar results. Is this supposed to happen? Is there a way around it besides never activating conda_auto_install? Thanks in advance, Tiago
Hello Tiago, conda_auto_install will keep on trying to install the dependencies for a tool, each and every time, even if that dependency does not exist. Starting with release 16.07 you can see if a conda dependency is installed for a specific repository, and you can opt to install conda dependencies during installation. I would strongly recommend to not activate conda_auto_install on a normal production server. Best, Marius On 21 December 2016 at 17:21, Tiago Macedo <tmacedo@igc.gulbenkian.pt> wrote:
Hello all,
I recently realized that setting conda_auto_install to true in galaxy.ini has a noticeable effect in slowing down the start of job execution. On the first job run after activating conda_auto_install the job takes a lot longer as conda is actually installing packages, which is expected. On the following runs, even if there are no packages to be installed jobs still take longer to start. What's more surprising is that after deactivating conda_auto_install jobs still take longer to run. My particular test case was a simple small (12MB) file upload. It consistently went from 15s to over 30s after activating conda_auto_install and remained at that even after deactivating it. We tested running a few tools which presented a noticeable and irreversible delay after activation of conda_auto_install, but I didn't measure these delays. I also tested this on fresh installs of galaxy 16.07 and 16.10 with very similar results. Is this supposed to happen? Is there a way around it besides never activating conda_auto_install?
Thanks in advance, Tiago
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (2)
-
Marius van den Beek
-
Tiago Macedo