Interesting. If I'm reading this correctly the problem is happening inside pkg_resources? (galaxy.eggs unzips eggs, but I think it does so on install [fetch_eggs] time not run time which would avoid this). If so this would seem to be a locking bug in pkg_resources. Dannon, we could put a guard around the imports in extract_dataset_part.py as an (overly aggressive and hacky) fix. -- jt On Tue, Sep 18, 2012 at 10:37 AM, Jorrit Boekel <jorrit.boekel@scilifelab.se> wrote:
- which lead to unzipping .so libraries from python eggs into the nodes' /home/galaxy/.python-eggs - this runs into lib/pkg_resources.py and its _bypass_ensure_directory method that creates the temporary dir for the egg unzip - since there are 8 processes on the node, sometimes this method tries to mkdir a directory that was just made by the previous process after the isdir.