Hi All,
Some of the developers will know me already but it's my first time being properly on this list as I'm now working concertedly on Galaxy. I'm responsible for Bio-Linux which currently ships with an out-of-date and slightly flakey galaxy, but I'm working to fix that.
I've hit a problem. This may or may not be the same as the issue seen by Luobin Yang.
When uploading a file, no job appears in the right-hand panel. If I click refresh I might see a job but it stays in the blue "Dataset is uploading" state forever.
I'm working on packaging Galaxy for Debian/Ubuntu/Bio-Linux, specifically Ubuntu 12.04 LTS (python 2.7.3). To rearrange Galaxy into something that resembles a Debian package takes a lot of tinkering and I've patched a few bits of the code, so when uploads started freezing like this I assumed it was something I'd done and started reverting patches.
But it turns out I can trigger this problem on a pristine download of the latest server release (20130926 from Bitbucket). All I do is:
grab and unpack latest version python ./scripts/fetch_eggs.py -c universe_wsgi.ini.sample python ./scripts/fetch_eggs.py -c universe_wsgi.ini.sample -e psycopg2 (note - I'm not actually using Posgres in this case and I'm assuming this is irrelevant but reporting for completenes) mkdir eggs-cache ; export PYTHON_EGG_CACHE=`pwd`/eggs-cache ./run.sh (wait for SQLite DB to build) Ctrl-C, ./run.sh, test, Ctrl-C, ./run.sh, test, etc...
After a few restarts the problem occurs. Failing that, it seems I can make failure more likely by tarring up the whole directory and untarring it again. I suspect all this is doing is affecting the disk cache timing and thus triggering some underlying race condition.
Once the server is started, behaviour is apparently consistent - ie. all uploads either fail or all work. Other tasks seem to work, so long as they don't depend on the stalled uploads, but I've only tried a couple of these (just a basic "select first lines").
I'm now convinced there is some sort of race condition on start-up that is responsible. On my DEB package it fails like this all the time, but an issue is clearly there in the unmolested code, just rarer.
I don't mind getting into the code to try and debug this but I don't know where to start, even with the source code docs and my rapidly increasing Python-fu. Trying to fathom the interactions between the threads and the browser and the SQLAlchemy layer I either need a good outline document or a developer to hold my hand for a bit. Can someone help me out with this one?
Thanks in advance,
TIM