Hi Peter,
Code looks interesting. This is for Univa Grid Engine?
In its current state its tailored for the Univa Grid Engine. But the drmaa library code that is based on job_info() and wait() should run on all grid engines. For the command line based code changes might be necessary for the different grid engines. (Currently there is only a small bug, because wait() is currently called twice for finished jobs: in the repeated polling and then in the final check. The second call won't work which causes a fallback to qacct. I will fix this soon.) I guess that the output of qstat and qacct will be different. But I think this can be configured one just needs a way to get the info which grid engine is running.
In terms of the upload jobs, are those not designed to be run as 'local' jobs and not with the 'real user' setting?
Sounds reasonable, but this is not what is happening on my installation of galaxy. Any idea where I could start to find the problem. Best, Matthias