How do you have it set up on the main public galaxy install? I imagine that people run enough big jobs that there there is enormous use of your shared file system. How did you scale that to so many nodes without bogging down the file system with large dataset transfers?
It seems that for now the solution of having a second Galaxy instance will work well, thank you very much John :) . But I'm still interested in a more permanent scaled solution. After reading up more on our shared file system it still seems like heavy traffic is bad, so could my initial idea still be good?