Hello,
Current setup: : have 1-2 VMs w/ 4-core each, 4GB mem mounting remote lustre shared fs : use remote job submission: cli/ssh : proxied, use remote auth login/logout
I'm prepping for a scenario where there will be 60-75 users/students who will simultaneously submit several complete genomics workflows to the single VM (I actually have 3 identical VMs). Full runs can go from 30min to an hour and a half each.
I have 1 of the VMs set up with the standalone paste-based with 5 job handler and 5 web server processes. Would this be enough for this scenario? (or 10/10, 20/20, more?).
I have the 2nd VM being set up using the uWSGI method.
Would it be worth it to proceed with the standalone paste-based or just skip it and exclusively only concentrate on the uWSGI setup (or keep both?). I'd like to automate everything (remote job submissions for full workflows) and just let it (60 test users or so) run for 1 month to check reliability, anticipate problems.
I'd appreciate feedback, thoughts from those who have already gone through a similar experience.
TIA