Dear All,
During the ESG annual meeting in Freiburg we discussed more in detail about the meta scheduling implementation.
So far we have been discussing adding this to DIRAC. However, the UseGalaxy.eu team felt that it would be better to create a separate API endpoint for the meta scheduling (specifically using https://fastapi.tiangolo.com/ and https://www.openapis.org/). We all agreed this is a better approach.
Also, we discussed that TPV has a global overview of all destinations in Galaxy, and DIRAC only sees the Compute Elements under a Virtual Organistaion. Specifically, this is where the scheduling happens in TPV, after it has filtered out all the candidate destinations for a given job (tool): https://github.com/usegalaxy-eu/infrastructure-playbook/blob/master/files/ga...
It is that part of the code that will call the new API endpoint to return the best destination.
We informally discussed that @paul.degeest@psb.vib-ugent.be paul.degeest@psb.vib-ugent.be will contact David Lopez (UseGalaxy.eu) to get started with the FastAPI implementation. @Abdulrahman Azab abdulrahman.azab@usit.uio.no will be contributing his code for the meta scheduling. I assume that the best first step is to create a GitHub repository under https://github.com/usegalaxy-eu so we can then point interested people to it?
I have added this to our shared T4.3 document:
https://docs.google.com/document/d/1YGWsyXlIkyBD_vpkwrWD6HUxdRC1zL6RcdyV2M1E...
Please let me know if there are any questions.
Best regards,
Sebastian
Hi,
I created https://github.com/usegalaxy-eu/tpv-metascheduler-api
We can rename and fine tune at any point, let me know if you want to have access by providing me your github handle.
We will handle access relaxed, but ideally we should contribute via PRs.
Exciting! Bjoern
Am 09.10.23 um 11:22 schrieb Sebastian Luna-Valero:
Dear All,
During the ESG annual meeting in Freiburg we discussed more in detail about the meta scheduling implementation.
So far we have been discussing adding this to DIRAC. However, the UseGalaxy.eu http://UseGalaxy.eu team felt that it would be better to create a separate API endpoint for the meta scheduling (specifically using https://fastapi.tiangolo.com/ https://fastapi.tiangolo.com/ and https://www.openapis.org/ https://www.openapis.org/). We all agreed this is a better approach.
Also, we discussed that TPV has a global overview of all destinations in Galaxy, and DIRAC only sees the Compute Elements under a Virtual Organistaion. Specifically, this is where the scheduling happens in TPV, after it has filtered out all the candidate destinations for a given job (tool): https://github.com/usegalaxy-eu/infrastructure-playbook/blob/master/files/ga... https://github.com/usegalaxy-eu/infrastructure-playbook/blob/master/files/galaxy/tpv/tool_defaults.yml#L47
It is that part of the code that will call the new API endpoint to return the best destination.
We informally discussed that @paul.degeest@psb.vib-ugent.be mailto:paul.degeest@psb.vib-ugent.be will contact David Lopez (UseGalaxy.eu) to get started with the FastAPI implementation. @Abdulrahman Azab mailto:abdulrahman.azab@usit.uio.no will be contributing his code for the meta scheduling. I assume that the best first step is to create a GitHub repository under https://github.com/usegalaxy-eu https://github.com/usegalaxy-eu so we can then point interested people to it?
I have added this to our shared T4.3 document:
https://docs.google.com/document/d/1YGWsyXlIkyBD_vpkwrWD6HUxdRC1zL6RcdyV2M1E... https://docs.google.com/document/d/1YGWsyXlIkyBD_vpkwrWD6HUxdRC1zL6RcdyV2M1EAGY/edit#heading=h.jnrgjdvsin2y
Please let me know if there are any questions.
Best regards,
Sebastian
-- Sebastian Luna-Valero (he/him/his) | Cloud Community Support Specialist | EGI Foundation (Amsterdam, The Netherlands) W: www.egi.eu http://www.egi.eu/ | E: sebastian.luna.valero@egi.eu mailto:sebastian.luna.valero@egi.eu | Twitter: SLunaValero
EGI: Advanced Computing for Research The EGI Foundation is ISO 9001:2015 and ISO/IEC 20000-1:2018 certified
Esg-wp4 mailing list -- esg-wp4@lists.galaxyproject.org To unsubscribe send an email to esg-wp4-leave@lists.galaxyproject.org
esg-wp4@lists.galaxyproject.org
-
Björn Grüning -
Sebastian Luna-Valero