Hi Vlad

• I don't remember who mentioned a/several project(s) that synchronise data libraries with a given folder?

well, since I did not attend yesterday's meeting, I couldn't mention it...

In our case, all fastq files generated by our sequencing facility are linked (not copied) to Data Libraries. There is a python script (running every night) checking the facilitiy's LIMS for any new data. And then based on the information in the LIMS, several calls to the Galaxy API generate new sub-folders and link the data.

So it is a very customized solution fitting our local LIMS, storage set-up, etc.

Regards, Hans-Rudolf

Hi Vlad,

it was me who mentioned the library sync. I've heard of similar cases that Hans-Rudolf is describing and some more sophisticated ones that tried to do bi-directional sync. I couldn't find a link or any more information so I asked in one of the Galaxy channels -- maybe someone has a better memory ( link https://matrix.to/#/!fVQwotRzniGGJbeUUi:gitter.im/$-y1pfvymkrrvGXOvxhVwVky4ZWdNOI1lGM94iZFztEU?via=gitter.im&via=matrix.org). The thing is, that it is likely always going to be very custom and heavily based on what your other systems look like. Long time ago we had attempted to implement more full-featured LIMS in Galaxy, but that has long been gone since it was always just too general.

Cheers, Martin

On Fri, Jul 19, 2024 at 3:03 PM VLAD VISAN < vlad.visan@univ-grenoble-alpes.fr> wrote:

Hi Hans, all,

Thank you for sharing your solution. I've been looking into the API and have used this endpoint https://usegalaxy.org/api/docs#/undocumented/post_api_libraries_datasets_ https://usegalaxy.org/api/docs#/undocumented/post_api_libraries_datasets_(strangely BioBlend seems to wrap around this less expressive one https://usegalaxy.org/api/docs#/undocumented/post_api_libraries__library_id__contents instead), very useful! If enough people have this synchronisation use-case, there could be a case for a Galaxy-provided link between a File Source (assuming your LIMS can be imported through one) and a Data Library.

Regards,

Vlad

---

*De: *"Hans-Rudolf Hotz" hrh@fmi.ch *À: *"VLAD VISAN" vlad.visan@univ-grenoble-alpes.fr, "small-scale-admins" small-scale-admins@lists.galaxyproject.org *Envoyé: *Vendredi 19 Juillet 2024 14:07:08 *Objet: *Re: [Small-scale-admins] Re: Today: Meeting of 'small scale servers' group

Hi Vlad

...

• I don't remember who mentioned a/several project(s) that synchronise data libraries with a given folder?

well, since I did not attend yesterday's meeting, I couldn't mention it...

In our case, all fastq files generated by our sequencing facility are linked (not copied) to Data Libraries. There is a python script (running every night) checking the facilitiy's LIMS for any new data. And then based on the information in the LIMS, several calls to the Galaxy API generate new sub-folders and link the data.

So it is a very customized solution fitting our local LIMS, storage set-up, etc.

Regards, Hans-Rudolf _______________________________________________ Small-scale-admins mailing list -- small-scale-admins@lists.galaxyproject.org To unsubscribe send an email to small-scale-admins-leave@lists.galaxyproject.org

Hi all,

Thank you Martin and Helena for the information. From [ https://github.com/usegalaxy-eu/shared-data | https://github.com/usegalaxy-eu/shared-data ] I found a few useful links:

* [ https://ephemeris.readthedocs.io/en/latest/commands/setup-data-libraries.htm... | a util tool ] to populate a data library from the command line * [ https://github.com/galaxyproject/training-material/blob/3a1c6f78aae2e4b7b010... | an example of the format ] expected for the above tool

Regards,

Vlad

De: "Helena Rasche" helena.rasche@gmail.com À: "Martin Čech" cech.marten@gmail.com Cc: "VLAD VISAN" vlad.visan@univ-grenoble-alpes.fr, "Hans-Rudolf Hotz" hrh@fmi.ch, "small-scale-admins" small-scale-admins@lists.galaxyproject.org Envoyé: Vendredi 19 Juillet 2024 16:50:52 Objet: Re: [Small-scale-admins] Re: Today: Meeting of 'small scale servers' group

Hi Vlad, all,

[ https://github.com/usegalaxy-eu/shared-data | https://github.com/usegalaxy-eu/shared-data ] was one attempt at uni-directional sync, yaml files describing data / URLs are kept continually in sync with target galaxies. Maybe not something very reusable but potentially useful as inspiration?

mvg Helena On Fri, 19 Jul 2024, 15:38 Martin Čech, < [ mailto:cech.marten@gmail.com | cech.marten@gmail.com ] > wrote:

Hi Vlad, it was me who mentioned the library sync. I've heard of similar cases that Hans-Rudolf is describing and some more sophisticated ones that tried to do bi-directional sync. I couldn't find a link or any more information so I asked in one of the Galaxy channels -- maybe someone has a better memory ( [ https://matrix.to/#/!fVQwotRzniGGJbeUUi:gitter.im/$-y1pfvymkrrvGXOvxhVwVky4Z... | link ] ). The thing is, that it is likely always going to be very custom and heavily based on what your other systems look like. Long time ago we had attempted to implement more full-featured LIMS in Galaxy, but that has long been gone since it was always just too general.

Cheers, Martin

On Fri, Jul 19, 2024 at 3:03 PM VLAD VISAN < [ mailto:vlad.visan@univ-grenoble-alpes.fr | vlad.visan@univ-grenoble-alpes.fr ] > wrote:

BQ_BEGIN

Hi Hans, all,

Thank you for sharing your solution. I've been looking into the API and have used [ https://usegalaxy.org/api/docs#/undocumented/post_api_libraries_datasets_ | this endpoint ] [ https://usegalaxy.org/api/docs#/undocumented/post_api_libraries_datasets_ ] (strangely BioBlend seems to wrap around [ https://usegalaxy.org/api/docs#/undocumented/post_api_libraries__library_id_... | this less expressive one ] instead), very useful! If enough people have this synchronisation use-case, there could be a case for a Galaxy-provided link between a File Source (assuming your LIMS can be imported through one) and a Data Library.

Regards,

Vlad

De: "Hans-Rudolf Hotz" < [ mailto:hrh@fmi.ch | hrh@fmi.ch ] > À: "VLAD VISAN" < [ mailto:vlad.visan@univ-grenoble-alpes.fr | vlad.visan@univ-grenoble-alpes.fr ] >, "small-scale-admins" < [ mailto:small-scale-admins@lists.galaxyproject.org | small-scale-admins@lists.galaxyproject.org ] > Envoyé: Vendredi 19 Juillet 2024 14:07:08 Objet: Re: [Small-scale-admins] Re: Today: Meeting of 'small scale servers' group

Hi Vlad

• I don't remember who mentioned a/several project(s) that synchronise

data libraries with a given folder?

well, since I did not attend yesterday's meeting, I couldn't mention it...

In our case, all fastq files generated by our sequencing facility are linked (not copied) to Data Libraries. There is a python script (running every night) checking the facilitiy's LIMS for any new data. And then based on the information in the LIMS, several calls to the Galaxy API generate new sub-folders and link the data.

So it is a very customized solution fitting our local LIMS, storage set-up, etc.

Regards, Hans-Rudolf _______________________________________________ Small-scale-admins mailing list -- [ mailto:small-scale-admins@lists.galaxyproject.org | small-scale-admins@lists.galaxyproject.org ] To unsubscribe send an email to [ mailto:small-scale-admins-leave@lists.galaxyproject.org | small-scale-admins-leave@lists.galaxyproject.org ]

_______________________________________________ Small-scale-admins mailing list -- [ mailto:small-scale-admins@lists.galaxyproject.org | small-scale-admins@lists.galaxyproject.org ] To unsubscribe send an email to [ mailto:small-scale-admins-leave@lists.galaxyproject.org | small-scale-admins-leave@lists.galaxyproject.org ]

BQ_END