Data library and symbolic links
Hi all, I created data libraries using options "Upload files from filesystem paths" and "_Link to files without copying into Galaxy_". It worked but I use symbolic links because my data are frequently updated The problem is that Galaxy takes the real link (inside "external_filename" column in "dataset" table) and doesn't find the files when it is deleted. For example, I uploaded a dataset for a data library without copying : /databank/uniprot/current/fasta/all.fasta -> /databank/uniprot/01_12_2012/fasta/all.fasta The real path is put in the database and it's a problem when it is deleted. Is there a way for galaxy to take the symbolic link directly in the database instead of the real path. Thanks in advance! Cheers, -- Cyril Monjeaud Equipe Symbiose / Plate-forme GenOuest Bureau D156 IRISA-INRIA, Campus de Beaulieu 35042 Rennes cedex, France Tél: +33 (0) 2 99 84 74 17
Hello, The original question/reply is posted here: http://lists.bx.psu.edu/pipermail/galaxy-user/2012-December/005665.html But the dev community is welcomed to add in additional comments/advice to this re-post! Thanks, Jen Galaxy team On 12/14/12 6:49 AM, MONJEAUD wrote:
Hi all,
I created data libraries using options "Upload files from filesystem paths" and "_Link to files without copying into Galaxy_". It worked but I use symbolic links because my data are frequently updated
The problem is that Galaxy takes the real link (inside "external_filename" column in "dataset" table) and doesn't find the files when it is deleted.
For example, I uploaded a dataset for a data library without copying : /databank/uniprot/current/fasta/all.fasta -> /databank/uniprot/01_12_2012/fasta/all.fasta The real path is put in the database and it's a problem when it is deleted.
Is there a way for galaxy to take the symbolic link directly in the database instead of the real path.
Thanks in advance! Cheers, --
Cyril Monjeaud Equipe Symbiose / Plate-forme GenOuest Bureau D156 IRISA-INRIA, Campus de Beaulieu 35042 Rennes cedex, France Tél: +33 (0) 2 99 84 74 17
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson http://galaxyproject.org
Hi Cyril, Jen's response includes the correct information about how symlinks are handled, and why. Changing the symlink underlying a dataset is not a great idea since it will cause the dataset's metadata to be incorrect and break Galaxy's concept of reproducibility. I would suggest that you utilize the library dataset versioning features to accomplish what you're trying to do. Hope this helps, --nate On Dec 14, 2012, at 10:07 AM, Jennifer Jackson wrote:
Hello,
The original question/reply is posted here: http://lists.bx.psu.edu/pipermail/galaxy-user/2012-December/005665.html
But the dev community is welcomed to add in additional comments/advice to this re-post!
Thanks,
Jen Galaxy team
On 12/14/12 6:49 AM, MONJEAUD wrote:
Hi all,
I created data libraries using options "Upload files from filesystem paths" and "Link to files without copying into Galaxy". It worked but I use symbolic links because my data are frequently updated
The problem is that Galaxy takes the real link (inside "external_filename" column in "dataset" table) and doesn't find the files when it is deleted.
For example, I uploaded a dataset for a data library without copying : /databank/uniprot/current/fasta/all.fasta -> /databank/uniprot/01_12_2012/fasta/all.fasta The real path is put in the database and it's a problem when it is deleted.
Is there a way for galaxy to take the symbolic link directly in the database instead of the real path.
Thanks in advance! Cheers, --
Cyril Monjeaud Equipe Symbiose / Plate-forme GenOuest Bureau D156 IRISA-INRIA, Campus de Beaulieu 35042 Rennes cedex, France Tél: +33 (0) 2 99 84 74 17
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson
http://galaxyproject.org ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Thanks for your answer!!! Cyril On 12/14/2012 04:17 PM, Nate Coraor wrote:
Hi Cyril,
Jen's response includes the correct information about how symlinks are handled, and why. Changing the symlink underlying a dataset is not a great idea since it will cause the dataset's metadata to be incorrect and break Galaxy's concept of reproducibility. I would suggest that you utilize the library dataset versioning features to accomplish what you're trying to do.
Hope this helps, --nate
On Dec 14, 2012, at 10:07 AM, Jennifer Jackson wrote:
Hello,
The original question/reply is posted here: http://lists.bx.psu.edu/pipermail/galaxy-user/2012-December/005665.html
But the dev community is welcomed to add in additional comments/advice to this re-post!
Thanks,
Jen Galaxy team
On 12/14/12 6:49 AM, MONJEAUD wrote:
Hi all,
I created data libraries using options "Upload files from filesystem paths" and "Link to files without copying into Galaxy". It worked but I use symbolic links because my data are frequently updated
The problem is that Galaxy takes the real link (inside "external_filename" column in "dataset" table) and doesn't find the files when it is deleted.
For example, I uploaded a dataset for a data library without copying : /databank/uniprot/current/fasta/all.fasta -> /databank/uniprot/01_12_2012/fasta/all.fasta The real path is put in the database and it's a problem when it is deleted.
Is there a way for galaxy to take the symbolic link directly in the database instead of the real path.
Thanks in advance! Cheers, --
Cyril Monjeaud Equipe Symbiose / Plate-forme GenOuest Bureau D156 IRISA-INRIA, Campus de Beaulieu 35042 Rennes cedex, France Tél: +33 (0) 2 99 84 74 17
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/ -- Jennifer Jackson
http://galaxyproject.org ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Cyril Monjeaud Equipe Symbiose / Plate-forme GenOuest Bureau D156 IRISA-INRIA, Campus de Beaulieu 35042 Rennes cedex, France Tél: +33 (0) 2 99 84 74 17
participants (3)
-
Jennifer Jackson -
MONJEAUD -
Nate Coraor