Hi Jochen,
Yes, those new directories would coincide with when you passed 100,000 datasets on your instance, so congrats! You can see the code that builds this hashed path here: https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/util/__init__.py...
So, for example, dataset id `99999` Will be in folder 'datasets/099/'. Dataset '100000' will be in 'datasets/000/100/'.
Sorry for the slow response, hope this clears up the storage path logic!
-Dannon
On Tue, May 21, 2019 at 11:38 AM Jochen Bick jochen.bick@usys.ethz.ch wrote:
Coming back to my question:
this is how an ls -ls looks in my database folder...
- Apr 14:40 000/
- Nov 2015 005/
- Nov 2018 007/
- Nov 2018 008/
- Dez 2015 010/
- Jan 2017 012/
- Jan 2017 013/
- Jun 2017 014/
- Jul 2017 015/
- Jan 2018 018/
- Jun 2017 020/
- Sep 2016 021/
- Mär 2018 032/
- Jan 09:38 033/
- Jan 10:32 034/
- Jul 2018 035/
- Okt 2018 036/
- Okt 2018 037/
- Okt 2018 038/
- Okt 2018 039/
- Aug 2018 070/
- Aug 2018 071/
- Aug 2018 072/
- Aug 2018 073/
- Aug 2018 074/
- Aug 2018 075/
- Aug 2018 076/
- Aug 2018 077/
- Aug 2018 078/
- Aug 2018 079/
- Aug 2018 080/
- Aug 2018 081/
- Aug 2018 082/
- Aug 2018 083/
- Aug 2018 084/
- Aug 2018 085/
- Aug 2018 086/
- Aug 2018 087/
- Aug 2018 088/
- Aug 2018 089/
- Aug 2018 090/
- Aug 2018 091/
- Aug 2018 092/
- Aug 2018 093/
- Aug 2018 094/
- Aug 2018 095/
- Aug 2018 096/
- Aug 2018 097/
- Sep 2018 098/
- Okt 2018 099/
if you look into the 000 folder it restarts:
- Dez 19:36 100/
- Nov 10:45 101/
- Dez 19:39 102/
- Dez 19:37 103/
- Jan 15:13 104/
- Jan 11:31 105/
- Jan 14:33 106/
- Feb 10:46 107/
- Feb 15:47 108/
- Mär 11:00 109/
- Apr 15:22 110/
- Mai 17:17 111/
is this the expected behavior?
The only thing we did was we archived some of the very early generated datasets. Is this maybe the problem?
Cheers Jochen
On 14.05.19 15:32, Jochen Bick wrote:
Hi,
today we found out that some of our files in your galaxy instance are new in "database/files/000/" which is the first folder of your database files. Could some please explain me how galaxy generates new file names? and how this is possible?
Thanks in advance Jochen
-- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland
Phone +41 52 354 92 06 jochen.bick@usys.ethz.ch mailto:jochen.bick@usys.ethz.ch www.ap.ethz.ch
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: %(web_page_url)s
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/