Hey Iyad,
This is an area under active development. You can track the progress of it on this card here: https://trello.com/c/nwYQNFPK/56-data-library-evolution
The new data libraries are *much* faster, and if you'd like to enable them for your local site. You should be able to see this in current versions of galaxy under Shared Data -> Data Libraries Beta.
-Dannon
On Thu, May 29, 2014 at 8:26 AM, Kandalaft, Iyad Iyad.Kandalaft@agr.gc.ca wrote:
Hello everyone
I have a lot of sequencing data that I wanted to load into one Galaxy data library. I enabled loading the data by filesystem path and successfully “linked to the files” rather than copying them into galaxy. Unfortunately, as I add more folders and data files, the library takes longer and longer to load. At this point, the apache proxy just times out before galaxy returns any data. Should I have more data libraries and fewer files within them? Is there a reason for not implementing paging in the data libraries?
The web server hosting galaxy shows only 80% loads
30473 galaxy 20 0 1070m 443m 5160 S 81.6 1.4 9:59.89 python
The database server is at 13% load (kind of low)
20890 mysql 20 0 3762m 551m 9104 S 13.4 0.9 796:16.69 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-bin=/var/log/mysql/bin.log --slow-query-log-file=/var/log/mysql/slow-queries.log
Regards,
Iyad Kandalaft
Bioinformatics Application Developer
Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
KW Neatby Bldg | éd. KW Neatby
960 Carling Ave| 960, avenue Carling
Ottawa, ON | Ottawa (ON) K1A 0C6
E-mail Address / Adresse courriel: Iyad.Kandalaft@agr.gc.ca
Telephone | Téléphone 613- 759-1228
Facsimile | Télécopieur 613-759-1701
Government of Canada | Gouvernement du Canada
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/