Hey Iyad,

This is an area under active development.  You can track the progress of it on this card here:  https://trello.com/c/nwYQNFPK/56-data-library-evolution

The new data libraries are *much* faster, and if you'd like to enable them for your local site.  You should be able to see this in current versions of galaxy under Shared Data -> Data Libraries Beta.

-Dannon


On Thu, May 29, 2014 at 8:26 AM, Kandalaft, Iyad <Iyad.Kandalaft@agr.gc.ca> wrote:

Hello everyone

 

I have a lot of sequencing data that I wanted to load into one Galaxy data library.  I enabled loading the data by filesystem path and successfully “linked to the files” rather than copying them into galaxy.  Unfortunately, as I add more folders and data files, the library takes longer and longer to load.  At this point, the apache proxy just times out before galaxy returns any data.  Should I have more data libraries and fewer files within them?  Is there a reason for not implementing paging in the data libraries?

 

The web server hosting galaxy shows only 80% loads

30473 galaxy    20   0 1070m 443m 5160 S 81.6  1.4   9:59.89 python

 

The database server is at 13% load (kind of low)

20890 mysql     20   0 3762m 551m 9104 S 13.4  0.9 796:16.69 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-bin=/var/log/mysql/bin.log --slow-query-log-file=/var/log/mysql/slow-queries.log

 

 

Regards,

Iyad Kandalaft

 

Bioinformatics Application Developer

Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada

KW Neatby Bldg | éd. KW Neatby 

960 Carling Ave| 960, avenue Carling

Ottawa, ON | Ottawa (ON) K1A 0C6

E-mail Address / Adresse courriel: Iyad.Kandalaft@agr.gc.ca

Telephone | Téléphone 613- 759-1228

Facsimile | Télécopieur 613-759-1701

Government of Canada | Gouvernement du Canada

 


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/