Hello everyone I have a lot of sequencing data that I wanted to load into one Galaxy data library. I enabled loading the data by filesystem path and successfully "linked to the files" rather than copying them into galaxy. Unfortunately, as I add more folders and data files, the library takes longer and longer to load. At this point, the apache proxy just times out before galaxy returns any data. Should I have more data libraries and fewer files within them? Is there a reason for not implementing paging in the data libraries? The web server hosting galaxy shows only 80% loads 30473 galaxy 20 0 1070m 443m 5160 S 81.6 1.4 9:59.89 python The database server is at 13% load (kind of low) 20890 mysql 20 0 3762m 551m 9104 S 13.4 0.9 796:16.69 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-bin=/var/log/mysql/bin.log --slow-query-log-file=/var/log/mysql/slow-queries.log Regards, Iyad Kandalaft Bioinformatics Application Developer Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada KW Neatby Bldg | éd. KW Neatby 960 Carling Ave| 960, avenue Carling Ottawa, ON | Ottawa (ON) K1A 0C6 E-mail Address / Adresse courriel: Iyad.Kandalaft@agr.gc.ca<mailto:Iyad.Kandalaft@agr.gc.ca> Telephone | Téléphone 613- 759-1228 Facsimile | Télécopieur 613-759-1701 Government of Canada | Gouvernement du Canada
Hey Iyad, This is an area under active development. You can track the progress of it on this card here: https://trello.com/c/nwYQNFPK/56-data-library-evolution The new data libraries are *much* faster, and if you'd like to enable them for your local site. You should be able to see this in current versions of galaxy under Shared Data -> Data Libraries Beta. -Dannon On Thu, May 29, 2014 at 8:26 AM, Kandalaft, Iyad <Iyad.Kandalaft@agr.gc.ca> wrote:
Hello everyone
I have a lot of sequencing data that I wanted to load into one Galaxy data library. I enabled loading the data by filesystem path and successfully “linked to the files” rather than copying them into galaxy. Unfortunately, as I add more folders and data files, the library takes longer and longer to load. At this point, the apache proxy just times out before galaxy returns any data. Should I have more data libraries and fewer files within them? Is there a reason for not implementing paging in the data libraries?
The web server hosting galaxy shows only 80% loads
30473 galaxy 20 0 1070m 443m 5160 S 81.6 1.4 9:59.89 python
The database server is at 13% load (kind of low)
20890 mysql 20 0 3762m 551m 9104 S 13.4 0.9 796:16.69 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-bin=/var/log/mysql/bin.log --slow-query-log-file=/var/log/mysql/slow-queries.log
Regards,
Iyad Kandalaft
Bioinformatics Application Developer
Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
KW Neatby Bldg | éd. KW Neatby
960 Carling Ave| 960, avenue Carling
Ottawa, ON | Ottawa (ON) K1A 0C6
E-mail Address / Adresse courriel: Iyad.Kandalaft@agr.gc.ca
Telephone | Téléphone 613- 759-1228
Facsimile | Télécopieur 613-759-1701
Government of Canada | Gouvernement du Canada
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (2)
-
Dannon Baker -
Kandalaft, Iyad