Re: [galaxy-dev] Shared Data Library, large number of files
Can you please share your '/client/galaxy/scripts/layout/menu.js' file with me? (e.g. using https://gist.github.com or similar). thanks M. On Wed, Mar 8, 2017 at 1:34 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I've tried this on both v16.07 and v16.10
D
On Wed, Mar 8, 2017 at 10:29 AM, Martin Čech <marten@bx.psu.edu> wrote:
Daniel:
what version is your Galaxy?
Martin
On Wed, Mar 8, 2017 at 12:40 PM D K <danielfortin86@gmail.com> wrote:
I've copied and pasted the javascript error below. Any help would be greatly appreciated. Thanks!
Uncaught TypeError: Cannot read property 'masthead' of undefined at n.<anonymous> (menu.js:120) at Function.each (jquery.js:370) at HTMLDocument.<anonymous> (menu.js:117) at l (jquery.js:3232) at Object.add [as done] (jquery.js:3291) at t.fn.init.vt.fn.ready (jquery.js:3542) at callback (menu.js:116) at Object.t.success (backbone.js:1051) at l (jquery.js:3232) at Object.fireWith [as resolveWith] (jquery.js:3362)
On Thu, Mar 2, 2017 at 10:43 AM, D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I do get a javascript error, I've attached a screenshot of the relevant window. I also tried logging in as a non-admin user and had the same issue. In this Galaxy instance are 5 data libraries, each with about 150 individual folders, and each folder has a variable number of fastq.gz files.
Thanks!
On Thu, Mar 2, 2017 at 10:13 AM, Martin Čech <marten@bx.psu.edu> wrote:
Daniel:
Could you please open browser console and look for javascript errors on the page that fails to load for you? Additionally I assume you are an admin when trying this (admins load all of the instance's libraries) could you try loading the library list as a normal user? How many libraries do you have on the instance?
Thanks
Martin
On Thu, Mar 2, 2017 at 12:38 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin / Hans / Devon,
I can confirm that I am using the new UI from the top menu. Right now when I click on the link I often do not even get the list of libraries whereas when I click on the admin Data Library link I at least see the list of libraries (although I can't open any of them since they are so large). I've tried refreshing the page, deleting the cookies for the session and using Incognito mode, using chrome and firefox both from a linux and windows machine and can confirm this behavior on all systems. However, occasionally when clicking the Shared Data Library link from the top I do get a list of libraries and can navigate them, but this has only happened once during my tests and I cannot replicate this behavior.
Thanks for the assistance!
On Thu, Mar 2, 2017 at 5:47 AM, Martin Čech <marten@bx.psu.edu> wrote:
Hi all,
*Devon* is correct -the old data library UI under Admin menu is still in place (because it has some features perceived as important to some admins). However majority of users and most of the admins are using the interface under 'Shared Data' I hope and the old UI is deprecated since 15.10 ( https://docs.galaxyproject.org/en/master/releases/15.10_announce.html). I created a PR to adjust the label in the admin interface to make this clear: https://github.com/galaxyproject/galaxy/pull/3694
The old UI is very slow for large libraries but the new UI under Shared Data/Data Libraries in the top menu shouldn't have this kind of issues for hundreds of folders/datasets that are mentioned here.
*Daniel* are you using the new UI form the top menu? If so are you experiencing this only with a specific folder full of linked datasets or does it occur for any folder with many datasets in it?
Martin
On Thu, Mar 2, 2017 at 3:49 AM Devon Ryan <dpryan@dpryan.com> wrote:
I observe the same thing on 16.07. As an example, I'm just opened a data library under Admin->Data->Data Libraries and this took about a minute and a half. It looks like this is related to the quantity of data in this folder. The associated entry in the uwsgi log is as follows:
[pid: 357|app: 0|req: 2059/5647] 10.1.6.77 () {44 vars in 1220 bytes} [Thu Mar 2 08:40:19 2017] GET
/library_common/browse_library?sort=name&f-description=All&f-name=All&f-deleted=False&cntrller=library_admin&operation=browse&id=355326571f7132ff => generated 21761790 bytes in 108644 msecs (HTTP/1.1 200) 2 headers in 73 bytes (14 switches on core 2)
So this library is causing ~21 megs of data to be returned to the browser, which I presume is the source of the issue.
As an aside, the old data library API is still in use in the admin section until at least 16.07. Perhaps this has been changed since then, I hadn't planed to update our system until the end of the month. For what it's worth, opening the same library via "shared data" takes ~100 msec.
Devon -- Devon Ryan, Ph.D. Email: dpryan@dpryan.com Data Manager/Bioinformatician Max Planck Institute of Immunobiology and Epigenetics Stübeweg 51 79108 Freiburg Germany
On Thu, Mar 2, 2017 at 8:30 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
this sounds more like a browser issue to me which browser do you use? what happens when you use a different browser?
and
what happens when you just reload the page
Hans-Rudolf
On 03/01/2017 06:17 PM, D K wrote:
Actually, when I click on the Shared Data-> Data Libraries link I get only the Galaxy banner and a blank page, so I'm not even seeing the data libraries themselves.
If I use the Admin link and then click Data libraries, I can see the actual libraries. But when I try to enter one of the libraries with lots of files it hangs.
On Wed, Mar 1, 2017 at 9:03 AM, D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
Hi Hans,
I'm running 16.10. If I'm not mistaken, the old Data Library interface is still accessible in the Admin menu isn't it? I have the same issue when navigating to a Data Library either from the Admin or Shared Data links. I also tried this in 16.07 with the same result. If it makes a difference, I'm using PostgreSQL and Apache.
Dwight
On Tue, Feb 28, 2017 at 11:47 PM, Hans-Rudolf Hotz <hrh@fmi.ch <mailto:hrh@fmi.ch>> wrote:
Hi Daniel
Just double checking which version of galaxy are you running? In other words: are you still using the old Data Library interface? (if I remember correctly, the old UI was deprecated in 15.10) We had similar issues with the old UI.
The new one works fine and is very fast. E.g. we have a 'MISEQ' folder with more than 100 sub folders and a HISEQ folder with more than 600 sub folders. Each containing up to 100plus fastq and text files.
Regards, Hans-Rudolf
On 03/01/2017 12:13 AM, D K wrote:
Hi,
I've noticed that the more files / folders (think hundreds of folders each with 20+ fastq files) I add to a Shared Data Library, the longer and longer it takes to initially navigate the folder structure (after clicking on the specific Shared Data Library). Just opening up the Shared Data Library can take several minutes and often the browser times out. Is there a way to speed this up?
If it's relevant, when I add shared data, it's linked rather than copied into Galaxy. Any suggestions would be greatly appreciated
Thanks!
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/>
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ <http://galaxyproject.org/search/mailinglists/>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Martin, I've pasted the file you requested here: http://collabedit.com/xcx5b Thanks! On Wed, Mar 8, 2017 at 11:04 AM, Martin Čech <marten@bx.psu.edu> wrote:
Can you please share your '/client/galaxy/scripts/layout/menu.js' file with me? (e.g. using https://gist.github.com or similar).
thanks
M.
On Wed, Mar 8, 2017 at 1:34 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I've tried this on both v16.07 and v16.10
D
On Wed, Mar 8, 2017 at 10:29 AM, Martin Čech <marten@bx.psu.edu> wrote:
Daniel:
what version is your Galaxy?
Martin
On Wed, Mar 8, 2017 at 12:40 PM D K <danielfortin86@gmail.com> wrote:
I've copied and pasted the javascript error below. Any help would be greatly appreciated. Thanks!
Uncaught TypeError: Cannot read property 'masthead' of undefined at n.<anonymous> (menu.js:120) at Function.each (jquery.js:370) at HTMLDocument.<anonymous> (menu.js:117) at l (jquery.js:3232) at Object.add [as done] (jquery.js:3291) at t.fn.init.vt.fn.ready (jquery.js:3542) at callback (menu.js:116) at Object.t.success (backbone.js:1051) at l (jquery.js:3232) at Object.fireWith [as resolveWith] (jquery.js:3362)
On Thu, Mar 2, 2017 at 10:43 AM, D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I do get a javascript error, I've attached a screenshot of the relevant window. I also tried logging in as a non-admin user and had the same issue. In this Galaxy instance are 5 data libraries, each with about 150 individual folders, and each folder has a variable number of fastq.gz files.
Thanks!
On Thu, Mar 2, 2017 at 10:13 AM, Martin Čech <marten@bx.psu.edu> wrote:
Daniel:
Could you please open browser console and look for javascript errors on the page that fails to load for you? Additionally I assume you are an admin when trying this (admins load all of the instance's libraries) could you try loading the library list as a normal user? How many libraries do you have on the instance?
Thanks
Martin
On Thu, Mar 2, 2017 at 12:38 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin / Hans / Devon,
I can confirm that I am using the new UI from the top menu. Right now when I click on the link I often do not even get the list of libraries whereas when I click on the admin Data Library link I at least see the list of libraries (although I can't open any of them since they are so large). I've tried refreshing the page, deleting the cookies for the session and using Incognito mode, using chrome and firefox both from a linux and windows machine and can confirm this behavior on all systems. However, occasionally when clicking the Shared Data Library link from the top I do get a list of libraries and can navigate them, but this has only happened once during my tests and I cannot replicate this behavior.
Thanks for the assistance!
On Thu, Mar 2, 2017 at 5:47 AM, Martin Čech <marten@bx.psu.edu> wrote:
Hi all,
*Devon* is correct -the old data library UI under Admin menu is still in place (because it has some features perceived as important to some admins). However majority of users and most of the admins are using the interface under 'Shared Data' I hope and the old UI is deprecated since 15.10 ( https://docs.galaxyproject.org/en/master/releases/15.10_announce.html). I created a PR to adjust the label in the admin interface to make this clear: https://github.com/galaxyproject/galaxy/pull/3694
The old UI is very slow for large libraries but the new UI under Shared Data/Data Libraries in the top menu shouldn't have this kind of issues for hundreds of folders/datasets that are mentioned here.
*Daniel* are you using the new UI form the top menu? If so are you experiencing this only with a specific folder full of linked datasets or does it occur for any folder with many datasets in it?
Martin
On Thu, Mar 2, 2017 at 3:49 AM Devon Ryan <dpryan@dpryan.com> wrote:
I observe the same thing on 16.07. As an example, I'm just opened a data library under Admin->Data->Data Libraries and this took about a minute and a half. It looks like this is related to the quantity of data in this folder. The associated entry in the uwsgi log is as follows:
[pid: 357|app: 0|req: 2059/5647] 10.1.6.77 () {44 vars in 1220 bytes} [Thu Mar 2 08:40:19 2017] GET /library_common/browse_library?sort=name&f-description=All&f-name=All&f- deleted=False&cntrller=library_admin&operation=browse&id=355326571f7132ff => generated 21761790 bytes in 108644 msecs (HTTP/1.1 200) 2 headers in 73 bytes (14 switches on core 2)
So this library is causing ~21 megs of data to be returned to the browser, which I presume is the source of the issue.
As an aside, the old data library API is still in use in the admin section until at least 16.07. Perhaps this has been changed since then, I hadn't planed to update our system until the end of the month. For what it's worth, opening the same library via "shared data" takes ~100 msec.
Devon -- Devon Ryan, Ph.D. Email: dpryan@dpryan.com Data Manager/Bioinformatician Max Planck Institute of Immunobiology and Epigenetics Stübeweg 51 79108 Freiburg Germany
On Thu, Mar 2, 2017 at 8:30 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
this sounds more like a browser issue to me which browser do you use? what happens when you use a different
what happens when you just reload the page
Hans-Rudolf
On 03/01/2017 06:17 PM, D K wrote:
Actually, when I click on the Shared Data-> Data Libraries link I get only the Galaxy banner and a blank page, so I'm not even seeing the
data
libraries themselves.
If I use the Admin link and then click Data libraries, I can see the actual libraries. But when I try to enter one of the libraries with lots of files it hangs.
On Wed, Mar 1, 2017 at 9:03 AM, D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
Hi Hans,
I'm running 16.10. If I'm not mistaken, the old Data Library interface is still accessible in the Admin menu isn't it? I have
browser? and the
same issue when navigating to a Data Library either from the Admin or Shared Data links. I also tried this in 16.07 with the same result. If it makes a difference, I'm using PostgreSQL and Apache.
Dwight
On Tue, Feb 28, 2017 at 11:47 PM, Hans-Rudolf Hotz <hrh@fmi.ch <mailto:hrh@fmi.ch>> wrote:
Hi Daniel
Just double checking which version of galaxy are you running?
In
other words: are you still using the old Data Library
interface?
(if I remember correctly, the old UI was deprecated in 15.10) We had similar issues with the old UI.
The new one works fine and is very fast. E.g. we have a 'MISEQ' folder with more than 100 sub folders and a HISEQ folder with more than 600 sub folders. Each containing up to 100plus fastq and text files.
Regards, Hans-Rudolf
On 03/01/2017 12:13 AM, D K wrote:
Hi,
I've noticed that the more files / folders (think hundreds of folders each with 20+ fastq files) I add to a Shared Data Library, the longer and longer it takes to initially navigate the folder structure (after clicking on the specific Shared Data Library). Just opening up the Shared Data Library can take several minutes and often the browser times out. Is there a way to speed this up?
If it's relevant, when I add shared data, it's linked
rather
than copied into Galaxy. Any suggestions would be greatly appreciated
Thanks!
_____________________________
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/>
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ <http://galaxyproject.org/search/mailinglists/>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
The code at L120 does not exist in 16.07. It was broken in 16.10 and fixed for 17.01. I made a PR to backport the fix to 16.10: https://github.com/galaxyproject/galaxy/pull/3728 You shouldn't be seeing the 'Cannot read property 'masthead' of undefined' error in 16.07 and it should be fixed in 16.10 after that PR is merged. 17.01 or dev shouldn't have this problem already. Please let me know how this works for you. Thanks, Martin On Wed, Mar 8, 2017 at 3:09 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I've pasted the file you requested here:
Thanks!
On Wed, Mar 8, 2017 at 11:04 AM, Martin Čech <marten@bx.psu.edu> wrote:
Can you please share your '/client/galaxy/scripts/layout/menu.js' file with me? (e.g. using https://gist.github.com or similar).
thanks
M.
On Wed, Mar 8, 2017 at 1:34 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I've tried this on both v16.07 and v16.10
D
On Wed, Mar 8, 2017 at 10:29 AM, Martin Čech <marten@bx.psu.edu> wrote:
Daniel:
what version is your Galaxy?
Martin
On Wed, Mar 8, 2017 at 12:40 PM D K <danielfortin86@gmail.com> wrote:
I've copied and pasted the javascript error below. Any help would be greatly appreciated. Thanks!
Uncaught TypeError: Cannot read property 'masthead' of undefined at n.<anonymous> (menu.js:120) at Function.each (jquery.js:370) at HTMLDocument.<anonymous> (menu.js:117) at l (jquery.js:3232) at Object.add [as done] (jquery.js:3291) at t.fn.init.vt.fn.ready (jquery.js:3542) at callback (menu.js:116) at Object.t.success (backbone.js:1051) at l (jquery.js:3232) at Object.fireWith [as resolveWith] (jquery.js:3362)
On Thu, Mar 2, 2017 at 10:43 AM, D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I do get a javascript error, I've attached a screenshot of the relevant window. I also tried logging in as a non-admin user and had the same issue. In this Galaxy instance are 5 data libraries, each with about 150 individual folders, and each folder has a variable number of fastq.gz files.
Thanks!
On Thu, Mar 2, 2017 at 10:13 AM, Martin Čech <marten@bx.psu.edu> wrote:
Daniel:
Could you please open browser console and look for javascript errors on the page that fails to load for you? Additionally I assume you are an admin when trying this (admins load all of the instance's libraries) could you try loading the library list as a normal user? How many libraries do you have on the instance?
Thanks
Martin
On Thu, Mar 2, 2017 at 12:38 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin / Hans / Devon,
I can confirm that I am using the new UI from the top menu. Right now when I click on the link I often do not even get the list of libraries whereas when I click on the admin Data Library link I at least see the list of libraries (although I can't open any of them since they are so large). I've tried refreshing the page, deleting the cookies for the session and using Incognito mode, using chrome and firefox both from a linux and windows machine and can confirm this behavior on all systems. However, occasionally when clicking the Shared Data Library link from the top I do get a list of libraries and can navigate them, but this has only happened once during my tests and I cannot replicate this behavior.
Thanks for the assistance!
On Thu, Mar 2, 2017 at 5:47 AM, Martin Čech <marten@bx.psu.edu> wrote:
Hi all,
*Devon* is correct -the old data library UI under Admin menu is still in place (because it has some features perceived as important to some admins). However majority of users and most of the admins are using the interface under 'Shared Data' I hope and the old UI is deprecated since 15.10 ( https://docs.galaxyproject.org/en/master/releases/15.10_announce.html). I created a PR to adjust the label in the admin interface to make this clear: https://github.com/galaxyproject/galaxy/pull/3694
The old UI is very slow for large libraries but the new UI under Shared Data/Data Libraries in the top menu shouldn't have this kind of issues for hundreds of folders/datasets that are mentioned here.
*Daniel* are you using the new UI form the top menu? If so are you experiencing this only with a specific folder full of linked datasets or does it occur for any folder with many datasets in it?
Martin
On Thu, Mar 2, 2017 at 3:49 AM Devon Ryan <dpryan@dpryan.com> wrote:
I observe the same thing on 16.07. As an example, I'm just opened a data library under Admin->Data->Data Libraries and this took about a minute and a half. It looks like this is related to the quantity of data in this folder. The associated entry in the uwsgi log is as follows:
[pid: 357|app: 0|req: 2059/5647] 10.1.6.77 () {44 vars in 1220 bytes} [Thu Mar 2 08:40:19 2017] GET
/library_common/browse_library?sort=name&f-description=All&f-name=All&f-deleted=False&cntrller=library_admin&operation=browse&id=355326571f7132ff => generated 21761790 bytes in 108644 msecs (HTTP/1.1 200) 2 headers in 73 bytes (14 switches on core 2)
So this library is causing ~21 megs of data to be returned to the browser, which I presume is the source of the issue.
As an aside, the old data library API is still in use in the admin section until at least 16.07. Perhaps this has been changed since then, I hadn't planed to update our system until the end of the month. For what it's worth, opening the same library via "shared data" takes ~100 msec.
Devon -- Devon Ryan, Ph.D. Email: dpryan@dpryan.com Data Manager/Bioinformatician Max Planck Institute of Immunobiology and Epigenetics Stübeweg 51 79108 Freiburg Germany
On Thu, Mar 2, 2017 at 8:30 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
this sounds more like a browser issue to me which browser do you use? what happens when you use a different browser?
and
what happens when you just reload the page
Hans-Rudolf
On 03/01/2017 06:17 PM, D K wrote:
Actually, when I click on the Shared Data-> Data Libraries link I get only the Galaxy banner and a blank page, so I'm not even seeing the data libraries themselves.
If I use the Admin link and then click Data libraries, I can see the actual libraries. But when I try to enter one of the libraries with lots of files it hangs.
On Wed, Mar 1, 2017 at 9:03 AM, D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
Hi Hans,
I'm running 16.10. If I'm not mistaken, the old Data Library interface is still accessible in the Admin menu isn't it? I have the same issue when navigating to a Data Library either from the Admin or Shared Data links. I also tried this in 16.07 with the same result. If it makes a difference, I'm using PostgreSQL and Apache.
Dwight
On Tue, Feb 28, 2017 at 11:47 PM, Hans-Rudolf Hotz <hrh@fmi.ch <mailto:hrh@fmi.ch>> wrote:
Hi Daniel
Just double checking which version of galaxy are you running? In other words: are you still using the old Data Library interface? (if I remember correctly, the old UI was deprecated in 15.10) We had similar issues with the old UI.
The new one works fine and is very fast. E.g. we have a 'MISEQ' folder with more than 100 sub folders and a HISEQ folder with more than 600 sub folders. Each containing up to 100plus fastq and text files.
Regards, Hans-Rudolf
On 03/01/2017 12:13 AM, D K wrote:
Hi,
I've noticed that the more files / folders (think hundreds of folders each with 20+ fastq files) I add to a Shared Data Library, the longer and longer it takes to initially navigate the folder structure (after clicking on the specific Shared Data Library). Just opening up the Shared Data Library can take several minutes and often the browser times out. Is there a way to speed this up?
If it's relevant, when I add shared data, it's linked rather than copied into Galaxy. Any suggestions would be greatly appreciated
Thanks!
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/>
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ <http://galaxyproject.org/search/mailinglists/>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Martin, Finally got a chance to test out your PR. This does in fact fix the issue. Thanks again for your help! On Wed, Mar 8, 2017 at 12:19 PM, Martin Čech <marten@bx.psu.edu> wrote:
The code at L120 does not exist in 16.07. It was broken in 16.10 and fixed for 17.01. I made a PR to backport the fix to 16.10: https://github.com/ galaxyproject/galaxy/pull/3728
You shouldn't be seeing the 'Cannot read property 'masthead' of undefined' error in 16.07 and it should be fixed in 16.10 after that PR is merged. 17.01 or dev shouldn't have this problem already.
Please let me know how this works for you.
Thanks,
Martin
On Wed, Mar 8, 2017 at 3:09 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I've pasted the file you requested here:
Thanks!
On Wed, Mar 8, 2017 at 11:04 AM, Martin Čech <marten@bx.psu.edu> wrote:
Can you please share your '/client/galaxy/scripts/layout/menu.js' file with me? (e.g. using https://gist.github.com or similar).
thanks
M.
On Wed, Mar 8, 2017 at 1:34 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I've tried this on both v16.07 and v16.10
D
On Wed, Mar 8, 2017 at 10:29 AM, Martin Čech <marten@bx.psu.edu> wrote:
Daniel:
what version is your Galaxy?
Martin
On Wed, Mar 8, 2017 at 12:40 PM D K <danielfortin86@gmail.com> wrote:
I've copied and pasted the javascript error below. Any help would be greatly appreciated. Thanks!
Uncaught TypeError: Cannot read property 'masthead' of undefined at n.<anonymous> (menu.js:120) at Function.each (jquery.js:370) at HTMLDocument.<anonymous> (menu.js:117) at l (jquery.js:3232) at Object.add [as done] (jquery.js:3291) at t.fn.init.vt.fn.ready (jquery.js:3542) at callback (menu.js:116) at Object.t.success (backbone.js:1051) at l (jquery.js:3232) at Object.fireWith [as resolveWith] (jquery.js:3362)
On Thu, Mar 2, 2017 at 10:43 AM, D K <danielfortin86@gmail.com> wrote:
Hi Martin,
I do get a javascript error, I've attached a screenshot of the relevant window. I also tried logging in as a non-admin user and had the same issue. In this Galaxy instance are 5 data libraries, each with about 150 individual folders, and each folder has a variable number of fastq.gz files.
Thanks!
On Thu, Mar 2, 2017 at 10:13 AM, Martin Čech <marten@bx.psu.edu> wrote:
Daniel:
Could you please open browser console and look for javascript errors on the page that fails to load for you? Additionally I assume you are an admin when trying this (admins load all of the instance's libraries) could you try loading the library list as a normal user? How many libraries do you have on the instance?
Thanks
Martin
On Thu, Mar 2, 2017 at 12:38 PM D K <danielfortin86@gmail.com> wrote:
Hi Martin / Hans / Devon,
I can confirm that I am using the new UI from the top menu. Right now when I click on the link I often do not even get the list of libraries whereas when I click on the admin Data Library link I at least see the list of libraries (although I can't open any of them since they are so large). I've tried refreshing the page, deleting the cookies for the session and using Incognito mode, using chrome and firefox both from a linux and windows machine and can confirm this behavior on all systems. However, occasionally when clicking the Shared Data Library link from the top I do get a list of libraries and can navigate them, but this has only happened once during my tests and I cannot replicate this behavior.
Thanks for the assistance!
On Thu, Mar 2, 2017 at 5:47 AM, Martin Čech <marten@bx.psu.edu> wrote:
Hi all,
*Devon* is correct -the old data library UI under Admin menu is still in place (because it has some features perceived as important to some admins). However majority of users and most of the admins are using the interface under 'Shared Data' I hope and the old UI is deprecated since 15.10 ( https://docs.galaxyproject.org/en/master/releases/15.10_announce.html). I created a PR to adjust the label in the admin interface to make this clear: https://github.com/galaxyproject/galaxy/pull/3694
The old UI is very slow for large libraries but the new UI under Shared Data/Data Libraries in the top menu shouldn't have this kind of issues for hundreds of folders/datasets that are mentioned here.
*Daniel* are you using the new UI form the top menu? If so are you experiencing this only with a specific folder full of linked datasets or does it occur for any folder with many datasets in it?
Martin
On Thu, Mar 2, 2017 at 3:49 AM Devon Ryan <dpryan@dpryan.com> wrote:
I observe the same thing on 16.07. As an example, I'm just opened a data library under Admin->Data->Data Libraries and this took about a minute and a half. It looks like this is related to the quantity of data in this folder. The associated entry in the uwsgi log is as follows:
[pid: 357|app: 0|req: 2059/5647] 10.1.6.77 () {44 vars in 1220 bytes} [Thu Mar 2 08:40:19 2017] GET /library_common/browse_library?sort=name&f-description=All&f-name=All&f- deleted=False&cntrller=library_admin&operation=browse&id=355326571f7132ff => generated 21761790 bytes in 108644 msecs (HTTP/1.1 200) 2 headers in 73 bytes (14 switches on core 2)
So this library is causing ~21 megs of data to be returned to the browser, which I presume is the source of the issue.
As an aside, the old data library API is still in use in the admin section until at least 16.07. Perhaps this has been changed since then, I hadn't planed to update our system until the end of the month. For what it's worth, opening the same library via "shared data" takes ~100 msec.
Devon -- Devon Ryan, Ph.D. Email: dpryan@dpryan.com Data Manager/Bioinformatician Max Planck Institute of Immunobiology and Epigenetics Stübeweg 51 79108 Freiburg Germany
On Thu, Mar 2, 2017 at 8:30 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
this sounds more like a browser issue to me which browser do you use? what happens when you use a different
what happens when you just reload the page
Hans-Rudolf
On 03/01/2017 06:17 PM, D K wrote:
Actually, when I click on the Shared Data-> Data Libraries link I get only the Galaxy banner and a blank page, so I'm not even seeing the
data
libraries themselves.
If I use the Admin link and then click Data libraries, I can see the actual libraries. But when I try to enter one of the libraries with lots of files it hangs.
On Wed, Mar 1, 2017 at 9:03 AM, D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
Hi Hans,
I'm running 16.10. If I'm not mistaken, the old Data Library interface is still accessible in the Admin menu isn't it? I have
browser? and the
same issue when navigating to a Data Library either from the Admin or Shared Data links. I also tried this in 16.07 with the same result. If it makes a difference, I'm using PostgreSQL and Apache.
Dwight
On Tue, Feb 28, 2017 at 11:47 PM, Hans-Rudolf Hotz <hrh@fmi.ch <mailto:hrh@fmi.ch>> wrote:
Hi Daniel
Just double checking which version of galaxy are you running?
In
other words: are you still using the old Data Library
interface?
(if I remember correctly, the old UI was deprecated in 15.10) We had similar issues with the old UI.
The new one works fine and is very fast. E.g. we have a 'MISEQ' folder with more than 100 sub folders and a HISEQ folder with more than 600 sub folders. Each containing up to 100plus fastq and text files.
Regards, Hans-Rudolf
On 03/01/2017 12:13 AM, D K wrote:
Hi,
I've noticed that the more files / folders (think hundreds of folders each with 20+ fastq files) I add to a Shared Data Library, the longer and longer it takes to initially navigate the folder structure (after clicking on the specific Shared Data Library). Just opening up the Shared Data Library can take several minutes and often the browser times out. Is there a way to speed this up?
If it's relevant, when I add shared data, it's linked
rather
than copied into Galaxy. Any suggestions would be greatly appreciated
Thanks!
_____________________________
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/>
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ <http://galaxyproject.org/search/mailinglists/>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi, that PR has been merged so, if you are still on the release_16.10 branch, you can update your Galaxy instance with a `git pull` . Cheers, Nicola On 15/03/17 16:53, D K wrote:
Hi Martin,
Finally got a chance to test out your PR. This does in fact fix the issue. Thanks again for your help!
On Wed, Mar 8, 2017 at 12:19 PM, Martin Čech <marten@bx.psu.edu <mailto:marten@bx.psu.edu>> wrote:
The code at L120 does not exist in 16.07. It was broken in 16.10 and fixed for 17.01. I made a PR to backport the fix to 16.10: https://github.com/galaxyproject/galaxy/pull/3728 <https://github.com/galaxyproject/galaxy/pull/3728>
You shouldn't be seeing the 'Cannot read property 'masthead' of undefined' error in 16.07 and it should be fixed in 16.10 after that PR is merged. 17.01 or dev shouldn't have this problem already.
Please let me know how this works for you.
Thanks,
Martin
On Wed, Mar 8, 2017 at 3:09 PM D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
Hi Martin,
I've pasted the file you requested here:
Thanks!
On Wed, Mar 8, 2017 at 11:04 AM, Martin Čech <marten@bx.psu.edu <mailto:marten@bx.psu.edu>> wrote:
Can you please share your '/client/galaxy/scripts/layout/menu.js' file with me? (e.g. using https://gist.github.com or similar).
thanks
M.
On Wed, Mar 8, 2017 at 1:34 PM D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
Hi Martin,
I've tried this on both v16.07 and v16.10
D
On Wed, Mar 8, 2017 at 10:29 AM, Martin Čech <marten@bx.psu.edu <mailto:marten@bx.psu.edu>> wrote:
Daniel:
what version is your Galaxy?
Martin
On Wed, Mar 8, 2017 at 12:40 PM D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
I've copied and pasted the javascript error below. Any help would be greatly appreciated. Thanks!
Uncaught TypeError: Cannot read property 'masthead' of undefined at n.<anonymous> (menu.js:120) at Function.each (jquery.js:370) at HTMLDocument.<anonymous> (menu.js:117) at l (jquery.js:3232) at Object.add [as done] (jquery.js:3291) at t.fn.init.vt.fn.ready (jquery.js:3542) at callback (menu.js:116) at Object.t.success (backbone.js:1051) at l (jquery.js:3232) at Object.fireWith [as resolveWith] (jquery.js:3362)
On Thu, Mar 2, 2017 at 10:43 AM, D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
Hi Martin,
I do get a javascript error, I've attached a screenshot of the relevant window. I also tried logging in as a non-admin user and had the same issue. In this Galaxy instance are 5 data libraries, each with about 150 individual folders, and each folder has a variable number of fastq.gz files.
Thanks!
On Thu, Mar 2, 2017 at 10:13 AM, Martin Čech <marten@bx.psu.edu <mailto:marten@bx.psu.edu>> wrote:
Daniel:
Could you please open browser console and look for javascript errors on the page that fails to load for you? Additionally I assume you are an admin when trying this (admins load all of the instance's libraries) could you try loading the library list as a normal user? How many libraries do you have on the instance?
Thanks
Martin
On Thu, Mar 2, 2017 at 12:38 PM D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>> wrote:
Hi Martin / Hans / Devon,
I can confirm that I am using the new UI from the top menu. Right now when I click on the link I often do not even get the list of libraries whereas when I click on the admin Data Library link I at least see the list of libraries (although I can't open any of them since they are so large). I've tried refreshing the page, deleting the cookies for the session and using Incognito mode, using chrome and firefox both from a linux and windows machine and can confirm this behavior on all systems. However, occasionally when clicking the Shared Data Library link from the top I do get a list of libraries and can navigate them, but this has only happened once during my tests and I cannot replicate this behavior.
Thanks for the assistance!
On Thu, Mar 2, 2017 at 5:47 AM, Martin Čech <marten@bx.psu.edu <mailto:marten@bx.psu.edu>> wrote:
Hi all,
*Devon* is correct -the old data library UI under Admin menu is still in place (because it has some features perceived as important to some admins). However majority of users and most of the admins are using the interface under 'Shared Data' I hope and the old UI is deprecated since 15.10 (https://docs.galaxyproject.org/en/master/releases/15.10_announce.html <https://docs.galaxyproject.org/en/master/releases/15.10_announce.html>). I created a PR to adjust the label in the admin interface to make this clear: https://github.com/galaxyproject/galaxy/pull/3694 <https://github.com/galaxyproject/galaxy/pull/3694>
The old UI is very slow for large libraries but the new UI under Shared Data/Data Libraries in the top menu shouldn't have this kind of issues for hundreds of folders/datasets that are mentioned here.
*Daniel* are you using the new UI form the top menu? If so are you experiencing this only with a specific folder full of linked datasets or does it occur for any folder with many datasets in it?
Martin
On Thu, Mar 2, 2017 at 3:49 AM Devon Ryan <dpryan@dpryan.com <mailto:dpryan@dpryan.com>> wrote:
I observe the same thing on 16.07. As an example, I'm just opened a data library under Admin->Data->Data Libraries and this took about a minute and a half. It looks like this is related to the quantity of data in this folder. The associated entry in the uwsgi log is as follows:
[pid: 357|app: 0|req: 2059/5647] 10.1.6.77 () {44 vars in 1220 bytes} [Thu Mar 2 08:40:19 2017] GET /library_common/browse_library?sort=name&f-description=All&f-name=All&f-deleted=False&cntrller=library_admin&operation=browse&id=355326571f7132ff => generated 21761790 bytes in 108644 msecs (HTTP/1.1 200) 2 headers in 73 bytes (14 switches on core 2)
So this library is causing ~21 megs of data to be returned to the browser, which I presume is the source of the issue.
As an aside, the old data library API is still in use in the admin section until at least 16.07. Perhaps this has been changed since then, I hadn't planed to update our system until the end of the month. For what it's worth, opening the same library via "shared data" takes ~100 msec.
Devon -- Devon Ryan, Ph.D. Email: dpryan@dpryan.com <mailto:dpryan@dpryan.com> Data Manager/Bioinformatician Max Planck Institute of Immunobiology and Epigenetics Stübeweg 51 79108 Freiburg Germany
On Thu, Mar 2, 2017 at 8:30 AM, Hans-Rudolf Hotz <hrh@fmi.ch <mailto:hrh@fmi.ch>> wrote: > > this sounds more like a browser issue to me > which browser do you use? what happens when you use a different browser? and > what happens when you just reload the page > > > Hans-Rudolf > > On 03/01/2017 06:17 PM, D K wrote: >> >> Actually, when I click on the Shared Data-> Data Libraries link I get >> only the Galaxy banner and a blank page, so I'm not even seeing the data >> libraries themselves. >> >> If I use the Admin link and then click Data libraries, I can see the >> actual libraries. But when I try to enter one of the libraries with lots >> of files it hangs. >> >> >> On Wed, Mar 1, 2017 at 9:03 AM, D K <danielfortin86@gmail.com <mailto:danielfortin86@gmail.com> >> <mailto:danielfortin86@gmail.com <mailto:danielfortin86@gmail.com>>> wrote: >> >> Hi Hans, >> >> I'm running 16.10. If I'm not mistaken, the old Data Library >> interface is still accessible in the Admin menu isn't it? I have the >> same issue when navigating to a Data Library either from the Admin >> or Shared Data links. I also tried this in 16.07 with the same >> result. If it makes a difference, I'm using PostgreSQL and Apache. >> >> Dwight >> >> >> >> On Tue, Feb 28, 2017 at 11:47 PM, Hans-Rudolf Hotz <hrh@fmi.ch <mailto:hrh@fmi.ch> >> <mailto:hrh@fmi.ch <mailto:hrh@fmi.ch>>> wrote: >> >> Hi Daniel >> >> Just double checking which version of galaxy are you running? In >> other words: are you still using the old Data Library interface? >> (if I remember correctly, the old UI was deprecated in 15.10) >> We had similar issues with the old UI. >> >> The new one works fine and is very fast. E.g. we have a 'MISEQ' >> folder with more than 100 sub folders and a HISEQ folder with >> more than 600 sub folders. Each containing up to 100plus fastq >> and text files. >> >> Regards, Hans-Rudolf >> >> >> >> On 03/01/2017 12:13 AM, D K wrote: >> >> Hi, >> >> I've noticed that the more files / folders (think hundreds >> of folders >> each with 20+ fastq files) I add to a Shared Data Library, >> the longer >> and longer it takes to initially navigate the folder >> structure (after >> clicking on the specific Shared Data Library). Just opening >> up the >> Shared Data Library can take several minutes and often the >> browser times >> out. Is there a way to speed this up? >> >> If it's relevant, when I add shared data, it's linked rather >> than copied >> into Galaxy. Any suggestions would be greatly appreciated >> >> Thanks! >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/> >> <https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/>> >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ <http://galaxyproject.org/search/mailinglists/> >> <http://galaxyproject.org/search/mailinglists/ <http://galaxyproject.org/search/mailinglists/>> >> >> >> > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/> > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ <http://galaxyproject.org/search/mailinglists/> ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/>
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ <http://galaxyproject.org/search/mailinglists/>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
D K -
Martin Čech -
Nicola Soranzo