Re: [galaxy-dev] Shared Data Library, large number of files

Can you please share your '/client/galaxy/scripts/layout/menu.js' file with me? (e.g. using https://gist.github.com or similar). thanks M. On Wed, Mar 8, 2017 at 1:34 PM D K <danielfortin86(a)gmail.com&gt; wrote: > Hi Martin, > > I've tried this on both v16.07 and v16.10 > > D > > > > On Wed, Mar 8, 2017 at 10:29 AM, Martin Čech <marten(a)bx.psu.edu&gt; wrote: > > Daniel: > > what version is your Galaxy? > > Martin > > On Wed, Mar 8, 2017 at 12:40 PM D K <danielfortin86(a)gmail.com&gt; wrote: > > I've copied and pasted the javascript error below. Any help would be > greatly appreciated. Thanks! > > Uncaught TypeError: Cannot read property 'masthead' of undefined > at n.<anonymous> (menu.js:120) > at Function.each (jquery.js:370) > at HTMLDocument.<anonymous> (menu.js:117) > at l (jquery.js:3232) > at Object.add [as done] (jquery.js:3291) > at t.fn.init.vt.fn.ready (jquery.js:3542) > at callback (menu.js:116) > at Object.t.success (backbone.js:1051) > at l (jquery.js:3232) > at Object.fireWith [as resolveWith] (jquery.js:3362) > > > On Thu, Mar 2, 2017 at 10:43 AM, D K <danielfortin86(a)gmail.com&gt; wrote: > > Hi Martin, > > I do get a javascript error, I've attached a screenshot of the relevant > window. I also tried logging in as a non-admin user and had the same issue. > In this Galaxy instance are 5 data libraries, each with about 150 > individual folders, and each folder has a variable number of fastq.gz files. > > Thanks! > > On Thu, Mar 2, 2017 at 10:13 AM, Martin Čech <marten(a)bx.psu.edu&gt; wrote: > > Daniel: > > Could you please open browser console and look for javascript errors on > the page that fails to load for you? Additionally I assume you are an admin > when trying this (admins load all of the instance's libraries) could you > try loading the library list as a normal user? How many libraries do you > have on the instance? > > Thanks > > Martin > > On Thu, Mar 2, 2017 at 12:38 PM D K <danielfortin86(a)gmail.com&gt; wrote: > > Hi Martin / Hans / Devon, > > I can confirm that I am using the new UI from the top menu. Right now > when I click on the link I often do not even get the list of libraries > whereas when I click on the admin Data Library link I at least see the list > of libraries (although I can't open any of them since they are so large). > I've tried refreshing the page, deleting the cookies for the session and > using Incognito mode, using chrome and firefox both from a linux and > windows machine and can confirm this behavior on all systems. However, > occasionally when clicking the Shared Data Library link from the top I do > get a list of libraries and can navigate them, but this has only happened > once during my tests and I cannot replicate this behavior. > > Thanks for the assistance! > > On Thu, Mar 2, 2017 at 5:47 AM, Martin Čech <marten(a)bx.psu.edu&gt; wrote: > > Hi all, > > *Devon* is correct -the old data library UI under Admin menu is still in > place (because it has some features perceived as important to some admins). > However majority of users and most of the admins are using the interface > under 'Shared Data' I hope and the old UI is deprecated since 15.10 ( > https://docs.galaxyproject.org/en/master/releases/15.10_announce.html). > I created a PR to adjust the label in the admin interface to make this > clear: https://github.com/galaxyproject/galaxy/pull/3694 > > The old UI is very slow for large libraries but the new UI under Shared > Data/Data Libraries in the top menu shouldn't have this kind of issues for > hundreds of folders/datasets that are mentioned here. > > *Daniel* are you using the new UI form the top menu? If so are you > experiencing this only with a specific folder full of linked datasets or > does it occur for any folder with many datasets in it? > > Martin > > > > On Thu, Mar 2, 2017 at 3:49 AM Devon Ryan <dpryan(a)dpryan.com&gt; wrote: > > I observe the same thing on 16.07. As an example, I'm just opened a > data library under Admin->Data->Data Libraries and this took about a > minute and a half. It looks like this is related to the quantity of > data in this folder. The associated entry in the uwsgi log is as > follows: > > [pid: 357|app: 0|req: 2059/5647] 10.1.6.77 () {44 vars in 1220 bytes} > [Thu Mar 2 08:40:19 2017] GET > /library_common/browse_library?sort=name&f-description=All&f-name=All&f- > deleted=False&cntrller=library_admin&operation=browse&id=355326571f7132ff > => generated 21761790 bytes in 108644 msecs (HTTP/1.1 200) 2 headers > in 73 bytes (14 switches on core 2) > > So this library is causing ~21 megs of data to be returned to the > browser, which I presume is the source of the issue. > > As an aside, the old data library API is still in use in the admin > section until at least 16.07. Perhaps this has been changed since > then, I hadn't planed to update our system until the end of the month. > For what it's worth, opening the same library via "shared data" takes > ~100 msec. > > Devon > -- > Devon Ryan, Ph.D. > Email: dpryan(a)dpryan.com > Data Manager/Bioinformatician > Max Planck Institute of Immunobiology and Epigenetics > Stübeweg 51 > 79108 Freiburg > Germany > > > On Thu, Mar 2, 2017 at 8:30 AM, Hans-Rudolf Hotz <hrh(a)fmi.ch&gt; wrote: > > > > this sounds more like a browser issue to me > > which browser do you use? what happens when you use a different > browser? and > > what happens when you just reload the page > > > > > > Hans-Rudolf > > > > On 03/01/2017 06:17 PM, D K wrote: > >> > >> Actually, when I click on the Shared Data-> Data Libraries link I get > >> only the Galaxy banner and a blank page, so I'm not even seeing the > data > >> libraries themselves. > >> > >> If I use the Admin link and then click Data libraries, I can see the > >> actual libraries. But when I try to enter one of the libraries with > lots > >> of files it hangs. > >> > >> > >> On Wed, Mar 1, 2017 at 9:03 AM, D K <danielfortin86(a)gmail.com > >> <mailto:danielfortin86@gmail.com>> wrote: > >> > >> Hi Hans, > >> > >> I'm running 16.10. If I'm not mistaken, the old Data Library > >> interface is still accessible in the Admin menu isn't it? I have > the > >> same issue when navigating to a Data Library either from the Admin > >> or Shared Data links. I also tried this in 16.07 with the same > >> result. If it makes a difference, I'm using PostgreSQL and Apache. > >> > >> Dwight > >> > >> > >> > >> On Tue, Feb 28, 2017 at 11:47 PM, Hans-Rudolf Hotz <hrh(a)fmi.ch > >> <mailto:hrh@fmi.ch>> wrote: > >> > >> Hi Daniel > >> > >> Just double checking which version of galaxy are you running? > In > >> other words: are you still using the old Data Library > interface? > >> (if I remember correctly, the old UI was deprecated in 15.10) > >> We had similar issues with the old UI. > >> > >> The new one works fine and is very fast. E.g. we have a 'MISEQ' > >> folder with more than 100 sub folders and a HISEQ folder with > >> more than 600 sub folders. Each containing up to 100plus fastq > >> and text files. > >> > >> Regards, Hans-Rudolf > >> > >> > >> > >> On 03/01/2017 12:13 AM, D K wrote: > >> > >> Hi, > >> > >> I've noticed that the more files / folders (think hundreds > >> of folders > >> each with 20+ fastq files) I add to a Shared Data Library, > >> the longer > >> and longer it takes to initially navigate the folder > >> structure (after > >> clicking on the specific Shared Data Library). Just opening > >> up the > >> Shared Data Library can take several minutes and often the > >> browser times > >> out. Is there a way to speed this up? > >> > >> If it's relevant, when I add shared data, it's linked > rather > >> than copied > >> into Galaxy. Any suggestions would be greatly appreciated > >> > >> Thanks! > >> > >> > >> _____________________________ > ______________________________ > >> Please keep all replies on the list by using "reply all" > >> in your mail client. To manage your subscriptions to this > >> and other Galaxy lists, please use the interface at: > >> https://lists.galaxyproject.org/ > >> <https://lists.galaxyproject.org/> > >> > >> To search Galaxy mailing lists use the unified search at: > >> http://galaxyproject.org/search/mailinglists/ > >> <http://galaxyproject.org/search/mailinglists/> > >> > >> > >> > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > https://lists.galaxyproject.org/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > > > > > >