Hi Brad,

I agree that implementing an ajax approach to rendering libraries may be a good solution, but also may be difficult.  We'll analyze this a bit more and see if it will be reasonable.  

We made a decision when we implemented libraries that we would provide fine grained security at the dataset level.  The trade-off, obviously, is that it takes time to check every dataset.  Another approach to solve this would be to not provide as fine-grained security, and have security at the folder level rather than the dataset level.  If this is done, all dataset within a folder would be required to have the same security.  What are your thoughts on this approach?

Thanks Brad,

Greg


On Mar 10, 2011, at 10:07 AM, Brad Chapman wrote:

Greg;

[large data library performance]
Security checks are performed on every active ( undeleted ) dataset
within a data library as it's contents are rendered upon opening. For
the current user, every dataset is checked to determine if the user
can access it, and if so, then checks are made to see if the user has
permission to perform operations on the dataset that fall into the add
/ modify / manage permissions areas. These checks incur db hits for
each dataset, so if your data library include many datasets ( several
hundred or more ), then it will take a bit of time to render it upon
opening. The size of the dataset files is not an issue here, but the
number of datasets within the data library.

We are running into this limit quite a bit in practice as our data libraries
grow. Splitting it does provide as a quick workaround. What would you
think about loading folder data on-demand via Ajax? Our data is
stored in folders and sub-folders within the library, so this would
let us scale up library items without having to arbitrarily split
the data libraries.

I took a quick look at the code with this in might and it looked,
well, hard. But that could be totally due to my ignorance of the
implementation, what do you think?

Brad
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Greg Von Kuster
Galaxy Development Team
greg@bx.psu.edu