Scaling tools: data available?
Hi there We're researching the resource (CPU time and memory) requirements for RNA STAR at the moment. Specifically, we'd like to build up a database of input size to resource usage so that we can use this to feed a dynamic destination mapper (currently our STAR configuration uses a thumbsuck for RAM and CPU requirements that is not optimal but works in many cases). Has anyone collected these metrics before? And more generally, is anyone collecting metrics for a range of bioinformatics tools? A database like this could be very useful! Thanks! Peter
Hi peter, I've been working on this in the past but got caught up in other projects. There's work for a server-side component here (maybe 50% done), the client side stuff could all be done through collectl but I was having issues actually collecting collectl data that had put the project on hold temporarily. https://github.com/erasche/galactic-radio-telescope I'd be very, very interested in finishing this project up. Cheers, Eric On 02/21/2016 11:13 PM, Peter van Heusden wrote:
Hi there
We're researching the resource (CPU time and memory) requirements for RNA STAR at the moment. Specifically, we'd like to build up a database of input size to resource usage so that we can use this to feed a dynamic destination mapper (currently our STAR configuration uses a thumbsuck for RAM and CPU requirements that is not optimal but works in many cases).
Has anyone collected these metrics before? And more generally, is anyone collecting metrics for a range of bioinformatics tools? A database like this could be very useful!
Thanks! Peter
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- -- Eric Rasche Programmer II Center for Phage Technology Rm 312A, BioBio Texas A&M University College Station, TX 77843 404-692-2048 <tel:404-692-2048> esr@tamu.edu <mailto:esr@tamu.edu> Not responding quickly enough? <https://calendar.google.com/calendar/embed?src=esr%40tamu.edu&ctz=America/Chicago>
Hi Eric I'm having a look at the GRT. We're collecting metrics on our local Galaxy server, I'm just trying to understand how you envision these metrics getting into the GRT? Peter On 22 February 2016 at 07:31, Eric Rasche <esr@tamu.edu> wrote:
Hi peter,
I've been working on this in the past but got caught up in other projects. There's work for a server-side component here (maybe 50% done), the client side stuff could all be done through collectl but I was having issues actually collecting collectl data that had put the project on hold temporarily.
https://github.com/erasche/galactic-radio-telescope
I'd be very, very interested in finishing this project up.
Cheers, Eric
On 02/21/2016 11:13 PM, Peter van Heusden wrote:
Hi there
We're researching the resource (CPU time and memory) requirements for RNA STAR at the moment. Specifically, we'd like to build up a database of input size to resource usage so that we can use this to feed a dynamic destination mapper (currently our STAR configuration uses a thumbsuck for RAM and CPU requirements that is not optimal but works in many cases).
Has anyone collected these metrics before? And more generally, is anyone collecting metrics for a range of bioinformatics tools? A database like this could be very useful!
Thanks! Peter
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- -- Eric Rasche Programmer II
Center for Phage Technology Rm 312A, BioBio Texas A&M University College Station, TX 77843 404-692-2048 esr@tamu.edu Not responding quickly enough? <https://calendar.google.com/calendar/embed?src=esr%40tamu.edu&ctz=America/Chicago>
Hi Peter, The vision I had was a cron script admins could install which would send thr past day/week's collectl/memory/parameter/dataset size logs to GRT. Again, the GRT side stalled when I hit issues with collectl locally. If you have time/energy to work on that we should chat :-) Otherwise I plan to find time for it before GCC, minus the collectl stuff, just capturing runtimes as a start. Ciao, Eric 14. mar. 2016 1.09 p.m. skrev "Peter van Heusden" <pvh@sanbi.ac.za>:
Hi Eric
I'm having a look at the GRT. We're collecting metrics on our local Galaxy server, I'm just trying to understand how you envision these metrics getting into the GRT?
Peter
On 22 February 2016 at 07:31, Eric Rasche <esr@tamu.edu> wrote:
Hi peter,
I've been working on this in the past but got caught up in other projects. There's work for a server-side component here (maybe 50% done), the client side stuff could all be done through collectl but I was having issues actually collecting collectl data that had put the project on hold temporarily.
https://github.com/erasche/galactic-radio-telescope
I'd be very, very interested in finishing this project up.
Cheers, Eric
On 02/21/2016 11:13 PM, Peter van Heusden wrote:
Hi there
We're researching the resource (CPU time and memory) requirements for RNA STAR at the moment. Specifically, we'd like to build up a database of input size to resource usage so that we can use this to feed a dynamic destination mapper (currently our STAR configuration uses a thumbsuck for RAM and CPU requirements that is not optimal but works in many cases).
Has anyone collected these metrics before? And more generally, is anyone collecting metrics for a range of bioinformatics tools? A database like this could be very useful!
Thanks! Peter
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- -- Eric Rasche Programmer II
Center for Phage Technology Rm 312A, BioBio Texas A&M University College Station, TX 77843 404-692-2048 esr@tamu.edu Not responding quickly enough? <https://calendar.google.com/calendar/embed?src=esr%40tamu.edu&ctz=America/Chicago>
participants (2)
-
Eric Rasche -
Peter van Heusden