Hi Joachim, Something that may help with benchmarking: At the July 2012 GalaxyAdmins meetup, Anne Black-Ziegelbein talked about how they evaluated filesystem options. She also included benchmarking scripts and data. See http://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2012_07_09 Dave C On Tue, Sep 10, 2013 at 10:12 AM, Adam Brenner <aebrenne@uci.edu> wrote:
optimize performance? The purpose is to share the database over NFS to the Galaxy VM.
Are you moving the PostgreSQL/MySQL database over to ZFS or the actual storage of your datasets over to ZFS? I am assuming the latter. Remember, ZFS is just a file system, you still need a protocol, like NFS, to export the data to each of your machines. This is going to be your bottleneck, luckily the NFS clients supports write caching as described in our test here[1].
On our HPC cluster, we run xfs filesystem on top of Gluster and we have another filesystem using xfs on top of FraunhoferFS. I did test out ZFS on Linux roughly a year ago and in terms of Read/Write on a single machine it was slower than native ext4 and XFS. This of course was to be expected. However the added benefit of ZFS may be more favorable in your case, Snapshots, disk management, ZIL/SSD caching, etc. If you use a distributed filesystem like Gluster, FraunhoferFS, or even Lustre (2.x branch supports ZFS!) you will most certainly get some very good read/write speeds.
However, it sounds like you are using a single machine, so your read/write is going to be slower than native ext4 and XFS --- trust but verify, run your own read/write tests.
When I was using ZFS, this ZFSBuild[2] website was most helpful.
Let me know if you have any other questions, -Adam
[1]: http://www.gluster.org/pipermail/gluster-users/2012-September/034295.html [2]: http://www.zfsbuild.com/
-- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences
Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/
University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne@uci.edu
Hi all,
I am performing some tests to move my galaxy database to ZFS. Does anybody have experience with ZFS on linux, and some recommendations/experiences to optimize performance? The purpose is to share the database over NFS to
On Tue, Sep 10, 2013 at 3:29 AM, Joachim Jacob | VIB | <joachim.jacob@vib.be> wrote: the
Galaxy VM.
Thanks, Joachim.
-- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/