Hi Joachim,
Something that may help with benchmarking: At the July 2012 GalaxyAdmins meetup, Anne Black-Ziegelbein talked about how they evaluated filesystem options. She also included benchmarking scripts and data. See http://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2012_07_09
Dave C
On Tue, Sep 10, 2013 at 10:12 AM, Adam Brenner aebrenne@uci.edu wrote:
optimize performance? The purpose is to share the database over NFS to
the
Galaxy VM.
Are you moving the PostgreSQL/MySQL database over to ZFS or the actual storage of your datasets over to ZFS? I am assuming the latter. Remember, ZFS is just a file system, you still need a protocol, like NFS, to export the data to each of your machines. This is going to be your bottleneck, luckily the NFS clients supports write caching as described in our test here[1].
On our HPC cluster, we run xfs filesystem on top of Gluster and we have another filesystem using xfs on top of FraunhoferFS. I did test out ZFS on Linux roughly a year ago and in terms of Read/Write on a single machine it was slower than native ext4 and XFS. This of course was to be expected. However the added benefit of ZFS may be more favorable in your case, Snapshots, disk management, ZIL/SSD caching, etc. If you use a distributed filesystem like Gluster, FraunhoferFS, or even Lustre (2.x branch supports ZFS!) you will most certainly get some very good read/write speeds.
However, it sounds like you are using a single machine, so your read/write is going to be slower than native ext4 and XFS --- trust but verify, run your own read/write tests.
When I was using ZFS, this ZFSBuild[2] website was most helpful.
Let me know if you have any other questions, -Adam
-- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences
Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/
University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne@uci.edu
On Tue, Sep 10, 2013 at 3:29 AM, Joachim Jacob | VIB | joachim.jacob@vib.be wrote:
Hi all,
I am performing some tests to move my galaxy database to ZFS. Does
anybody
have experience with ZFS on linux, and some recommendations/experiences
to
optimize performance? The purpose is to share the database over NFS to
the
Galaxy VM.
Thanks, Joachim.
-- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/