We are running RHCS2.3 (jewel) with ganesha 2.4.2 and cephfs fsal, compiled from srpm. experimenting with CTDB for controlling ganesha HA since we run samba on same servers.
Haven't done much functionality/stress testing but on face value basic stuff seems to work well (file operations).
In terms of performance, last time I tested ganesha it seemed comparable to ceph-fuse (RHCS2.x/jewel, i think luminous ceph-fuse is better). Though I haven't done rigorous metadata tests or multiple client tests. Also our ganesha servers are quite small, as we are thus far only serving cephfs natively. eg 4G ram 1 core. Here are some FIO results:
jobs in order are:
1. async 1M
2. sync 1M
3. async 4k
4. sync 4k
5. seq read 1M
6. rand read 4k
Ceph cluster is RHCS 2.3 (10.2.7)
CEPH-FUSE (10.2.x)
WRITE: io=143652MB, aggrb=490328KB/s, minb=490328KB/s, maxb=490328KB/s, mint=300002msec, maxt=300002msec
WRITE: io=14341MB, aggrb=48947KB/s, minb=48947KB/s, maxb=48947KB/s, mint=300018msec, maxt=300018msec
WRITE: io=9808.2MB, aggrb=33478KB/s, minb=33478KB/s, maxb=33478KB/s, mint=300001msec, maxt=300001msec
WRITE: io=424476KB, aggrb=1414KB/s, minb=1414KB/s, maxb=1414KB/s, mint=300003msec, maxt=300003ms
READ: io=158069MB, aggrb=539527KB/s, minb=539527KB/s, maxb=539527KB/s, mint=300008msec, maxt=300008msec
READ: io=1881.2MB, aggrb=6420KB/s, minb=6420KB/s, maxb=6420KB/s, mint=300001msec, maxt=300001msec
ganesha (nfs3)
WRITE: io=157891MB, aggrb=538923KB/s, minb=538923KB/s, maxb=538923KB/s, mint=300006msec, maxt=300006msec
WRITE: io=38700MB, aggrb=132093KB/s, minb=132093KB/s, maxb=132093KB/s, mint=300006msec, maxt=300006msec
WRITE: io=3072.0MB, aggrb=10148KB/s, minb=10148KB/s, maxb=10148KB/s, mint=309957msec, maxt=309957msec
WRITE: io=397516KB, aggrb=1325KB/s, minb=1325KB/s, maxb=1325KB/s, mint=300001msec, maxt=300001msec
READ: io=82521MB, aggrb=281669KB/s, minb=281669KB/s, maxb=281669KB/s, mint=300002msec, maxt=300002msec
READ: io=1322.2MB, aggrb=4513KB/s, minb=4513KB/s, maxb=4513KB/s, mint=300001msec, maxt=300001msec
cephfs kernel client
WRITE: io=471041MB, aggrb=1568.8MB/s, minb=1568.8MB/s, maxb=1568.8MB/s, mint=300394msec, maxt=300394msec
WRITE: io=50005MB, aggrb=170680KB/s, minb=170680KB/s, maxb=170680KB/s, mint=300006msec, maxt=300006msec
WRITE: io=169092MB, aggrb=577166KB/s, minb=577166KB/s, maxb=577166KB/s, mint=300000msec, maxt=300000msec
WRITE: io=530548KB, aggrb=1768KB/s, minb=1768KB/s, maxb=1768KB/s, mint=300003msec, maxt=300003msec
READ: io=121501MB, aggrb=414720KB/s, minb=414720KB/s, maxb=414720KB/s, mint=300002msec, maxt=300002msec
READ: io=3264.6MB, aggrb=11142KB/s, minb=11142KB/s, maxb=11142KB/s, mint=300001msec, maxt=300001msec
happy to share fio job file if anyone wants it.
On 9 November 2017 at 08:41, Sage Weil <sweil@xxxxxxxxxx> wrote:
Who is running nfs-ganesha's FSAL to export CephFS? What has your
experience been?
(We are working on building proper testing and support for this into
Mimic, but the ganesha FSAL has been around for years.)
Thanks!
sage
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Rafael Lopez
Research Devops Engineer
T: +61 3 9905 9118
M: +61 (0)427682670
E: rafael.lopez@xxxxxxxxxx
Monash University eResearch Centre
M: +61 (0)427682670
E: rafael.lopez@xxxxxxxxxx
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
