Hi cephers,
I'm working through our testing cycle to upgrade our main ceph cluster
from Luminous to Mimic, and I ran into a problem with ceph_fuse. With
Luminous, a single client can pretty much max out a 10Gbps network
connection writing sequentially on our cluster with Luminous ceph_fuse.
When I tried the Mimic ceph_fuse, I could only get about 150MB/sec
through a single connection. After a bit of debugging and instrumenting
the ceph-fuse code, I saw that the Luminous ceph-fuse gets 128kB writes
(about 8000 calls/sec, leading to about 1000MB/sec write throughput),
while the Mimic client sees 4kB writes (about 38k calls/sec, around
150MB/sec throughput). There seems to be an undocumented config option,
"fuse_big_writes", which controls the "big_writes" fuse mount option.
In Luminous this defaults to true, while in Mimic it is false. Once I
added this option to ceph.conf, the performance of Mimic ceph-fuse
improved back to the Luminous level.
It would perhaps be helpful to say something about this option in the
docs. The git commit seems to be
f37f2ea10c8298fc222208777c74c8264b4b0cb6, the pull request mentions that
the cause for removal of this option was the removal of the option in
libfuse 3.0 (where it defaults to true). Unfortunately CentOS packages
fuse 2.9.2. Thus an upgrade to Mimic on CentOS with ceph-fuse (without
knowing about this undocumented option) leads to 4k writes in the fuse
layer with very poor performance ...
Andras
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
