On Tue, Jul 19, 2016 at 10:25 AM, Fabiano de O. Lucchese <flucchese@xxxxxxxxx> wrote: > I configured the cluster to replicate data twice (3 copies), so these > numbers fall within my expectations. So far so good, but here's comes the > issue: I configured CephFS and mounted a share locally on one of my servers. > When I write data to it, it shows abnormally high performance at the > beginning for about 5 seconds, stalls for about 20 seconds and then picks up > again. For long running tests, the observed write throughput is very close > to what the rados bench provided (about 640 MB/s), but for short-lived > tests, I get peak performances of over 5GB/s. I know that journaling is > expected to cause spiky performance patters like that, but not to this > level, which makes me think that CephFS is buffering my writes and returning > the control back to client before persisting them to the jounal, which looks > undesirable. The client is buffering the writes to RADOS which would give you the abnormally high initial performance until the cache needs flushed. You might try tweaking certain osd settings: http://docs.ceph.com/docs/hammer/rados/configuration/osd-config-ref/ in particular: "osd client message size cap". Also: http://docs.ceph.com/docs/hammer/rados/configuration/journal-ref/ -- Patrick Donnelly _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |