Yes, that's right. It would be nice if there was a mount option to have such parameters adjusted on a per-file system basis. I should mention that I observed a significant performance improvement for HDD throughput of the local disk as well when adjusting these parameters for ceph. This is largely due to the "too much memory problem" on big servers. The kernel defaults are suitable for machines with 4-8G of RAM. Any enterprise server will beat that with the consequence of insanely large amounts of dirty buffers, leading to buffer flush panic overloading in particular, network file systems (there is a nice article by SUSE https://www.suse.com/support/kb/doc/?id=000017857). Adjusting these parameters to play nice with ceph might actually improve overall performance as a side effect. I would give it a go. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Sage Meng <lkkey80@xxxxxxxxx> Sent: 12 November 2020 16:00:08 To: Frank Schilder Cc: ceph-users@xxxxxxx Subject: Re: Is there a way to make Cephfs kernel client to write data to ceph osd smoothly with buffer io vm.dirty_bytes and vm.dirty_background_bytes are all system-wide control parameters, it will influence all the system jobs by adjusting them. Better to have a Ceph Special way to make the transfer more smoothly. Frank Schilder <frans@xxxxxx<mailto:frans@xxxxxx>> 于2020年11月11日周三 下午3:28写道： These kernel parameters influence the flushing of data, and also performance: vm.dirty_bytes vm.dirty_background_bytes Smaller vm.dirty_background_bytes will make the transfer more smooth and the ceph cluster will like that. However, it reduces the chances of merge operations in cache and the ceph cluster will not like that. The tuning is heavily workload dependent. Test with realistic workloads and a reasonably large spectrum of values. I got good results by tuning down vm.dirty_background_bytes just to the point when it reduced client performance of copying large files. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Sage Meng <lkkey80@xxxxxxxxx<mailto:lkkey80@xxxxxxxxx>> Sent: 06 November 2020 13:45:53 To: ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> Subject: Is there a way to make Cephfs kernel client to write data to ceph osd smoothly with buffer io Hi All, Cephfs kernel client is influenced by kernel page cache when we write data to it, outgoing data will be huge when os starts flush page cache. So Is there a way to make Cephfs kernel client to write data to ceph osd smoothly when buffer io is used ? _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> To unsubscribe send an email to ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |