On Fri, Dec 2, 2016 at 7:47 PM, Steve Jankowski <steve@xxxxxxxxxx> wrote: > Anyone using rrdtool with Ceph via rados or cephfs ? > > > If so, how many rrd files and how many rrd file updates per minute. > > > We have a large population of rrd files that's growing beyond a single > machine. We're already using SSD and rrdcached with great success, but it's > not enough for the growth that's coming. > > > A distributed file store would tick a lot of check boxes, but it needs to > survive the high volume small write IOPS produced by rrdtool. I would imagine cephfs would work decently well if you turned up the cache size high enough to match the number of db files you were writing, and your page cache was big enough. IIRC rrd/whispedb do not issue a lot of syncs, and the workload mainly ends up as a huge thrash of the page cache followed by occasional drains to disk. However, that only works as long as your frontend machine has enough cpu/ram to buffer up all the updates nicely. If you're reaching the point where your data exceeds what a local SSD can handle, then you're probably also close to the point where the server generally is going to start creaking under the load. My unscientific opinion is that once you've exceeded what rrd can do one one fat server, it's time to look at a distributed time series database: as well as being distributed, influx/prometheus also have much more favourable local IO patterns I believe (rrd is horrible from an IO perspective because of the way it scatters updates across files and rewrites things). John > > > Thanks, > > Steve > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |