On Fri, Apr 12, 2019 at 5:12 PM Jason Dillaman <jdillama@xxxxxxxxxx> wrote: > I guess that is getting back to my point. If the MDS already has this > information, why does it need to also duplicate this information in > the MGR? The rationale for the OSDs sending their client-level > statistics to the MGR is because the MGR needs to aggregate them into > a usable form. If the MDS already has the data, why can't any "fs top" > utility just query the data from the handful of MDS servers? While most current clusters are small, the MDSes are nominally capable of scaling to a few hundred nodes... I mean, maybe doing that querying is still a good idea. But I don't really think it's that different from the OSDs either and my understanding is that things like Prometheus need to be able to pull statistics out of a single endpoint. > Again, just my 2 cents but it all ties back to my option that the MGR > is becoming a single-point-of-failure for Ceph since it doesn't > horizontally scale its workload. If we can avoid sending data to the > MGR if we don't need to, it would seem like a good thing. Hrmm. On the one hand, yes, we like to decentralize work. On the other, the whole purpose of the manager is to provide that centralized point for non-critical but nice-to-have global data like this. I've definitely done my best to keep anything that needs to work for cluster operation out of the manager, but I'm not worried about piling more data into a thing whose whole purpose is absorbing that data so the monitors don't have to and we can build global intelligences! -Greg
