Hi Xuehan, Thanks for presenting during CDM! The final report I mentioned for the Red Hat / Harvey Mudd clinic that looked at the point-in-time consistency and time sync problem is at http://newdream.net/~sage/RedHatFinalReport%2014-15.pdf If I understood correctly, this is essentially what you are proposing, except that the delay is done on the OSD instead of the client. I think this improves a few things: - fewer OSDs, fewer clocks to sync - in most deployments, OSD to OSD latency will be lower - clock error bound will be smaller - fewer nodes whose clocks might jitter at the wrong time and invalidate the checkpoint Also, I *think* we can simply delay the OSD replies, preventing any causal link with a subsequent operation; new requests can still be processed, and hopefully the impact on overall workload will be very small. Does that simplify your approach or make it more complex? I couldn't tell if there were other reasons or advantages to doing it on the client. Thanks! sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html