For multiple clients accessing the same volume, same object, I guess the clients need to do some synchronization between them to guarantee what it reads is what it wrote. That is to say, if two clients, say A and B, A is doing write, B is doing read. If B wants to read what A writes, it needs to know A has completed the write before issuing the read. If A hasn't completed the write yet, it could be possible that the write fails, so you can't expect B reads what A writes. I think this makes sense from the client/application perspective. -----Original Message----- From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Sage Weil Sent: Wednesday, December 10, 2014 8:43 AM To: Cook, Nigel Cc: Yehuda Sadeh; Josh Durgin; Samuel Just; Wang, Zhiqiang; Jason Dillaman; ceph-devel Subject: Re: rados read ordering On Wed, 10 Dec 2014, Cook, Nigel wrote: > Folks > > I'm wondering if this is related to the question I posed a few days > ago.. > > Can CEPH support 2 clients simultaneously accessing a single volume - > for example a database cluster - and honor read and write order of > blocks across the multiple clients? > > Can you comment? I don't think so. This (non)change is about a single client submitting two reads to a single object (say, different blocks in the same disk) and whether the OSD is allowed to respond out of order (say, because some blocks are in cache and some aren't). In the shared volume case, it is generally not important what happens with requests that are submitted in parallel.. they can take different amounts of time on the wire and which happens first (i.e., which arrives at the OSD first) depends on happenstance. What does matter is that any read that happens after a complete write reflects that read, which is why the concern is around caching. sage > > Regards, > Nigel Cook +1 720 319 7508 > > Sent from a mobile device. > Please excuse both my brevity and sp3lling > > On Dec 8, 2014 10:12 AM, Yehuda Sadeh <yehuda@xxxxxxxxxx> wrote: > On Mon, Dec 8, 2014 at 9:03 AM, Sage Weil <sweil@xxxxxxxxxx> wrote: > > The current RADOS behavior is that reads (on any given object) are always > > processed in the order they are submitted by the client. This causes a > > few headaches for the cache tiering that it would be nice to avoid. It > > also occurs to me that there are likely cases where we could go a lot > > faster by not strictly ordering things. For example, a stat can respond > > more quickly than a large read, and some reads may hit cache while others > > go to disk. This doesn't happen currently because of the (lame) way we do > > reads synchronously, but hope that can change too. > > > > I propose we drop this semantic. If a client wants reads to have a strict > > ordering, they can set the existing RWORDERED flag (which also orders them > > with respect to writes). That's not the most general thing ever, but I'm > > not sure we care about callers who want reads ordered with respect to each > > other but not writes. > > > > The real question is whether there are any users that want/need this > > currently. I can't think of any offhand. In several places we submit > > multiple *writes* and expect them to be strictly ordered (e.g., we > > set a completion on teh last write only). I don't think we do this > > anywhere for reads though... > > > > Josh, Yehuda, Jason--can you think of any in RBD or RGW that would depend > > on this? > > > > None that I can think of. For objects data, we already stripe it > across multiple objects, and the underlying assumption is that we're > going to get responses out of order so we make sure we commit > in-order. Guards are used on the head object, and the read is > synchronous there anyway. I can't think of any other place where we'd > have an issue. > > Yehuda > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
