each radosgw does maintain its own cache for certain metadata like users and buckets. when one radosgw writes to a metadata object, it broadcasts a notification (using rados watch/notify) to other radosgws to update/invalidate their caches. the initiating radosgw waits for all watch/notify responses before responding to the client. this way a given client sees read-after-write consistency even if they read from a different radosgw On Fri, Sep 22, 2023 at 5:53 PM Matthias Ferdinand <mf+ml.ceph@xxxxxxxxx> wrote: > > On Tue, Sep 12, 2023 at 07:13:13PM +0200, Matthias Ferdinand wrote: > > On Mon, Sep 11, 2023 at 02:37:59PM -0400, Matt Benjamin wrote: > > > Yes, it's also strongly consistent. It's also last writer wins, though, so > > > two clients somehow permitted to contend for updating policy could > > > overwrite each other's changes, just as with objects. > > this would be a tremendous administrative bonus, but could also be a > caching/performance problem. > > Amazon explicitly says they have eventual consistency for caching > reasons: > https://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_general.html#troubleshoot_general_eventual-consistency > > For Dell ECS I don't seem to find it mentioned in their docs, but they > too are eventually consistent. > > I guess the bucket policies in Ceph get written to special rados > objects (strongly consistent by design), but how are rgw daemons > notified about these updates for immediate effect? Or do rgw daemons > re-read the bucket policy for each and every request to this bucket? > > thanks in advance > Matthias > > > > On Mon, Sep 11, 2023 at 2:21 PM Matthias Ferdinand <mf+ml.ceph@xxxxxxxxx> > > > wrote: > > > > > > > Hi, > > > > > > > > while I don't currently use rgw, I still am curious about consistency > > > > guarantees. > > > > > > > > Usually, S3 has strong read-after-write consistency guarantees (for > > > > requests that do not overlap). According to > > > > https://docs.ceph.com/en/latest/dev/radosgw/bucket_index/ > > > > in Ceph this is also true for per-object ACLs. > > > > > > > > Is there also a strong consistency guarantee for (bucket) policies? The > > > > documentation at > > > > https://docs.ceph.com/en/latest/radosgw/bucketpolicy/ > > > > apparently does not say anything about this. > > > > > > > > How would multiple rgw instances synchronize a policy change? Is this > > > > effective immediate with strong consistency or is there some propagation > > > > delay (hopefully on with some upper bound)? > > > > > > > > > > > > Best regards > > > > Matthias > > > > _______________________________________________ > > > > ceph-users mailing list -- ceph-users@xxxxxxx > > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > > > > > > > > > > > -- > > > > > > Matt Benjamin > > > Red Hat, Inc. > > > 315 West Huron Street, Suite 140A > > > Ann Arbor, Michigan 48103 > > > > > > http://www.redhat.com/en/technologies/storage > > > > > > tel. 734-821-5101 > > > fax. 734-769-8938 > > > cel. 734-216-5309 > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |