On Thu, May 5, 2022 at 7:01 AM mayank srivastava <srivastavamayank133@xxxxxxxxx> wrote: > > Hi Team, > > I have been trying to understand the flow of the PutBucketReplication API. > For this I have the following setup: > I am running 2 instances of RGW on a single vm. My first instance runs on port 8000, while the other does on 8001. > My main zone resides on instance 127.0.0.1:8000 and secondary on 127.0.0.1:8001. I have created 2 buckets 'bucket1' and 'bucket2' respectively in zones. > Where I treat bucket1 as source and bucket2 as destination. > > For PutBucketReplication API I am making the following request: > > PUT http://127.0.0.1:8001/bucket2?replication > <ReplicationConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/";><Role></Role><Rule><Status>Enabled</Status><ID>Rule-1</ID><Filter> <And><Tag><Key>key1</Key><Value>value1</Value></Tag><Tag><Key>key2</Key><Value>value2</Value></Tag></And></Filter><Source><Zone>us-east</Zone></Source><Priority>11</Priority><DeleteMarkerReplication><Status>Disabled</Status></DeleteMarkerReplication><Destination><Bucket>bucket2</Bucket><Zone>us-west</Zone></Destination></Rule></ReplicationConfiguration> > > And I noticed under 'RGWPutBucketReplication::execute()' it is calling a method named 'forward_request_to_master()' which creates a REST request and forwards it to the main zone. > I have captured the request which is sent to main zone as displayed below: > > PUT http://127.0.0.1:8000/bucket2?replication&rgwx-uid=tester1&rgwx-zonegroup=2496e18c-cceb-4dbe-a0fc-5b693193bd8b > <ReplicationConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/";><Role></Role><Rule><Status>Enabled</Status><ID>Rule-1</ID><Filter> <And><Tag><Key>key1</Key><Value>value1</Value></Tag><Tag><Key>key2</Key><Value>value2</Value></Tag></And></Filter><Source><Zone>us-east</Zone></Source><Priority>11</Priority><DeleteMarkerReplication><Status>Disabled</Status></DeleteMarkerReplication><Destination><Bucket>bucket2</Bucket><Zone>us-west</Zone></Destination></Rule></ReplicationConfiguration> > > Now I also noticed that even after forwarding the request to the main zone, the secondary zone doesn't return and continues with the request and later updates bucket info via 'RadosBucket::put_info()', which is also done by the main zone. > > So I am not able to understand this behavior, that why is the request is being carried out by both the secondary zone and main zone. > Please let me know if I am missing something. hi Mayank, this bucket replication policy is stored in the bucket instance metadata, and multisite has a special consistency model for metadata. a single 'metadata master zone' is authoritative for all metadata, so any metadata writes must be applied there first, then replicated to other zones this forward_request_to_master() pattern is how we support metadata write requests targeted at other zones. first, forward_request_to_master() ensures that the write succeeds on the metadata master zone, then it applies the same write to the local zone. that second step may not seem necessary, because metadata sync would replicate this change eventually, but by applying the write inline we can promise read-after-write consistency to the client. that is, the client can send a GetBucketReplication request immediately after PutBucketReplication to read back the updated policy > > Thank You, > Mayank Srivastava > _______________________________________________ > Dev mailing list -- dev@xxxxxxx > To unsubscribe send an email to dev-leave@xxxxxxx _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
