Re: Radosgw multisite configuration with S3 API?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Thank you for the follow up.

 I did manage to get the replication configured and working using the radosgw-admin commands described in the docs for mirroring specific buckets. You need to have a group policy that has sync "allowed" and then new sync group with its own specific pipes that are "enabled". 

I've yet to figure out if this is possible using purely S3 API calls, or even if it is worth trying since the radosgw-admin commands seem to be working for our purposes.



From: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@xxxxxxx>
Sent: Wednesday, April 27, 2022 10:00 AM
To: dev@xxxxxxx <dev@xxxxxxx>
Subject: Re: Radosgw multisite configuration with S3 API?
 
Hi Wyll,

Not a rgw developer but from my test and what I understood from the code
the Put/GetBucketReplication create/read replication rule specific to a
bucket and does not read the one that you globally created.

For instance here is a rule that I created via the PutBucketReplication
(with `aws s3api put-bucket-replication`) which is not visible If I list
all the sync group globally (so without the --bucket=conflict in my case):
```
[15:52][root@rgwmulti2-rgw2 (arthur_dev:ceph/rgwmulti2/traefik*1) ~]#
radosgw-admin sync group get --bucket=conflict
[
    {
        "key": "s3-bucket-replication:disabled",
        "val": {
            "id": "s3-bucket-replication:disabled",
            "data_flow": {},
            "pipes": [],
            "status": "allowed"
        }
    },
    {
        "key": "s3-bucket-replication:enabled",
        "val": {
            "id": "s3-bucket-replication:enabled",
            "data_flow": {},
            "pipes": [
                {
                    "id": "replication",
                    "source": {
                        "bucket": "*",
                        "zones": [
                            "*"
                        ]
                    },
                    "dest": {
                        "bucket": "conflict",
                        "zones": [
                            "*"
                        ]
                    },
                    "params": {
                        "source": {
                            "filter": {
                                "tags": []
                            }
                        },
                        "dest": {},
                        "priority": 0,
                        "mode": "user",
                        "user": "conflict"
                    }
                }
            ],
            "status": "enabled"
        }
    }
]
```

The setup that I tested with some success is basically the one from the
doc `Two Zones, Complete Mirror` but instead I just don't enable the
replication just leaving it as `allowed` and thus the users would just
enable the replication themselves with the s3 API.

Cheers,

--
Arthur Outhenin-Chalandre




On 4/26/22 17:14, Wyll Ingersoll wrote:
>
> In Pacific (16.2.7) can the radosgw multisite configuration policies be
> manipulated with the AWS S3 APIs such as GetBucketReplication,
> PutBucketReplication?
>
> I have multisite replication working on a per-bucket basis that was set
> up using the radosgw-admin (sync group *** commands).  Can these
> policies be accessed in any way using the actual S3 REST API?
>
> The documentation (https://docs.ceph.com/en/pacific/radosgw/s3/
> <https://docs.ceph.com/en/pacific/radosgw/s3/>) seems to indicate that
> the "Bucket Replication" API is partially supported (across zones
> only).  When I use the "GetBucketReplication" API on a bucket that is
> configured with a multisite sync group and pipe, it does not return any
> policy data.
>
> If I use the raw S3 API, I see the following result when I use the
> boto3.get_bucket_replication(bucket='bucket-1'):
>
> {
>
>     "ResponseMetadata": {
>
>         "RequestId": "tx00000cdf1db80cb3fda06-006268085f-247a6fc4-keeper-1",
>
>         "HostId": "",
>
>         "HTTPStatusCode": 200,
>
>         "HTTPHeaders": {
>
>             "x-amz-request-id":
> "tx00000cdf1db80cb3fda06-006268085f-247a6fc4-keeper-1",
>
>             "content-type": "application/xml",
>
>             "content-length": "152",
>
>             "date": "Tue, 26 Apr 2022 14:57:35 GMT",
>
>             "connection": "Keep-Alive"
>
>         },
>
>         "RetryAttempts": 0
>
>     },
>
>     "ReplicationConfiguration": {
>
>         "Role": ""
>
>     }
>
> }
>
>
>
>
> thanks,
>    Wyllys Ingersoll
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx

[Index of Archives]     [CEPH Users]     [Ceph Devel]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux