Hi folks, I am currently in the process of building a Multisite Cluster (v18.2.2) with two Ceph clusters (cluster1 and cluster2). Each Cluster is its own Zone and has its own zonegroup. The idea is to sync metadata but not data. The user should select the zonegroup (region) via the location constraint. I figured out how to set up the two zonegroups but now the following problem appears: When i create a bucket via the cluster2 rgw with the cluster1 zonegroup as a target my minio client shows the following error: mc mb test/test1 --region cluster1 mc: <ERROR> Unable to make bucket `test/test1`. The specified location-constraint is not valid the cluster1 rgw shows the following error: s3:create_bucket location constraint (cluster1) doesn't match zonegroup (cluster2) the cluster2 rgw shows no logs When i do the same with the aws cli i get no error but the rgw seems to ignore the location constraint and just creates the bucket in cluster2: aws s3api --endpoint http://cluster2 --profile test create-bucket --bucket test2 --region cluster1 When I copy a file to a bucket in cluster2 via the cluster1 rgw i get the following error on the cluster1 rgw: op->ERRORHANDLER: err_no=-2024 new_err_no=-2024 The Period is synced on both clusters and the rgws are restarted: { "id": "7189ba3c-c7de-4ea2-8d14-307e4355a9e3", "epoch": 6, "predecessor_uuid": "0253dec1-d804-470f-92b6-90a64ffb5548", "sync_status": [], "period_map": { "id": "7189ba3c-c7de-4ea2-8d14-307e4355a9e3", "zonegroups": [ { "id": "4daa58d4-3eb5-4e59-bffe-e929f64c70b5", "name": "cluster1", "api_name": "cluster1", "is_master": true, "endpoints": [ "http://172.31.4.12:80"; ], "hostnames": [ "172.31.4.12", "172.31.4.16" ], "hostnames_s3website": [], "master_zone": "ae3aa0e9-c580-410a-8ed6-2f01657a06f8", "zones": [ { "id": "ae3aa0e9-c580-410a-8ed6-2f01657a06f8", "name": "cluster1-a", "endpoints": [ "http://172.31.4.12:80"; ], "log_meta": false, "log_data": false, "bucket_index_max_shards": 11, "read_only": false, "tier_type": "", "sync_from_all": true, "sync_from": [], "redirect_zone": "", "supported_features": [ "compress-encrypted", "resharding" ] } ], "placement_targets": [ { "name": "default-placement", "tags": [], "storage_classes": [ "STANDARD" ] } ], "default_placement": "default-placement", "realm_id": "5e8f4256-cf89-40db-b2a9-bb717f980b4f", "sync_policy": { "groups": [] }, "enabled_features": [ "resharding" ] }, { "id": "8d1d8050-0dea-4c5a-9bc3-d6631ef132f8", "name": "cluster2", "api_name": "cluster2", "is_master": false, "endpoints": [ "http://172.31.4.16:80"; ], "hostnames": [ "172.31.4.12", "172.31.4.16" ], "hostnames_s3website": [], "master_zone": "2a882ccc-b540-4de2-abe3-f026fccb9aef", "zones": [ { "id": "2a882ccc-b540-4de2-abe3-f026fccb9aef", "name": "cluster2-a", "endpoints": [ "http://172.31.4.16:80"; ], "log_meta": false, "log_data": false, "bucket_index_max_shards": 11, "read_only": false, "tier_type": "", "sync_from_all": true, "sync_from": [], "redirect_zone": "", "supported_features": [ "compress-encrypted", "resharding" ] } ], "placement_targets": [ { "name": "default-placement", "tags": [], "storage_classes": [ "STANDARD" ] } ], "default_placement": "default-placement", "realm_id": "5e8f4256-cf89-40db-b2a9-bb717f980b4f", "sync_policy": { "groups": [] }, "enabled_features": [ "resharding" ] } ], "short_zone_ids": [ { "key": "2a882ccc-b540-4de2-abe3-f026fccb9aef", "val": 4229298211 }, { "key": "ae3aa0e9-c580-410a-8ed6-2f01657a06f8", "val": 1754305595 } ] }, "master_zonegroup": "4daa58d4-3eb5-4e59-bffe-e929f64c70b5", "master_zone": "ae3aa0e9-c580-410a-8ed6-2f01657a06f8", "period_config": { "bucket_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "user_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "user_ratelimit": { "max_read_ops": 0, "max_write_ops": 0, "max_read_bytes": 0, "max_write_bytes": 0, "enabled": false }, "bucket_ratelimit": { "max_read_ops": 0, "max_write_ops": 0, "max_read_bytes": 0, "max_write_bytes": 0, "enabled": false }, "anonymous_ratelimit": { "max_read_ops": 0, "max_write_ops": 0, "max_read_bytes": 0, "max_write_bytes": 0, "enabled": false } }, "realm_id": "5e8f4256-cf89-40db-b2a9-bb717f980b4f", "realm_name": "testrealm", "realm_epoch": 2 } According to my understanding this setup should work but I probably missed some config setting. Any idea? Thanks in advance! _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |