Rados Gateway Replication - Containers not accessible via slave zone !

[Vinod H I <vinvinod@xxxxxxxxx>]

Hi,

I am trying to test disaster recovery of rados gateways.

I setup a federated architecture for rados gateway as explained in the docs.

I am using ceph version - 0.80.7

I have setup only one region, "us", with two zones. 

    "us-west" slave zone having user "us-east"

    "us-east" master zone having user "us-east"

The details of specific users are given below. 

Details of user for us-east-1 gateway.

{

• "user_id":"us-east",
• "display_name":"Region-US Zone-East",
• "email":"",
• "suspended":0,
• "max_buckets":1000,
• "auid":0,
• "subusers":[
  1. {
     • "id":"us-east:swift",
     • "permissions":"full-control"
     }
  ],
• "keys":[
  1. {
     • "user":"us-east:swift",
     • "access_key":"0DQH33TDOLDPNUOHDGLX",
     • "secret_key":""
     },
  2. {
     • "user":"us-east",
     • "access_key":"PAA0BEG7ALEEDYXOJ7NE",
     • "secret_key":"BBQNeJ9il5lVWU0u897KK3oJRcifQcQdntuqNufu"
     }
  ],
• "swift_keys":[
  1. {
     • "user":"us-east:swift",
     • "secret_key":"yLbRVIs7QIWcSYLS8KMqzdGWyc3LaKqqvaXJNdF6"
     }
  ],
• "caps":[
  ],
• "op_mask":"read, write, delete",
• "system":"true",
• "default_placement":"",
• "placement_tags":[
  ],
• "bucket_quota":{
  • "enabled":false,
  • "max_size_kb":-1,
  • "max_objects":-1
  },
• "user_quota":{
  • "enabled":false,
  • "max_size_kb":-1,
  • "max_objects":-1
  },
• "temp_url_keys":[
  ]

}

Details of user for us-west-1 gateway 

{

• "user_id":"us-west",
• "display_name":"Region-US Zone-West",
• "email":"",
• "suspended":0,
• "max_buckets":1000,
• "auid":0,
• "subusers":[
  1. {
     • "id":"us-west:swift",
     • "permissions":"full-control"
     }
  ],
• "keys":[
  1. {
     • "user":"us-west:swift",
     • "access_key":"ABAI43X3JZ2LE734XC71",
     • "secret_key":""
     },
  2. {
     • "user":"us-west",
     • "access_key":"98VDZ8ZTWZMFAT1YWXIL",
     • "secret_key":"wKQfBqJtYCZ4VSK26JIYN9tad2GC6t9BKyUsHEb3"
     }
  ],
• "swift_keys":[
  1. {
     • "user":"us-west:swift",
     • "secret_key":"KrjdheLazRpMRzUIpzLgxd0pjN81quFlnp97pwHs"
     }
  ],
• "caps":[
  ],
• "op_mask":"read, write, delete",
• "system":"true",
• "default_placement":"",
• "placement_tags":[
  ],
• "bucket_quota":{
  • "enabled":false,
  • "max_size_kb":-1,
  • "max_objects":-1
  },
• "user_quota":{
  • "enabled":false,
  • "max_size_kb":-1,
  • "max_objects":-1
  },
• "temp_url_keys":[
  ]

}

Now I created a bucket in "us-east" zone with read permissions for all.

vinod@LT05:~$ swift --verbose  -A http://us-east-1.lt.com/auth -U us-east:swift 

-K yLbRVIs7QIWcSYLS8KMqzdGWyc3LaKqqvaXJNdF6 stat Container1

     Account : v1

   Container : Container1

      Objects : 0

          Byte  : 0

   Read ACL: .r:*

   Write ACL : 

     Sync To  : 

   Sync Key  : 

           Vary  : Accept-Encoding

         Serve  : Apache/2.2.22 (Ubuntu)

   X-Container-Bytes-Used-Actual: 0

 Content-Type : text/plain; charset=utf-8

There are no containers on the us-west zone.

When i try to create a new container directly in the us-west zone, it returns status-403.

I guess this is because its the slave zone. 

But the doc says "You may read objects from secondary zones. 

Currently, the Gateway does not prevent you from writing to a secondary zone, but DON’T DO IT."

I am just curious why am I not able to create containers!

Now I sync zones using 'radosgw-agent' , using the command

sudo radosgw-agent --dest-access-key=wKQfBqJtYCZ4VSK26JIYN9tad2GC6t9BKyUsHEb3 

--dest-secret-key=wKQfBqJtYCZ4VSK26JIYN9tad2GC6t9BKyUsHEb3 

--src-access-key=PAA0BEG7ALEEDYXOJ7NE 

--src-secret-key=BBQNeJ9il5lVWU0u897KK3oJRcifQcQdntuqNufu 

--source=http://us-east-1.lt.com 

--sync-scope=full --log-file=/var/log/radosgw/zone-sync-us-east-west.log 

http://us-west-1.lt.com

There are no error logged during this process.

But I am not able to see this container on us-west zone.

vinod@LT05:~$ swift --verbose  -A http://us-west-1.lt.com/auth -U us-west:swift 

-K KrjdheLazRpMRzUIpzLgxd0pjN81quFlnp97pwHs stat

StorageURL: http://us-west-1.lt.com/swift/v1

Auth Token: AUTH_rgwtk0d00000075732d776573743a7377696674080418ee247db6d6c5986c54a00cc1145bcd8fba363322c25ba6508535b5f513c29b3a53

   Account  : v1

Containers : 0

   Objects   : 0

     Bytes    : 0

      Vary     : Accept-Encoding

    Server    : Apache/2.2.22 (Ubuntu)

X-Account-Bytes-Used-Actual: 0

Content-Type: text/plain; charset=utf-8

How can I access the container from us-west-1 rgw instance.

Do I need to manually create the us-east user on the us-west-1 instance also?

Now the there is common storage cluster for both the zones. Is it that the replication will work

only when the storage clusters are different?

--

Vinod H I

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com