I'm really not sure. I'm using the S3 interface rather than the Swift interface. Once my non-systm user replicated, I was able to access everything in the secondary cluster just fine.
Hopefully somebody else with Swift experience will chime in.
On Sat, Nov 22, 2014 at 12:47 AM, Vinod H I <vinvinod@xxxxxxxxx> wrote:
Thanks for the clarification.Now I have done exactly as you suggested."us-east" is the master zone and "us-west" is the secondary zone.Each zone has two system users "us-east" and "us-west".These system users have same access/secret keys in both zones.I have checked the pools to confirm that the non-system swift user which i created("east-user:swift") in the primary has been replicated to the secondary zone.The buckets which are created in primary by the swift user are also there in the pools of the secondary zone.But when i try to authenticate this swift user in secondary zone, it says access denied.Here are the relevant logs from the secondary zone, when i try to authenticate the swift user.2014-11-22 14:19:14.239976 7f73ecff9700 2 RGWDataChangesLog::ChangesRenewThread: start2014-11-22 14:19:14.243454 7f73fe236780 20 get_obj_state: rctx=0x2316ce0 obj=.us.rgw.root:region_info.us state=0x2319048 s->prefetch_data=02014-11-22 14:19:14.243454 7f73fe236780 10 cache get: name=.us.rgw.root+region_info.us : miss2014-11-22 14:19:14.252263 7f73fe236780 10 cache put: name=.us.rgw.root+region_info.us2014-11-22 14:19:14.252283 7f73fe236780 10 adding .us.rgw.root+region_info.us to cache LRU end2014-11-22 14:19:14.252310 7f73fe236780 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:14.252336 7f73fe236780 10 cache get: name=.us.rgw.root+region_info.us : type miss (requested=1, cached=6)2014-11-22 14:19:14.252376 7f73fe236780 20 get_obj_state: rctx=0x2316ce0 obj=.us.rgw.root:region_info.us state=0x2319958 s->prefetch_data=02014-11-22 14:19:14.252386 7f73fe236780 10 cache get: name=.us.rgw.root+region_info.us : hit2014-11-22 14:19:14.252391 7f73fe236780 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:14.252404 7f73fe236780 20 get_obj_state: rctx=0x2316ce0 obj=.us.rgw.root:region_info.us state=0x2319958 s->prefetch_data=02014-11-22 14:19:14.252409 7f73fe236780 20 state for obj=.us.rgw.root:region_info.us is not atomic, not appending atomic test2014-11-22 14:19:14.252412 7f73fe236780 20 rados->read obj-ofs=0 read_ofs=0 read_len=5242882014-11-22 14:19:14.264611 7f73fe236780 20 rados->read r=0 bl.length=2662014-11-22 14:19:14.264650 7f73fe236780 10 cache put: name=.us.rgw.root+region_info.us2014-11-22 14:19:14.264653 7f73fe236780 10 moving .us.rgw.root+region_info.us to cache LRU end2014-11-22 14:19:14.264766 7f73fe236780 20 get_obj_state: rctx=0x2319860 obj=.us-west.rgw.root:zone_info.us-west state=0x2313b98 s->prefetch_data=02014-11-22 14:19:14.264779 7f73fe236780 10 cache get: name=.us-west.rgw.root+zone_info.us-west : miss2014-11-22 14:19:14.276114 7f73fe236780 10 cache put: name=.us-west.rgw.root+zone_info.us-west2014-11-22 14:19:14.276131 7f73fe236780 10 adding .us-west.rgw.root+zone_info.us-west to cache LRU end2014-11-22 14:19:14.276142 7f73fe236780 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:14.276161 7f73fe236780 10 cache get: name=.us-west.rgw.root+zone_info.us-west : type miss (requested=1, cached=6)2014-11-22 14:19:14.276203 7f73fe236780 20 get_obj_state: rctx=0x2314660 obj=.us-west.rgw.root:zone_info.us-west state=0x2313b98 s->prefetch_data=02014-11-22 14:19:14.276212 7f73fe236780 10 cache get: name=.us-west.rgw.root+zone_info.us-west : hit2014-11-22 14:19:14.276218 7f73fe236780 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:14.276229 7f73fe236780 20 get_obj_state: rctx=0x2314660 obj=.us-west.rgw.root:zone_info.us-west state=0x2313b98 s->prefetch_data=02014-11-22 14:19:14.276235 7f73fe236780 20 state for obj=.us-west.rgw.root:zone_info.us-west is not atomic, not appending atomic test2014-11-22 14:19:14.276238 7f73fe236780 20 rados->read obj-ofs=0 read_ofs=0 read_len=5242882014-11-22 14:19:14.290757 7f73fe236780 20 rados->read r=0 bl.length=9972014-11-22 14:19:14.290797 7f73fe236780 10 cache put: name=.us-west.rgw.root+zone_info.us-west2014-11-22 14:19:14.290803 7f73fe236780 10 moving .us-west.rgw.root+zone_info.us-west to cache LRU end2014-11-22 14:19:14.290857 7f73fe236780 2 zone us-west is NOT master2014-11-22 14:19:14.290931 7f73fe236780 20 get_obj_state: rctx=0x2313cc0 obj=.us-west.rgw.root:region_map state=0x2311e08 s->prefetch_data=02014-11-22 14:19:14.290949 7f73fe236780 10 cache get: name=.us-west.rgw.root+region_map : miss2014-11-22 14:19:14.298169 7f73fe236780 10 cache put: name=.us-west.rgw.root+region_map2014-11-22 14:19:14.298184 7f73fe236780 10 adding .us-west.rgw.root+region_map to cache LRU end2014-11-22 14:19:14.298195 7f73fe236780 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:14.298212 7f73fe236780 10 cache get: name=.us-west.rgw.root+region_map : type miss (requested=1, cached=6)2014-11-22 14:19:14.298255 7f73fe236780 20 get_obj_state: rctx=0x2313cc0 obj=.us-west.rgw.root:region_map state=0x2311e08 s->prefetch_data=02014-11-22 14:19:14.298267 7f73fe236780 10 cache get: name=.us-west.rgw.root+region_map : hit2014-11-22 14:19:14.298272 7f73fe236780 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:14.298282 7f73fe236780 20 get_obj_state: rctx=0x2313cc0 obj=.us-west.rgw.root:region_map state=0x2311e08 s->prefetch_data=02014-11-22 14:19:14.298286 7f73fe236780 20 state for obj=.us-west.rgw.root:region_map is not atomic, not appending atomic test2014-11-22 14:19:14.298288 7f73fe236780 20 rados->read obj-ofs=0 read_ofs=0 read_len=5242882014-11-22 14:19:14.300462 7f73fe236780 20 rados->read r=0 bl.length=3342014-11-22 14:19:14.300486 7f73fe236780 10 cache put: name=.us-west.rgw.root+region_map2014-11-22 14:19:14.300490 7f73fe236780 10 moving .us-west.rgw.root+region_map to cache LRU end2014-11-22 14:19:14.545543 7f73fe236780 20 generating connection object for zone us-east2014-11-22 14:19:14.548178 7f73fe236780 0 framework: fastcgi2014-11-22 14:19:14.548767 7f73fe236780 0 starting handler: fastcgi2014-11-22 14:19:14.549768 7f73ceffd700 20 UserSyncThread: start2014-11-22 14:19:14.550665 7f73cf7fe700 20 BucketsSyncThread: start2014-11-22 14:19:14.553127 7f73cdffb700 10 allocated request req=0x7f73e000d0102014-11-22 14:19:14.553458 7f73cffff700 2 garbage collection: start2014-11-22 14:19:14.576569 7f73ceffd700 20 RGWRados::pool_iterate: got east-user.buckets2014-11-22 14:19:14.731146 7f73ceffd700 20 RGWRados::pool_iterate: got us-west2014-11-22 14:19:14.771842 7f73ceffd700 20 RGWRados::pool_iterate: got us-east2014-11-22 14:19:14.803904 7f73ceffd700 20 RGWRados::pool_iterate: got east-user2014-11-22 14:19:14.834887 7f73ceffd700 20 RGWUserStatsCache: sync user=us-west2014-11-22 14:19:14.839166 7f73ceffd700 0 ERROR: can't read user header: ret=-22014-11-22 14:19:14.839182 7f73ceffd700 0 ERROR: sync_user() failed, user=us-west ret=-22014-11-22 14:19:14.839186 7f73ceffd700 20 RGWUserStatsCache: sync user=us-east2014-11-22 14:19:14.846530 7f73ceffd700 0 ERROR: can't read user header: ret=-22014-11-22 14:19:14.846540 7f73ceffd700 0 ERROR: sync_user() failed, user=us-east ret=-22014-11-22 14:19:14.846543 7f73ceffd700 20 RGWUserStatsCache: sync user=east-user2014-11-22 14:19:14.852550 7f73ceffd700 20 user is idle, not doing a full sync (user=east-user)2014-11-22 14:19:15.994741 7f73cffff700 2 garbage collection: stop2014-11-22 14:19:21.917609 7f73cdffb700 20 enqueued request req=0x7f73e000d0102014-11-22 14:19:21.917657 7f73cdffb700 20 RGWWQ:2014-11-22 14:19:21.917661 7f73cdffb700 20 req: 0x7f73e000d0102014-11-22 14:19:21.917679 7f73cdffb700 10 allocated request req=0x7f73e0013dc02014-11-22 14:19:21.918043 7f73b07c0700 20 dequeued request req=0x7f73e000d0102014-11-22 14:19:21.918067 7f73b07c0700 20 RGWWQ: empty2014-11-22 14:19:21.918220 7f73b07c0700 20 DOCUMENT_ROOT=/var/www2014-11-22 14:19:21.918228 7f73b07c0700 20 FCGI_ROLE=RESPONDER2014-11-22 14:19:21.918228 7f73b07c0700 20 GATEWAY_INTERFACE=CGI/1.12014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_ACCEPT_ENCODING=identity2014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_AUTHORIZATION=2014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_HOST=us-west-1.lt.com2014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_X_AUTH_KEY=MHA4vFaDy5XsJqpF5NuZLcBMCoJcuot44ASDuReY2014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_X_AUTH_USER=east-user:swift2014-11-22 14:19:21.918228 7f73b07c0700 20 PATH=/usr/local/bin:/usr/bin:/bin2014-11-22 14:19:21.918228 7f73b07c0700 20 QUERY_STRING=2014-11-22 14:19:21.918228 7f73b07c0700 20 REMOTE_ADDR=192.168.7.1412014-11-22 14:19:21.918228 7f73b07c0700 20 REMOTE_PORT=508572014-11-22 14:19:21.918228 7f73b07c0700 20 REQUEST_METHOD=GET2014-11-22 14:19:21.918228 7f73b07c0700 20 REQUEST_URI=/auth/2014-11-22 14:19:21.918228 7f73b07c0700 20 SCRIPT_FILENAME=/var/www/s3gw.fcgi2014-11-22 14:19:21.918228 7f73b07c0700 20 SCRIPT_NAME=/auth/2014-11-22 14:19:21.918228 7f73b07c0700 20 SCRIPT_URI=http://us-west-1.lt.com/auth/2014-11-22 14:19:21.918229 7f73b07c0700 20 SCRIPT_URL=/auth/2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_ADDR=192.168.7.1172014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_ADMIN=vinvinod@xxxxxxxxx2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_NAME=us-west-1.lt.com2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_PORT=802014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_PROTOCOL=HTTP/1.12014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_SIGNATURE=2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_SOFTWARE=Apache/2.2.22 (Ubuntu)2014-11-22 14:19:21.918229 7f73b07c0700 1 ====== starting new request req=0x7f73e000d010 =====2014-11-22 14:19:21.918229 7f73b07c0700 2 req 1:0.000000::GET /auth/::initializing2014-11-22 14:19:21.918229 7f73b07c0700 10 host=us-west-1.lt.com rgw_dns_name=us-west-1.lt.com2014-11-22 14:19:21.918288 7f73b07c0700 2 req 1:0.000053:swift-auth:GET /auth/::getting op2014-11-22 14:19:21.918300 7f73b07c0700 2 req 1:0.000071:swift-auth:GET /auth/:swift_auth_get:authorizing2014-11-22 14:19:21.918307 7f73b07c0700 2 req 1:0.000078:swift-auth:GET /auth/:swift_auth_get:reading permissions2014-11-22 14:19:21.918313 7f73b07c0700 2 req 1:0.000084:swift-auth:GET /auth/:swift_auth_get:init op2014-11-22 14:19:21.918319 7f73b07c0700 2 req 1:0.000090:swift-auth:GET /auth/:swift_auth_get:verifying op mask2014-11-22 14:19:21.918325 7f73b07c0700 20 required_mask= 0 user.op_mask=72014-11-22 14:19:21.918330 7f73b07c0700 2 req 1:0.000100:swift-auth:GET /auth/:swift_auth_get:verifying op permissions2014-11-22 14:19:21.918336 7f73b07c0700 2 req 1:0.000107:swift-auth:GET /auth/:swift_auth_get:verifying op params2014-11-22 14:19:21.918341 7f73b07c0700 2 req 1:0.000112:swift-auth:GET /auth/:swift_auth_get:executing2014-11-22 14:19:21.918470 7f73b07c0700 20 get_obj_state: rctx=0x7f73dc002030 obj=.us-west.users.swift:east-user:swift state=0x7f73dc0066d8 s->prefetch_data=02014-11-22 14:19:21.918494 7f73b07c0700 10 cache get: name=.us-west.users.swift+east-user:swift : miss2014-11-22 14:19:21.931892 7f73b07c0700 10 cache put: name=.us-west.users.swift+east-user:swift2014-11-22 14:19:21.931892 7f73b07c0700 10 adding .us-west.users.swift+east-user:swift to cache LRU end2014-11-22 14:19:21.931892 7f73b07c0700 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:21.931892 7f73b07c0700 10 cache get: name=.us-west.users.swift+east-user:swift : type miss (requested=1, cached=6)2014-11-22 14:19:21.931893 7f73b07c0700 20 get_obj_state: rctx=0x7f73dc007300 obj=.us-west.users.swift:east-user:swift state=0x7f73dc006558 s->prefetch_data=02014-11-22 14:19:21.931893 7f73b07c0700 10 cache get: name=.us-west.users.swift+east-user:swift : hit2014-11-22 14:19:21.931893 7f73b07c0700 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:21.931893 7f73b07c0700 20 get_obj_state: rctx=0x7f73dc007300 obj=.us-west.users.swift:east-user:swift state=0x7f73dc006558 s->prefetch_data=02014-11-22 14:19:21.931893 7f73b07c0700 20 state for obj=.us-west.users.swift:east-user:swift is not atomic, not appending atomic test2014-11-22 14:19:21.931893 7f73b07c0700 20 rados->read obj-ofs=0 read_ofs=0 read_len=5242882014-11-22 14:19:21.932003 7f73b07c0700 20 rados->read r=0 bl.length=132014-11-22 14:19:21.932021 7f73b07c0700 10 cache put: name=.us-west.users.swift+east-user:swift2014-11-22 14:19:21.932023 7f73b07c0700 10 moving .us-west.users.swift+east-user:swift to cache LRU end2014-11-22 14:19:21.932054 7f73b07c0700 20 get_obj_state: rctx=0x7f73dc006b30 obj=.us-west.users.uid:east-user state=0x7f73dc006498 s->prefetch_data=02014-11-22 14:19:21.932062 7f73b07c0700 10 cache get: name=.us-west.users.uid+east-user : miss2014-11-22 14:19:21.933559 7f73b07c0700 10 cache put: name=.us-west.users.uid+east-user2014-11-22 14:19:21.933567 7f73b07c0700 10 adding .us-west.users.uid+east-user to cache LRU end2014-11-22 14:19:21.933572 7f73b07c0700 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:21.933580 7f73b07c0700 10 cache get: name=.us-west.users.uid+east-user : type miss (requested=1, cached=6)2014-11-22 14:19:21.933601 7f73b07c0700 20 get_obj_state: rctx=0x7f73dc006b30 obj=.us-west.users.uid:east-user state=0x7f73dc006498 s->prefetch_data=02014-11-22 14:19:21.933607 7f73b07c0700 10 cache get: name=.us-west.users.uid+east-user : hit2014-11-22 14:19:21.933611 7f73b07c0700 20 get_obj_state: s->obj_tag was set empty2014-11-22 14:19:21.933617 7f73b07c0700 20 get_obj_state: rctx=0x7f73dc006b30 obj=.us-west.users.uid:east-user state=0x7f73dc006498 s->prefetch_data=02014-11-22 14:19:21.933620 7f73b07c0700 20 state for obj=.us-west.users.uid:east-user is not atomic, not appending atomic test2014-11-22 14:19:21.933622 7f73b07c0700 20 rados->read obj-ofs=0 read_ofs=0 read_len=5242882014-11-22 14:19:21.934709 7f73b07c0700 20 rados->read r=0 bl.length=3102014-11-22 14:19:21.934725 7f73b07c0700 10 cache put: name=.us-west.users.uid+east-user2014-11-22 14:19:21.934727 7f73b07c0700 10 moving .us-west.users.uid+east-user to cache LRU end2014-11-22 14:19:21.934790 7f73b07c0700 2 req 1:0.016560:swift-auth:GET /auth/:swift_auth_get:http status=4032014-11-22 14:19:21.934794 7f73b07c0700 1 ====== req done req=0x7f73e000d010 http_status=403 ======2014-11-22 14:19:21.934800 7f73b07c0700 20 process_request() returned -1Why am I not able to authenticate?--On Fri, Nov 21, 2014 at 1:04 AM, Craig Lewis <clewis@xxxxxxxxxxxxxxxxxx> wrote:You need to create two system users, in both zones. They should have the same name, access key, and secret in both zones. By convention, these system users are named the same as the zones.You shouldn't use those system users for anything other than replication. You should create a non-system user to interact with the cluster. Just like you don't run as root all the time, you don't want to be a radosgw system user all the time. You only need to create this user in the primary zone.Once replication is working, it should copy the non-system user to the secondary cluster, as well as any buckets and objects this user creates.On Wed, Nov 19, 2014 at 1:16 AM, Vinod H I <vinvinod@xxxxxxxxx> wrote:_______________________________________________Hi,I am using firefly version 0.80.7.I am testing disaster recovery mechanism for rados gateways.I have followed the federated gateway setup as mentioned in the docs.There is one region with two zones on the same cluster.After sync(using radosgw-agent, with "--sync-scope=full"), container created by the swift user(with "--system" flag) on the master zone gateway is not visible for the swift user(with "--system" flag) on the slave zone.There are no error during the syncing process.I tried by creating a new slave zone user with same uid and access and secret keys as that of master. It did not work!Any idea on how to be able to read the synced containers from the slave zone?Is there any requirement that the two zones must be on separate clusters?
--Vinod H I
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Vinod H I
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com