On Sat, Nov 22, 2014 at 12:47 AM, Vinod H I <vinvinod@xxxxxxxxx> wrote:
> Thanks for the clarification.
> Now I have done exactly as you suggested.
> "us-east" is the master zone and "us-west" is the secondary zone.
> Each zone has two system users "us-east" and "us-west".
> These system users have same access/secret keys in both zones.
> I have checked the pools to confirm that the non-system swift user which i
> created("east-user:swift") in the primary has been replicated to the
> secondary zone.
> The buckets which are created in primary by the swift user are also there in
> the pools of the secondary zone.
> But when i try to authenticate this swift user in secondary zone, it says
> access denied.
>
> Here are the relevant logs from the secondary zone, when i try to
> authenticate the swift user.
>
> 2014-11-22 14:19:14.239976 7f73ecff9700 2
> RGWDataChangesLog::ChangesRenewThread: start
> 2014-11-22 14:19:14.243454 7f73fe236780 20 get_obj_state: rctx=0x2316ce0
> obj=.us.rgw.root:region_info.us state=0x2319048 s->prefetch_data=0
> 2014-11-22 14:19:14.243454 7f73fe236780 10 cache get:
> name=.us.rgw.root+region_info.us : miss
> 2014-11-22 14:19:14.252263 7f73fe236780 10 cache put:
> name=.us.rgw.root+region_info.us
> 2014-11-22 14:19:14.252283 7f73fe236780 10 adding
> .us.rgw.root+region_info.us to cache LRU end
> 2014-11-22 14:19:14.252310 7f73fe236780 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:14.252336 7f73fe236780 10 cache get:
> name=.us.rgw.root+region_info.us : type miss (requested=1, cached=6)
> 2014-11-22 14:19:14.252376 7f73fe236780 20 get_obj_state: rctx=0x2316ce0
> obj=.us.rgw.root:region_info.us state=0x2319958 s->prefetch_data=0
> 2014-11-22 14:19:14.252386 7f73fe236780 10 cache get:
> name=.us.rgw.root+region_info.us : hit
> 2014-11-22 14:19:14.252391 7f73fe236780 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:14.252404 7f73fe236780 20 get_obj_state: rctx=0x2316ce0
> obj=.us.rgw.root:region_info.us state=0x2319958 s->prefetch_data=0
> 2014-11-22 14:19:14.252409 7f73fe236780 20 state for
> obj=.us.rgw.root:region_info.us is not atomic, not appending atomic test
> 2014-11-22 14:19:14.252412 7f73fe236780 20 rados->read obj-ofs=0 read_ofs=0
> read_len=524288
> 2014-11-22 14:19:14.264611 7f73fe236780 20 rados->read r=0 bl.length=266
> 2014-11-22 14:19:14.264650 7f73fe236780 10 cache put:
> name=.us.rgw.root+region_info.us
> 2014-11-22 14:19:14.264653 7f73fe236780 10 moving
> .us.rgw.root+region_info.us to cache LRU end
> 2014-11-22 14:19:14.264766 7f73fe236780 20 get_obj_state: rctx=0x2319860
> obj=.us-west.rgw.root:zone_info.us-west state=0x2313b98 s->prefetch_data=0
> 2014-11-22 14:19:14.264779 7f73fe236780 10 cache get:
> name=.us-west.rgw.root+zone_info.us-west : miss
> 2014-11-22 14:19:14.276114 7f73fe236780 10 cache put:
> name=.us-west.rgw.root+zone_info.us-west
> 2014-11-22 14:19:14.276131 7f73fe236780 10 adding
> .us-west.rgw.root+zone_info.us-west to cache LRU end
> 2014-11-22 14:19:14.276142 7f73fe236780 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:14.276161 7f73fe236780 10 cache get:
> name=.us-west.rgw.root+zone_info.us-west : type miss (requested=1, cached=6)
> 2014-11-22 14:19:14.276203 7f73fe236780 20 get_obj_state: rctx=0x2314660
> obj=.us-west.rgw.root:zone_info.us-west state=0x2313b98 s->prefetch_data=0
> 2014-11-22 14:19:14.276212 7f73fe236780 10 cache get:
> name=.us-west.rgw.root+zone_info.us-west : hit
> 2014-11-22 14:19:14.276218 7f73fe236780 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:14.276229 7f73fe236780 20 get_obj_state: rctx=0x2314660
> obj=.us-west.rgw.root:zone_info.us-west state=0x2313b98 s->prefetch_data=0
> 2014-11-22 14:19:14.276235 7f73fe236780 20 state for
> obj=.us-west.rgw.root:zone_info.us-west is not atomic, not appending atomic
> test
> 2014-11-22 14:19:14.276238 7f73fe236780 20 rados->read obj-ofs=0 read_ofs=0
> read_len=524288
> 2014-11-22 14:19:14.290757 7f73fe236780 20 rados->read r=0 bl.length=997
> 2014-11-22 14:19:14.290797 7f73fe236780 10 cache put:
> name=.us-west.rgw.root+zone_info.us-west
> 2014-11-22 14:19:14.290803 7f73fe236780 10 moving
> .us-west.rgw.root+zone_info.us-west to cache LRU end
> 2014-11-22 14:19:14.290857 7f73fe236780 2 zone us-west is NOT master
> 2014-11-22 14:19:14.290931 7f73fe236780 20 get_obj_state: rctx=0x2313cc0
> obj=.us-west.rgw.root:region_map state=0x2311e08 s->prefetch_data=0
> 2014-11-22 14:19:14.290949 7f73fe236780 10 cache get:
> name=.us-west.rgw.root+region_map : miss
> 2014-11-22 14:19:14.298169 7f73fe236780 10 cache put:
> name=.us-west.rgw.root+region_map
> 2014-11-22 14:19:14.298184 7f73fe236780 10 adding
> .us-west.rgw.root+region_map to cache LRU end
> 2014-11-22 14:19:14.298195 7f73fe236780 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:14.298212 7f73fe236780 10 cache get:
> name=.us-west.rgw.root+region_map : type miss (requested=1, cached=6)
> 2014-11-22 14:19:14.298255 7f73fe236780 20 get_obj_state: rctx=0x2313cc0
> obj=.us-west.rgw.root:region_map state=0x2311e08 s->prefetch_data=0
> 2014-11-22 14:19:14.298267 7f73fe236780 10 cache get:
> name=.us-west.rgw.root+region_map : hit
> 2014-11-22 14:19:14.298272 7f73fe236780 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:14.298282 7f73fe236780 20 get_obj_state: rctx=0x2313cc0
> obj=.us-west.rgw.root:region_map state=0x2311e08 s->prefetch_data=0
> 2014-11-22 14:19:14.298286 7f73fe236780 20 state for
> obj=.us-west.rgw.root:region_map is not atomic, not appending atomic test
> 2014-11-22 14:19:14.298288 7f73fe236780 20 rados->read obj-ofs=0 read_ofs=0
> read_len=524288
> 2014-11-22 14:19:14.300462 7f73fe236780 20 rados->read r=0 bl.length=334
> 2014-11-22 14:19:14.300486 7f73fe236780 10 cache put:
> name=.us-west.rgw.root+region_map
> 2014-11-22 14:19:14.300490 7f73fe236780 10 moving
> .us-west.rgw.root+region_map to cache LRU end
> 2014-11-22 14:19:14.545543 7f73fe236780 20 generating connection object for
> zone us-east
> 2014-11-22 14:19:14.548178 7f73fe236780 0 framework: fastcgi
> 2014-11-22 14:19:14.548767 7f73fe236780 0 starting handler: fastcgi
> 2014-11-22 14:19:14.549768 7f73ceffd700 20 UserSyncThread: start
> 2014-11-22 14:19:14.550665 7f73cf7fe700 20 BucketsSyncThread: start
> 2014-11-22 14:19:14.553127 7f73cdffb700 10 allocated request
> req=0x7f73e000d010
> 2014-11-22 14:19:14.553458 7f73cffff700 2 garbage collection: start
> 2014-11-22 14:19:14.576569 7f73ceffd700 20 RGWRados::pool_iterate: got
> east-user.buckets
> 2014-11-22 14:19:14.731146 7f73ceffd700 20 RGWRados::pool_iterate: got
> us-west
> 2014-11-22 14:19:14.771842 7f73ceffd700 20 RGWRados::pool_iterate: got
> us-east
> 2014-11-22 14:19:14.803904 7f73ceffd700 20 RGWRados::pool_iterate: got
> east-user
> 2014-11-22 14:19:14.834887 7f73ceffd700 20 RGWUserStatsCache: sync
> user=us-west
> 2014-11-22 14:19:14.839166 7f73ceffd700 0 ERROR: can't read user header:
> ret=-2
> 2014-11-22 14:19:14.839182 7f73ceffd700 0 ERROR: sync_user() failed,
> user=us-west ret=-2
> 2014-11-22 14:19:14.839186 7f73ceffd700 20 RGWUserStatsCache: sync
> user=us-east
> 2014-11-22 14:19:14.846530 7f73ceffd700 0 ERROR: can't read user header:
> ret=-2
> 2014-11-22 14:19:14.846540 7f73ceffd700 0 ERROR: sync_user() failed,
> user=us-east ret=-2
> 2014-11-22 14:19:14.846543 7f73ceffd700 20 RGWUserStatsCache: sync
> user=east-user
> 2014-11-22 14:19:14.852550 7f73ceffd700 20 user is idle, not doing a full
> sync (user=east-user)
> 2014-11-22 14:19:15.994741 7f73cffff700 2 garbage collection: stop
> 2014-11-22 14:19:21.917609 7f73cdffb700 20 enqueued request
> req=0x7f73e000d010
> 2014-11-22 14:19:21.917657 7f73cdffb700 20 RGWWQ:
> 2014-11-22 14:19:21.917661 7f73cdffb700 20 req: 0x7f73e000d010
> 2014-11-22 14:19:21.917679 7f73cdffb700 10 allocated request
> req=0x7f73e0013dc0
> 2014-11-22 14:19:21.918043 7f73b07c0700 20 dequeued request
> req=0x7f73e000d010
> 2014-11-22 14:19:21.918067 7f73b07c0700 20 RGWWQ: empty
> 2014-11-22 14:19:21.918220 7f73b07c0700 20 DOCUMENT_ROOT=/var/www
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 FCGI_ROLE=RESPONDER
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 GATEWAY_INTERFACE=CGI/1.1
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_ACCEPT_ENCODING=identity
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_AUTHORIZATION=
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_HOST=us-west-1.lt.com
> 2014-11-22 14:19:21.918228 7f73b07c0700 20
> HTTP_X_AUTH_KEY=MHA4vFaDy5XsJqpF5NuZLcBMCoJcuot44ASDuReY
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 HTTP_X_AUTH_USER=east-user:swift
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 PATH=/usr/local/bin:/usr/bin:/bin
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 QUERY_STRING=
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 REMOTE_ADDR=192.168.7.141
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 REMOTE_PORT=50857
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 REQUEST_METHOD=GET
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 REQUEST_URI=/auth/
> 2014-11-22 14:19:21.918228 7f73b07c0700 20
> SCRIPT_FILENAME=/var/www/s3gw.fcgi
> 2014-11-22 14:19:21.918228 7f73b07c0700 20 SCRIPT_NAME=/auth/
> 2014-11-22 14:19:21.918228 7f73b07c0700 20
> SCRIPT_URI=http://us-west-1.lt.com/auth/
> 2014-11-22 14:19:21.918229 7f73b07c0700 20 SCRIPT_URL=/auth/
> 2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_ADDR=192.168.7.117
> 2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_ADMIN=vinvinod@xxxxxxxxx
> 2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_NAME=us-west-1.lt.com
> 2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_PORT=80
> 2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_PROTOCOL=HTTP/1.1
> 2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_SIGNATURE=
> 2014-11-22 14:19:21.918229 7f73b07c0700 20 SERVER_SOFTWARE=Apache/2.2.22
> (Ubuntu)
> 2014-11-22 14:19:21.918229 7f73b07c0700 1 ====== starting new request
> req=0x7f73e000d010 =====
> 2014-11-22 14:19:21.918229 7f73b07c0700 2 req 1:0.000000::GET
> /auth/::initializing
> 2014-11-22 14:19:21.918229 7f73b07c0700 10 host=us-west-1.lt.com
> rgw_dns_name=us-west-1.lt.com
> 2014-11-22 14:19:21.918288 7f73b07c0700 2 req 1:0.000053:swift-auth:GET
> /auth/::getting op
> 2014-11-22 14:19:21.918300 7f73b07c0700 2 req 1:0.000071:swift-auth:GET
> /auth/:swift_auth_get:authorizing
> 2014-11-22 14:19:21.918307 7f73b07c0700 2 req 1:0.000078:swift-auth:GET
> /auth/:swift_auth_get:reading permissions
> 2014-11-22 14:19:21.918313 7f73b07c0700 2 req 1:0.000084:swift-auth:GET
> /auth/:swift_auth_get:init op
> 2014-11-22 14:19:21.918319 7f73b07c0700 2 req 1:0.000090:swift-auth:GET
> /auth/:swift_auth_get:verifying op mask
> 2014-11-22 14:19:21.918325 7f73b07c0700 20 required_mask= 0 user.op_mask=7
> 2014-11-22 14:19:21.918330 7f73b07c0700 2 req 1:0.000100:swift-auth:GET
> /auth/:swift_auth_get:verifying op permissions
> 2014-11-22 14:19:21.918336 7f73b07c0700 2 req 1:0.000107:swift-auth:GET
> /auth/:swift_auth_get:verifying op params
> 2014-11-22 14:19:21.918341 7f73b07c0700 2 req 1:0.000112:swift-auth:GET
> /auth/:swift_auth_get:executing
> 2014-11-22 14:19:21.918470 7f73b07c0700 20 get_obj_state:
> rctx=0x7f73dc002030 obj=.us-west.users.swift:east-user:swift
> state=0x7f73dc0066d8 s->prefetch_data=0
> 2014-11-22 14:19:21.918494 7f73b07c0700 10 cache get:
> name=.us-west.users.swift+east-user:swift : miss
> 2014-11-22 14:19:21.931892 7f73b07c0700 10 cache put:
> name=.us-west.users.swift+east-user:swift
> 2014-11-22 14:19:21.931892 7f73b07c0700 10 adding
> .us-west.users.swift+east-user:swift to cache LRU end
> 2014-11-22 14:19:21.931892 7f73b07c0700 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:21.931892 7f73b07c0700 10 cache get:
> name=.us-west.users.swift+east-user:swift : type miss (requested=1,
> cached=6)
> 2014-11-22 14:19:21.931893 7f73b07c0700 20 get_obj_state:
> rctx=0x7f73dc007300 obj=.us-west.users.swift:east-user:swift
> state=0x7f73dc006558 s->prefetch_data=0
> 2014-11-22 14:19:21.931893 7f73b07c0700 10 cache get:
> name=.us-west.users.swift+east-user:swift : hit
> 2014-11-22 14:19:21.931893 7f73b07c0700 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:21.931893 7f73b07c0700 20 get_obj_state:
> rctx=0x7f73dc007300 obj=.us-west.users.swift:east-user:swift
> state=0x7f73dc006558 s->prefetch_data=0
> 2014-11-22 14:19:21.931893 7f73b07c0700 20 state for
> obj=.us-west.users.swift:east-user:swift is not atomic, not appending atomic
> test
> 2014-11-22 14:19:21.931893 7f73b07c0700 20 rados->read obj-ofs=0 read_ofs=0
> read_len=524288
> 2014-11-22 14:19:21.932003 7f73b07c0700 20 rados->read r=0 bl.length=13
> 2014-11-22 14:19:21.932021 7f73b07c0700 10 cache put:
> name=.us-west.users.swift+east-user:swift
> 2014-11-22 14:19:21.932023 7f73b07c0700 10 moving
> .us-west.users.swift+east-user:swift to cache LRU end
> 2014-11-22 14:19:21.932054 7f73b07c0700 20 get_obj_state:
> rctx=0x7f73dc006b30 obj=.us-west.users.uid:east-user state=0x7f73dc006498
> s->prefetch_data=0
> 2014-11-22 14:19:21.932062 7f73b07c0700 10 cache get:
> name=.us-west.users.uid+east-user : miss
> 2014-11-22 14:19:21.933559 7f73b07c0700 10 cache put:
> name=.us-west.users.uid+east-user
> 2014-11-22 14:19:21.933567 7f73b07c0700 10 adding
> .us-west.users.uid+east-user to cache LRU end
> 2014-11-22 14:19:21.933572 7f73b07c0700 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:21.933580 7f73b07c0700 10 cache get:
> name=.us-west.users.uid+east-user : type miss (requested=1, cached=6)
> 2014-11-22 14:19:21.933601 7f73b07c0700 20 get_obj_state:
> rctx=0x7f73dc006b30 obj=.us-west.users.uid:east-user state=0x7f73dc006498
> s->prefetch_data=0
> 2014-11-22 14:19:21.933607 7f73b07c0700 10 cache get:
> name=.us-west.users.uid+east-user : hit
> 2014-11-22 14:19:21.933611 7f73b07c0700 20 get_obj_state: s->obj_tag was set
> empty
> 2014-11-22 14:19:21.933617 7f73b07c0700 20 get_obj_state:
> rctx=0x7f73dc006b30 obj=.us-west.users.uid:east-user state=0x7f73dc006498
> s->prefetch_data=0
> 2014-11-22 14:19:21.933620 7f73b07c0700 20 state for
> obj=.us-west.users.uid:east-user is not atomic, not appending atomic test
> 2014-11-22 14:19:21.933622 7f73b07c0700 20 rados->read obj-ofs=0 read_ofs=0
> read_len=524288
> 2014-11-22 14:19:21.934709 7f73b07c0700 20 rados->read r=0 bl.length=310
> 2014-11-22 14:19:21.934725 7f73b07c0700 10 cache put:
> name=.us-west.users.uid+east-user
> 2014-11-22 14:19:21.934727 7f73b07c0700 10 moving
> .us-west.users.uid+east-user to cache LRU end
> 2014-11-22 14:19:21.934790 7f73b07c0700 2 req 1:0.016560:swift-auth:GET
> /auth/:swift_auth_get:http status=403
> 2014-11-22 14:19:21.934794 7f73b07c0700 1 ====== req done
> req=0x7f73e000d010 http_status=403 ======
> 2014-11-22 14:19:21.934800 7f73b07c0700 20 process_request() returned -1
>
> Why am I not able to authenticate?
>
What does radosgw-admin show about the user in the secondary zone?
$ radosgw-admin user info --rgw-zone=<zone> --uid=<uid>
Try comparing to the primary zone.
Yehuda
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
