If that's what you set in the config file, I assume that's what passed in. Why did you set that in your config file? You don't have a zonegroup named 'cn', right? On Mon, Feb 26, 2018 at 1:10 PM, David Turner <drakonstein@xxxxxxxxx> wrote: > I'm also not certain how to do the tcpdump for this. Do you have any > pointers to how to capture that for you? > > On Mon, Feb 26, 2018 at 4:09 PM David Turner <drakonstein@xxxxxxxxx> wrote: >> >> That's what I set it to in the config file. I probably should have >> mentioned that. >> >> On Mon, Feb 26, 2018 at 4:07 PM Yehuda Sadeh-Weinraub <yehuda@xxxxxxxxxx> >> wrote: >>> >>> According to the log here, it says that the location constraint it got >>> is "cn", can you take a look at a tcpdump, see if that's actually >>> what's passed in? >>> >>> On Mon, Feb 26, 2018 at 12:02 PM, David Turner <drakonstein@xxxxxxxxx> >>> wrote: >>> > I run with `debug rgw = 10` and was able to find these lines at the end >>> > of a >>> > request to create the bucket. >>> > >>> > Successfully creating a bucket with `bucket_location = US` looks like >>> > [1]this. Failing to create a bucket has "ERROR: S3 error: 400 >>> > (InvalidLocationConstraint): The specified location-constraint is not >>> > valid" >>> > on the CLI and [2]this (excerpt from the end of the request) in the rgw >>> > log >>> > (debug level 10). "create bucket location constraint" was not found in >>> > the >>> > log for successfully creating the bucket. >>> > >>> > >>> > [1] >>> > 2018-02-26 19:52:36.419251 7f4bc9bc8700 10 cache put: >>> > >>> > name=local-atl.rgw.data.root++.bucket.meta.testerton:bef43c26-daf3-47ef-a3a5-e1167e3f88ac.39099765.1 >>> > info.flags=0x17 >>> > 2018-02-26 19:52:36.419262 7f4bc9bc8700 10 adding >>> > >>> > local-atl.rgw.data.root++.bucket.meta.testerton:bef43c26-daf3-47ef-a3a5-e1167e3f88ac.39099765.1 >>> > to cache LRU end >>> > 2018-02-26 19:52:36.419266 7f4bc9bc8700 10 updating xattr: >>> > name=user.rgw.acl >>> > bl.length()=141 >>> > 2018-02-26 19:52:36.423863 7f4bc9bc8700 10 RGWWatcher::handle_notify() >>> > notify_id 344855809097728 cookie 139963970426880 notifier 39099765 >>> > bl.length()=361 >>> > 2018-02-26 19:52:36.423875 7f4bc9bc8700 10 cache put: >>> > name=local-atl.rgw.data.root++testerton info.flags=0x17 >>> > 2018-02-26 19:52:36.423882 7f4bc9bc8700 10 adding >>> > local-atl.rgw.data.root++testerton to cache LRU end >>> > >>> > [2] >>> > 2018-02-26 19:43:37.340289 7f466bbca700 2 req 428078:0.004204:s3:PUT >>> > /testraint/:create_bucket:executing >>> > 2018-02-26 19:43:37.340366 7f466bbca700 5 NOTICE: call to >>> > do_aws4_auth_completion >>> > 2018-02-26 19:43:37.340472 7f466bbca700 10 v4 auth ok -- >>> > do_aws4_auth_completion >>> > 2018-02-26 19:43:37.340715 7f466bbca700 10 create bucket location >>> > constraint: cn >>> > 2018-02-26 19:43:37.340766 7f466bbca700 0 location constraint (cn) >>> > can't be >>> > found. >>> > 2018-02-26 19:43:37.340794 7f466bbca700 2 req 428078:0.004701:s3:PUT >>> > /testraint/:create_bucket:completing >>> > 2018-02-26 19:43:37.341782 7f466bbca700 2 req 428078:0.005689:s3:PUT >>> > /testraint/:create_bucket:op status=-2208 >>> > 2018-02-26 19:43:37.341792 7f466bbca700 2 req 428078:0.005707:s3:PUT >>> > /testraint/:create_bucket:http status=400 >>> > >>> > On Mon, Feb 26, 2018 at 2:36 PM Yehuda Sadeh-Weinraub >>> > <yehuda@xxxxxxxxxx> >>> > wrote: >>> >> >>> >> I'm not sure if the rgw logs (debug rgw = 20) specify explicitly why a >>> >> bucket creation is rejected in these cases, but it might be worth >>> >> trying to look at these. If not, then a tcpdump of the specific failed >>> >> request might shed some light (would be interesting to look at the >>> >> generated LocationConstraint). >>> >> >>> >> Yehuda >>> >> >>> >> On Mon, Feb 26, 2018 at 11:29 AM, David Turner <drakonstein@xxxxxxxxx> >>> >> wrote: >>> >> > Our problem only appeared to be present in bucket creation. >>> >> > Listing, >>> >> > putting, etc objects in a bucket work just fine regardless of the >>> >> > bucket_location setting. I ran this test on a few different realms >>> >> > to >>> >> > see >>> >> > what would happen and only 1 of them had a problem. There isn't an >>> >> > obvious >>> >> > thing that steps out about it. The 2 local realms do not have >>> >> > multi-site, >>> >> > the internal realm has multi-site and the operations were performed >>> >> > on >>> >> > the >>> >> > primary zone for the zonegroup. >>> >> > >>> >> > Worked with non 'US' bucket_location for s3cmd to create bucket: >>> >> > realm=internal >>> >> > zonegroup=internal-ga >>> >> > zone=internal-atl >>> >> > >>> >> > Failed with non 'US' bucket_location for s3cmd to create bucket: >>> >> > realm=local-atl >>> >> > zonegroup=local-atl >>> >> > zone=local-atl >>> >> > >>> >> > Worked with non 'US' bucket_location for s3cmd to create bucket: >>> >> > realm=local >>> >> > zonegroup=local >>> >> > zone=local >>> >> > >>> >> > I was thinking it might have to do with all of the parts being named >>> >> > the >>> >> > same, but I made sure to do the last test to confirm. Interestingly >>> >> > it's >>> >> > only bucket creation that has a problem and it's fine as long as I >>> >> > put >>> >> > 'US' >>> >> > as the bucket_location. >>> >> > >>> >> > On Mon, Feb 19, 2018 at 6:48 PM F21 <f21.groups@xxxxxxxxx> wrote: >>> >> >> >>> >> >> I am using the official ceph/daemon docker image. It starts RGW and >>> >> >> creates a zonegroup and zone with their names set to an empty >>> >> >> string: >>> >> >> >>> >> >> >>> >> >> >>> >> >> https://github.com/ceph/ceph-container/blob/master/ceph-releases/luminous/ubuntu/16.04/daemon/start_rgw.sh#L36:54 >>> >> >> >>> >> >> $RGW_ZONEGROUP and $RGW_ZONE are both empty strings by default: >>> >> >> >>> >> >> >>> >> >> >>> >> >> https://github.com/ceph/ceph-container/blob/master/ceph-releases/luminous/ubuntu/16.04/daemon/variables_entrypoint.sh#L46 >>> >> >> >>> >> >> Here's what I get when I query RGW: >>> >> >> >>> >> >> $ radosgw-admin zonegroup list >>> >> >> { >>> >> >> "default_info": "", >>> >> >> "zonegroups": [ >>> >> >> "default" >>> >> >> ] >>> >> >> } >>> >> >> >>> >> >> $ radosgw-admin zone list >>> >> >> { >>> >> >> "default_info": "", >>> >> >> "zones": [ >>> >> >> "default" >>> >> >> ] >>> >> >> } >>> >> >> >>> >> >> On 20/02/2018 10:33 AM, Yehuda Sadeh-Weinraub wrote: >>> >> >> > What is the name of your zonegroup? >>> >> >> > >>> >> >> > On Mon, Feb 19, 2018 at 3:29 PM, F21 <f21.groups@xxxxxxxxx> >>> >> >> > wrote: >>> >> >> >> I've done some debugging and the LocationConstraint is not being >>> >> >> >> set >>> >> >> >> by >>> >> >> >> the >>> >> >> >> SDK by default. >>> >> >> >> >>> >> >> >> I do, however, need to set the region on the client to us-east-1 >>> >> >> >> for >>> >> >> >> it >>> >> >> >> to >>> >> >> >> work. Anything else will return an InvalidLocationConstraint >>> >> >> >> error. >>> >> >> >> >>> >> >> >> Francis >>> >> >> >> >>> >> >> >> >>> >> >> >> On 20/02/2018 8:40 AM, Yehuda Sadeh-Weinraub wrote: >>> >> >> >>> Sounds like the go sdk adds a location constraint to requests >>> >> >> >>> that >>> >> >> >>> don't go to us-east-1. RGW itself is definitely isn't tied to >>> >> >> >>> us-east-1, and does not know anything about it (unless you >>> >> >> >>> happen >>> >> >> >>> to >>> >> >> >>> have a zonegroup named us-east-1). Maybe there's a way to >>> >> >> >>> configure >>> >> >> >>> the sdk to avoid doing that? >>> >> >> >>> >>> >> >> >>> Yehuda >>> >> >> >>> >>> >> >> >>> On Sun, Feb 18, 2018 at 1:54 PM, F21 <f21.groups@xxxxxxxxx> >>> >> >> >>> wrote: >>> >> >> >>>> I am using the AWS Go SDK v2 >>> >> >> >>>> (https://github.com/aws/aws-sdk-go-v2) >>> >> >> >>>> to >>> >> >> >>>> talk >>> >> >> >>>> to my RGW instance using the s3 interface. I am running ceph >>> >> >> >>>> in >>> >> >> >>>> docker >>> >> >> >>>> using >>> >> >> >>>> the ceph/daemon docker images in demo mode. The RGW is started >>> >> >> >>>> with a >>> >> >> >>>> zonegroup and zone with their names set to an empty string by >>> >> >> >>>> the >>> >> >> >>>> scripts >>> >> >> >>>> in >>> >> >> >>>> the image. >>> >> >> >>>> >>> >> >> >>>> I have ForcePathStyle for the client set to true, because I >>> >> >> >>>> want >>> >> >> >>>> to >>> >> >> >>>> access >>> >> >> >>>> all my buckets using the path: myrgw.instance:8080/somebucket. >>> >> >> >>>> >>> >> >> >>>> I noticed that if I set the region for the client to anything >>> >> >> >>>> other >>> >> >> >>>> than >>> >> >> >>>> us-east-1, I get this error when creating a bucket: >>> >> >> >>>> InvalidLocationConstraint: The specified location-constraint >>> >> >> >>>> is >>> >> >> >>>> not >>> >> >> >>>> valid. >>> >> >> >>>> >>> >> >> >>>> If I set the region in the client to something made up, such >>> >> >> >>>> as >>> >> >> >>>> "ceph" >>> >> >> >>>> and >>> >> >> >>>> the LocationConstraint to "ceph", I still get the same error. >>> >> >> >>>> >>> >> >> >>>> The only way to get my buckets to create successfully is to >>> >> >> >>>> set >>> >> >> >>>> the >>> >> >> >>>> client's >>> >> >> >>>> region to us-east-1. I have grepped the ceph code base and >>> >> >> >>>> cannot >>> >> >> >>>> find >>> >> >> >>>> any >>> >> >> >>>> references to us-east-1. In addition, I looked at the AWS docs >>> >> >> >>>> for >>> >> >> >>>> calculating v4 signatures and us-east-1 is the default region >>> >> >> >>>> but >>> >> >> >>>> I >>> >> >> >>>> can >>> >> >> >>>> see >>> >> >> >>>> that the region string is used in the calculation (i.e. the >>> >> >> >>>> region >>> >> >> >>>> is >>> >> >> >>>> not >>> >> >> >>>> ignored when calculating the signature if it is set to >>> >> >> >>>> us-east-1). >>> >> >> >>>> >>> >> >> >>>> Why do my buckets create successfully if I set the region in >>> >> >> >>>> my s3 >>> >> >> >>>> client >>> >> >> >>>> to >>> >> >> >>>> us-east-1, but not otherwise? If I do not want to use >>> >> >> >>>> us-east-1 as >>> >> >> >>>> my >>> >> >> >>>> default region, for example, if I want us-west-1 as my default >>> >> >> >>>> region, >>> >> >> >>>> what >>> >> >> >>>> should I be configuring in ceph? >>> >> >> >>>> >>> >> >> >>>> Thanks, >>> >> >> >>>> >>> >> >> >>>> Francis >>> >> >> >>>> >>> >> >> >>>> _______________________________________________ >>> >> >> >>>> ceph-users mailing list >>> >> >> >>>> ceph-users@xxxxxxxxxxxxxx >>> >> >> >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >> >> >> >>> >> >> >> >>> >> >> >>> >> >> _______________________________________________ >>> >> >> ceph-users mailing list >>> >> >> ceph-users@xxxxxxxxxxxxxx >>> >> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |