I set it to that for randomness. I don't have a zonegroup named 'us' either, but that works fine. I don't see why 'cn' should be any different. The bucket_location that triggered me noticing this was 'gd1'. I don't know where that one came from, but I don't see why we should force people setting it to 'us' when that has nothing to do with the realm. If it needed to be set to 'local-atl' that would make sense, but 'us' works just fine. Perhaps 'us' working is what shouldn't work as opposed to allowing whatever else to be able to work.
I tested setting bucket_location to 'local-atl' and it did successfully create the bucket. So the question becomes, why do my other realms not care what that value is set to and why does this realm allow 'us' to be used when it isn't correct?
On Mon, Feb 26, 2018 at 4:12 PM Yehuda Sadeh-Weinraub <yehuda@xxxxxxxxxx> wrote:
If that's what you set in the config file, I assume that's what passed
in. Why did you set that in your config file? You don't have a
zonegroup named 'cn', right?
On Mon, Feb 26, 2018 at 1:10 PM, David Turner <drakonstein@xxxxxxxxx> wrote:
> I'm also not certain how to do the tcpdump for this. Do you have any
> pointers to how to capture that for you?
>
> On Mon, Feb 26, 2018 at 4:09 PM David Turner <drakonstein@xxxxxxxxx> wrote:
>>
>> That's what I set it to in the config file. I probably should have
>> mentioned that.
>>
>> On Mon, Feb 26, 2018 at 4:07 PM Yehuda Sadeh-Weinraub <yehuda@xxxxxxxxxx>
>> wrote:
>>>
>>> According to the log here, it says that the location constraint it got
>>> is "cn", can you take a look at a tcpdump, see if that's actually
>>> what's passed in?
>>>
>>> On Mon, Feb 26, 2018 at 12:02 PM, David Turner <drakonstein@xxxxxxxxx>
>>> wrote:
>>> > I run with `debug rgw = 10` and was able to find these lines at the end
>>> > of a
>>> > request to create the bucket.
>>> >
>>> > Successfully creating a bucket with `bucket_location = US` looks like
>>> > [1]this. Failing to create a bucket has "ERROR: S3 error: 400
>>> > (InvalidLocationConstraint): The specified location-constraint is not
>>> > valid"
>>> > on the CLI and [2]this (excerpt from the end of the request) in the rgw
>>> > log
>>> > (debug level 10). "create bucket location constraint" was not found in
>>> > the
>>> > log for successfully creating the bucket.
>>> >
>>> >
>>> > [1]
>>> > 2018-02-26 19:52:36.419251 7f4bc9bc8700 10 cache put:
>>> >
>>> > name=local-atl.rgw.data.root++.bucket.meta.testerton:bef43c26-daf3-47ef-a3a5-e1167e3f88ac.39099765.1
>>> > info.flags=0x17
>>> > 2018-02-26 19:52:36.419262 7f4bc9bc8700 10 adding
>>> >
>>> > local-atl.rgw.data.root++.bucket.meta.testerton:bef43c26-daf3-47ef-a3a5-e1167e3f88ac.39099765.1
>>> > to cache LRU end
>>> > 2018-02-26 19:52:36.419266 7f4bc9bc8700 10 updating xattr:
>>> > name=user.rgw.acl
>>> > bl.length()=141
>>> > 2018-02-26 19:52:36.423863 7f4bc9bc8700 10 RGWWatcher::handle_notify()
>>> > notify_id 344855809097728 cookie 139963970426880 notifier 39099765
>>> > bl.length()=361
>>> > 2018-02-26 19:52:36.423875 7f4bc9bc8700 10 cache put:
>>> > name=local-atl.rgw.data.root++testerton info.flags=0x17
>>> > 2018-02-26 19:52:36.423882 7f4bc9bc8700 10 adding
>>> > local-atl.rgw.data.root++testerton to cache LRU end
>>> >
>>> > [2]
>>> > 2018-02-26 19:43:37.340289 7f466bbca700 2 req 428078:0.004204:s3:PUT
>>> > /testraint/:create_bucket:executing
>>> > 2018-02-26 19:43:37.340366 7f466bbca700 5 NOTICE: call to
>>> > do_aws4_auth_completion
>>> > 2018-02-26 19:43:37.340472 7f466bbca700 10 v4 auth ok --
>>> > do_aws4_auth_completion
>>> > 2018-02-26 19:43:37.340715 7f466bbca700 10 create bucket location
>>> > constraint: cn
>>> > 2018-02-26 19:43:37.340766 7f466bbca700 0 location constraint (cn)
>>> > can't be
>>> > found.
>>> > 2018-02-26 19:43:37.340794 7f466bbca700 2 req 428078:0.004701:s3:PUT
>>> > /testraint/:create_bucket:completing
>>> > 2018-02-26 19:43:37.341782 7f466bbca700 2 req 428078:0.005689:s3:PUT
>>> > /testraint/:create_bucket:op status=-2208
>>> > 2018-02-26 19:43:37.341792 7f466bbca700 2 req 428078:0.005707:s3:PUT
>>> > /testraint/:create_bucket:http status=400
>>> >
>>> > On Mon, Feb 26, 2018 at 2:36 PM Yehuda Sadeh-Weinraub
>>> > <yehuda@xxxxxxxxxx>
>>> > wrote:
>>> >>
>>> >> I'm not sure if the rgw logs (debug rgw = 20) specify explicitly why a
>>> >> bucket creation is rejected in these cases, but it might be worth
>>> >> trying to look at these. If not, then a tcpdump of the specific failed
>>> >> request might shed some light (would be interesting to look at the
>>> >> generated LocationConstraint).
>>> >>
>>> >> Yehuda
>>> >>
>>> >> On Mon, Feb 26, 2018 at 11:29 AM, David Turner <drakonstein@xxxxxxxxx>
>>> >> wrote:
>>> >> > Our problem only appeared to be present in bucket creation.
>>> >> > Listing,
>>> >> > putting, etc objects in a bucket work just fine regardless of the
>>> >> > bucket_location setting. I ran this test on a few different realms
>>> >> > to
>>> >> > see
>>> >> > what would happen and only 1 of them had a problem. There isn't an
>>> >> > obvious
>>> >> > thing that steps out about it. The 2 local realms do not have
>>> >> > multi-site,
>>> >> > the internal realm has multi-site and the operations were performed
>>> >> > on
>>> >> > the
>>> >> > primary zone for the zonegroup.
>>> >> >
>>> >> > Worked with non 'US' bucket_location for s3cmd to create bucket:
>>> >> > realm=internal
>>> >> > zonegroup=internal-ga
>>> >> > zone=internal-atl
>>> >> >
>>> >> > Failed with non 'US' bucket_location for s3cmd to create bucket:
>>> >> > realm=local-atl
>>> >> > zonegroup=local-atl
>>> >> > zone=local-atl
>>> >> >
>>> >> > Worked with non 'US' bucket_location for s3cmd to create bucket:
>>> >> > realm=local
>>> >> > zonegroup=local
>>> >> > zone=local
>>> >> >
>>> >> > I was thinking it might have to do with all of the parts being named
>>> >> > the
>>> >> > same, but I made sure to do the last test to confirm. Interestingly
>>> >> > it's
>>> >> > only bucket creation that has a problem and it's fine as long as I
>>> >> > put
>>> >> > 'US'
>>> >> > as the bucket_location.
>>> >> >
>>> >> > On Mon, Feb 19, 2018 at 6:48 PM F21 <f21.groups@xxxxxxxxx> wrote:
>>> >> >>
>>> >> >> I am using the official ceph/daemon docker image. It starts RGW and
>>> >> >> creates a zonegroup and zone with their names set to an empty
>>> >> >> string:
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> https://github.com/ceph/ceph-container/blob/master/ceph-releases/luminous/ubuntu/16.04/daemon/start_rgw.sh#L36:54
>>> >> >>
>>> >> >> $RGW_ZONEGROUP and $RGW_ZONE are both empty strings by default:
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> https://github.com/ceph/ceph-container/blob/master/ceph-releases/luminous/ubuntu/16.04/daemon/variables_entrypoint.sh#L46
>>> >> >>
>>> >> >> Here's what I get when I query RGW:
>>> >> >>
>>> >> >> $ radosgw-admin zonegroup list
>>> >> >> {
>>> >> >> "default_info": "",
>>> >> >> "zonegroups": [
>>> >> >> "default"
>>> >> >> ]
>>> >> >> }
>>> >> >>
>>> >> >> $ radosgw-admin zone list
>>> >> >> {
>>> >> >> "default_info": "",
>>> >> >> "zones": [
>>> >> >> "default"
>>> >> >> ]
>>> >> >> }
>>> >> >>
>>> >> >> On 20/02/2018 10:33 AM, Yehuda Sadeh-Weinraub wrote:
>>> >> >> > What is the name of your zonegroup?
>>> >> >> >
>>> >> >> > On Mon, Feb 19, 2018 at 3:29 PM, F21 <f21.groups@xxxxxxxxx>
>>> >> >> > wrote:
>>> >> >> >> I've done some debugging and the LocationConstraint is not being
>>> >> >> >> set
>>> >> >> >> by
>>> >> >> >> the
>>> >> >> >> SDK by default.
>>> >> >> >>
>>> >> >> >> I do, however, need to set the region on the client to us-east-1
>>> >> >> >> for
>>> >> >> >> it
>>> >> >> >> to
>>> >> >> >> work. Anything else will return an InvalidLocationConstraint
>>> >> >> >> error.
>>> >> >> >>
>>> >> >> >> Francis
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> On 20/02/2018 8:40 AM, Yehuda Sadeh-Weinraub wrote:
>>> >> >> >>> Sounds like the go sdk adds a location constraint to requests
>>> >> >> >>> that
>>> >> >> >>> don't go to us-east-1. RGW itself is definitely isn't tied to
>>> >> >> >>> us-east-1, and does not know anything about it (unless you
>>> >> >> >>> happen
>>> >> >> >>> to
>>> >> >> >>> have a zonegroup named us-east-1). Maybe there's a way to
>>> >> >> >>> configure
>>> >> >> >>> the sdk to avoid doing that?
>>> >> >> >>>
>>> >> >> >>> Yehuda
>>> >> >> >>>
>>> >> >> >>> On Sun, Feb 18, 2018 at 1:54 PM, F21 <f21.groups@xxxxxxxxx>
>>> >> >> >>> wrote:
>>> >> >> >>>> I am using the AWS Go SDK v2
>>> >> >> >>>> (https://github.com/aws/aws-sdk-go-v2)
>>> >> >> >>>> to
>>> >> >> >>>> talk
>>> >> >> >>>> to my RGW instance using the s3 interface. I am running ceph
>>> >> >> >>>> in
>>> >> >> >>>> docker
>>> >> >> >>>> using
>>> >> >> >>>> the ceph/daemon docker images in demo mode. The RGW is started
>>> >> >> >>>> with a
>>> >> >> >>>> zonegroup and zone with their names set to an empty string by
>>> >> >> >>>> the
>>> >> >> >>>> scripts
>>> >> >> >>>> in
>>> >> >> >>>> the image.
>>> >> >> >>>>
>>> >> >> >>>> I have ForcePathStyle for the client set to true, because I
>>> >> >> >>>> want
>>> >> >> >>>> to
>>> >> >> >>>> access
>>> >> >> >>>> all my buckets using the path: myrgw.instance:8080/somebucket.
>>> >> >> >>>>
>>> >> >> >>>> I noticed that if I set the region for the client to anything
>>> >> >> >>>> other
>>> >> >> >>>> than
>>> >> >> >>>> us-east-1, I get this error when creating a bucket:
>>> >> >> >>>> InvalidLocationConstraint: The specified location-constraint
>>> >> >> >>>> is
>>> >> >> >>>> not
>>> >> >> >>>> valid.
>>> >> >> >>>>
>>> >> >> >>>> If I set the region in the client to something made up, such
>>> >> >> >>>> as
>>> >> >> >>>> "ceph"
>>> >> >> >>>> and
>>> >> >> >>>> the LocationConstraint to "ceph", I still get the same error.
>>> >> >> >>>>
>>> >> >> >>>> The only way to get my buckets to create successfully is to
>>> >> >> >>>> set
>>> >> >> >>>> the
>>> >> >> >>>> client's
>>> >> >> >>>> region to us-east-1. I have grepped the ceph code base and
>>> >> >> >>>> cannot
>>> >> >> >>>> find
>>> >> >> >>>> any
>>> >> >> >>>> references to us-east-1. In addition, I looked at the AWS docs
>>> >> >> >>>> for
>>> >> >> >>>> calculating v4 signatures and us-east-1 is the default region
>>> >> >> >>>> but
>>> >> >> >>>> I
>>> >> >> >>>> can
>>> >> >> >>>> see
>>> >> >> >>>> that the region string is used in the calculation (i.e. the
>>> >> >> >>>> region
>>> >> >> >>>> is
>>> >> >> >>>> not
>>> >> >> >>>> ignored when calculating the signature if it is set to
>>> >> >> >>>> us-east-1).
>>> >> >> >>>>
>>> >> >> >>>> Why do my buckets create successfully if I set the region in
>>> >> >> >>>> my s3
>>> >> >> >>>> client
>>> >> >> >>>> to
>>> >> >> >>>> us-east-1, but not otherwise? If I do not want to use
>>> >> >> >>>> us-east-1 as
>>> >> >> >>>> my
>>> >> >> >>>> default region, for example, if I want us-west-1 as my default
>>> >> >> >>>> region,
>>> >> >> >>>> what
>>> >> >> >>>> should I be configuring in ceph?
>>> >> >> >>>>
>>> >> >> >>>> Thanks,
>>> >> >> >>>>
>>> >> >> >>>> Francis
>>> >> >> >>>>
>>> >> >> >>>> _______________________________________________
>>> >> >> >>>> ceph-users mailing list
>>> >> >> >>>> ceph-users@xxxxxxxxxxxxxx
>>> >> >> >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>> >> >> >>
>>> >> >> >>
>>> >> >>
>>> >> >> _______________________________________________
>>> >> >> ceph-users mailing list
>>> >> >> ceph-users@xxxxxxxxxxxxxx
>>> >> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
