Sorry for the chain mail - here is the response from AWS: So it is indeed a bug with RGW - how do we update this? The format of this header will depend on the signature version that is used when signing and authenticating requests with AWS. The provided piece of documentation that uses the RFC 2616 format is only applicable for AWS Signature Version 2 [1], which is an older version of how AWS authenticates requests and has been primarily replaced by AWS Signature Version 4. When using AWS SigV2, the x-amz-date header should be in the RFC 2616 format. When using AWS Signature Version 4, the x-amz-date header as well as the timestamp in the String to Sign adheres to the ISO 8601 format [2]. Since all of our SDKs, including the Go SDK, utilize AWS SigV4 by default, the x-amz-date is set to the ISO 8601 format. SigV4 is the recommended way of signing requests to S3, and so I would recommend using SigV4 in your clients, which will require that the x-amz-date value is formatted according to ISO 8601. For more information on the overall SigV4 signing process, as well as signature calculation examples, please see [3-4]. There is also this note in the documentation you referenced https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html#RESTAuthenticationTimeStamp that confirms that is for SigV2: Note This topic explains authenticating requests using Signature Version 2. Amazon S3 now supports the latest Signature Version 4. This latest signature version is supported in all regions and any new regions after January 30, 2014 will support only Signature Version 4. For more information, go to Authenticating Requests (AWS Signature Version 4) <https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html> in the *Amazon Simple Storage Service API Reference*. Thanks. Subu On Tue, Jan 4, 2022 at 1:01 PM Subu Sankara Subramanian < subu.zsked@xxxxxxxxx> wrote: > Sorry for the late reply - vacation :/ > > Seems there is some confusion here: the AWS GO SDK sets the format as ISO > 8601 basic - > https://github.com/aws/aws-sdk-go/blob/main/aws/signer/v4/v4.go#L84. So I > don't know where the divergence is happening :(. FWIW the aws go sdk works > fine against S3 and fails for ceph. A very straightforward repro can be had > by running > https://github.com/wasabi-tech/s3-benchmark/blob/master/s3-benchmark.go > against ceph - Is it possible to see why this is? (FWIW, I have created an > AWS support case for this internally - I THINK s3 processes both formats, I > am not sure). > > Thanks. Subu > > On Sun, Dec 12, 2021 at 12:58 PM Robin H. Johnson <robbat2@xxxxxxxxxx> > wrote: > >> On Tue, Dec 07, 2021 at 12:39:12PM -0500, Casey Bodley wrote: >> > hi Subu, >> > >> > On Tue, Dec 7, 2021 at 12:10 PM Subu Sankara Subramanian >> > <subu.zsked@xxxxxxxxx> wrote: >> > > >> > > Folks, >> > > >> > > Is there a bug in ceph RGW date parsing? >> > > https://github.com/ceph/ceph/blob/master/src/rgw/rgw_auth_s3.cc#L223 >> - this >> > > line parses the date in x-amz-date as RFC 2616. BUT the format >> specified by >> > > Amazon S3 is ISO 8601 basic - YYYYMMDDTHHMMSSZ ( >> > > >> https://docs.aws.amazon.com/ses/latest/APIReference-V2/CommonParameters.html >> ). >> > >> > this is a link to the Amazon Simple Email Service v2 API, it doesn't >> > refer to aws v2 signatures. v2 signatures for s3 are documented at >> > >> https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html#RESTAuthenticationTimeStamp >> , >> > which states that "The value of the x-amz-date header must be in one >> > of the RFC 2616 formats (http://www.ietf.org/rfc/rfc2616.txt)." >> Yes, there is a bug, but it's not the one you're thinking of. >> >> The S3 specification says one thing (RFC2616 dates [3]), and AWS's S3 >> implementation does something slightly different: >> It accepts non-GMT time offsets, and correctly converts them to GMT/UTC. >> >> RGW on the other hand has a subtle failure: >> 1. RFC2616 section 3.3.1 [1] >> Specifies that HTTP timestamps must be in GMT. >> 2. Not all clients follow this, and may send it with other timezones! >> 3. RGW uses strptime(s, "%a, %d %b %Y %H:%M:%S %z", t) to try and parse >> the time. >> 4. glibc strptime accepts the %z specifier, but discards the value! [2] >> 5. If the server is set to UTC, and the client sends a non-GMT date, >> it will be parsed as if it's in GMT, and thus potentially be offset >> by some time (usually a multiple of hours due to the geo-political >> nature of timezones). >> 6. If making a SigV2 request, the timezone used 4th line of the string to >> sign (timestamp) is *not* GMT/0000 offset, e.g. "Sun, 12 Dec 2021 >> 12:52:48 -0800", RGW will respond with RequestTimeTooSkewed response. >> >> Partial demo for this bug at [4]. >> >> [1] https://datatracker.ietf.org/doc/html/rfc2616#section-3.3.1 >> [2] https://man7.org/linux/man-pages/man3/strptime.3.html#NOTES >> [3] >> https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html#RESTAuthenticationTimeStamp >> [4] https://gist.github.com/robbat2/930974e21dcdf2b31944f954c46dd904 >> >> Credit to one of my co-workers, Max Kuznetsov, for finding the initial >> conditions that lead this bug discovery. >> >> -- >> Robin Hugh Johnson >> Gentoo Linux: Dev, Infra Lead, Foundation Treasurer >> E-Mail : robbat2@xxxxxxxxxx >> GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 >> GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |