Sorry for the late reply - vacation :/ Seems there is some confusion here: the AWS GO SDK sets the format as ISO 8601 basic - https://github.com/aws/aws-sdk-go/blob/main/aws/signer/v4/v4.go#L84. So I don't know where the divergence is happening :(. FWIW the aws go sdk works fine against S3 and fails for ceph. A very straightforward repro can be had by running https://github.com/wasabi-tech/s3-benchmark/blob/master/s3-benchmark.go against ceph - Is it possible to see why this is? (FWIW, I have created an AWS support case for this internally - I THINK s3 processes both formats, I am not sure). Thanks. Subu On Sun, Dec 12, 2021 at 12:58 PM Robin H. Johnson <robbat2@xxxxxxxxxx> wrote: > On Tue, Dec 07, 2021 at 12:39:12PM -0500, Casey Bodley wrote: > > hi Subu, > > > > On Tue, Dec 7, 2021 at 12:10 PM Subu Sankara Subramanian > > <subu.zsked@xxxxxxxxx> wrote: > > > > > > Folks, > > > > > > Is there a bug in ceph RGW date parsing? > > > https://github.com/ceph/ceph/blob/master/src/rgw/rgw_auth_s3.cc#L223 > - this > > > line parses the date in x-amz-date as RFC 2616. BUT the format > specified by > > > Amazon S3 is ISO 8601 basic - YYYYMMDDTHHMMSSZ ( > > > > https://docs.aws.amazon.com/ses/latest/APIReference-V2/CommonParameters.html > ). > > > > this is a link to the Amazon Simple Email Service v2 API, it doesn't > > refer to aws v2 signatures. v2 signatures for s3 are documented at > > > https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html#RESTAuthenticationTimeStamp > , > > which states that "The value of the x-amz-date header must be in one > > of the RFC 2616 formats (http://www.ietf.org/rfc/rfc2616.txt)." > Yes, there is a bug, but it's not the one you're thinking of. > > The S3 specification says one thing (RFC2616 dates [3]), and AWS's S3 > implementation does something slightly different: > It accepts non-GMT time offsets, and correctly converts them to GMT/UTC. > > RGW on the other hand has a subtle failure: > 1. RFC2616 section 3.3.1 [1] > Specifies that HTTP timestamps must be in GMT. > 2. Not all clients follow this, and may send it with other timezones! > 3. RGW uses strptime(s, "%a, %d %b %Y %H:%M:%S %z", t) to try and parse > the time. > 4. glibc strptime accepts the %z specifier, but discards the value! [2] > 5. If the server is set to UTC, and the client sends a non-GMT date, > it will be parsed as if it's in GMT, and thus potentially be offset > by some time (usually a multiple of hours due to the geo-political > nature of timezones). > 6. If making a SigV2 request, the timezone used 4th line of the string to > sign (timestamp) is *not* GMT/0000 offset, e.g. "Sun, 12 Dec 2021 > 12:52:48 -0800", RGW will respond with RequestTimeTooSkewed response. > > Partial demo for this bug at [4]. > > [1] https://datatracker.ietf.org/doc/html/rfc2616#section-3.3.1 > [2] https://man7.org/linux/man-pages/man3/strptime.3.html#NOTES > [3] > https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html#RESTAuthenticationTimeStamp > [4] https://gist.github.com/robbat2/930974e21dcdf2b31944f954c46dd904 > > Credit to one of my co-workers, Max Kuznetsov, for finding the initial > conditions that lead this bug discovery. > > -- > Robin Hugh Johnson > Gentoo Linux: Dev, Infra Lead, Foundation Treasurer > E-Mail : robbat2@xxxxxxxxxx > GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 > GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |