That's so cool! We've been struggling to find ways to expose s3
extensions, and the client-side validation of clients like boto has made
this really difficult.
I'm curious how granular we can make these extensions. For example, if
we wanted to expose our ?allow-unordered extension to the ListObjects
op, would we have to replace the entire
botocore/data/s3/2006-03-01/service-2.json file? Or could we provide a
new json file that only specifies a modified ListObjectsRequest
structure? If we can make them that granular, it would limit conflicts
with upstream boto and be much easier to maintain. It could also make it
easy for our users to opt-in to certain extensions.
Exploring this for s3tests/pubsub in teuthology sounds like a good first
step!
On 9/16/19 11:16 AM, Yuval Lifshitz wrote:
Hi All,
Recently I added a support for bucket notifications, as part of the
feature the standard (s3) way for filtering notification was extended
with new fields that are not supported by AWS. This poses 2 issues:
- how to test it
- how users should use it
Crafting these messages "by hand" (e.g. by using curl) is not ideal
for both usecases.
However, according to this [1], there is an easy way to extend the
standard by changing a json file and then placing it under
~/.aws/models/s3/...
Or, placing it anywhere else, and then setting its path first in
the AWS_DATA_PATH search path (I tried the 1st options and it worked!).
I guess that this is not the only case where we want to extend/modify
what ASW/boto3 supports.
Would suggest that we add our own versions of the relevant files
(e.g.[2]) into the ceph repo (not sure where?).
This could be used for:
- teuthology infra can take these files and use them, so that the
clients could test our extension natively with boto3
- we can add documentation around these files, so they are available
to users (maybe deliver them in the installation packages)
Of course, we will have to track changes to the original file, and
merge from time top time...
Yuval
[1] https://github.com/boto/botocore/blob/develop/botocore/loaders.py#L33
[2]
https://github.com/boto/botocore/blob/develop/botocore/data/s3/2006-03-01/service-2.json
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
