Re: RGW Swift metadata dropped when S3 bucket versioning enabled

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Nov 28, 2018 at 10:07 AM Maxime Guyot <maxime@xxxxxxxxxxx> wrote:
>
> Hi Florian,
>
> You assumed correctly, the "test" container (private) was created with the "openstack container create test", then I am using the S3 API to enable/disable object versioning on it.
> I use the following Python snippet to enable/disable S3 bucket versioning:
>
> import boto, boto.s3, boto.s3.connection
> conn = conn = boto.connect_s3(aws_access_key_id='***', aws_secret_access_key='***', host='***', port=8080, calling_format=boto.s3.connection.OrdinaryCallingFormat())
> bucket = conn.get_bucket('test')
> bucket.configure_versioning(True) # Or False to disable S3 bucket versioning
> bucket.get_versioning_status()
>
> > Semi-related: I've seen some interesting things when mucking around with
> > a single container/bucket while switching APIs, when it comes to
> > container properties and metadata. For example, if you set a public read
> > ACL on an S3 bucket, the the corresponding Swift container is also
> > publicly readable but its read ACL looks empty (i.e. private) when you
> > ask via the Swift API.
>
> This can definitely become a problem if Swift API says "private" but data is actually publicly available.
> Since the doc says "S3 and Swift APIs share a common namespace, so you may write data with one API and retrieve it with the other", it might be useful to document this kind of limitations somewhere.

Note that swift acls and S3 acls don't quite map perfectly to each
other. When S3 public read acl on a bucket doesn't mean that data is
accessible, but rather that bucket can be listed. In swift the
container acls are about the objects inside. Not sure that there is an
equivalent swift acl that would only deal with ability to list objects
in the container.

Yehuda
>
> Cheers,
> / Maxime
>
> On Wed, 28 Nov 2018 at 17:58 Florian Haas <florian@xxxxxxxxxxxxxx> wrote:
>>
>> On 27/11/2018 20:28, Maxime Guyot wrote:
>> > Hi,
>> >
>> > I'm running into an issue with the RadosGW Swift API when the S3 bucket
>> > versioning is enabled. It looks like it silently drops any metadata sent
>> > with the "X-Object-Meta-foo" header (see example below).
>> > This is observed on a Luminous 12.2.8 cluster. Is that a normal thing?
>> > Am I misconfiguring something here?
>> >
>> >
>> > With S3 bucket versioning OFF:
>> > $ openstack object set --property foo=bar test test.dat
>> > $ os object show test test.dat
>> > +----------------+----------------------------------+
>> > | Field          | Value                            |
>> > +----------------+----------------------------------+
>> > | account        | v1                               |
>> > | container      | test                             |
>> > | content-length | 507904                           |
>> > | content-type   | binary/octet-stream              |
>> > | etag           | 03e8a398f343ade4e1e1d7c81a66e400 |
>> > | last-modified  | Tue, 27 Nov 2018 13:53:54 GMT    |
>> > | object         | test.dat                         |
>> > | properties     | Foo='bar'                        |  <= Metadata is here
>> > +----------------+----------------------------------+
>> >
>> > With S3 bucket versioning ON:
>>
>> Can you elaborate on what exactly you're doing here to enable S3 bucket
>> versioning? Do I assume correctly that you are creating the "test"
>> container using the swift or openstack client, then sending a
>> VersioningConfiguration request against the "test" bucket, as explained
>> in
>> https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html#how-to-enable-disable-versioning-intro?
>>
>> > $ openstack object set --property foo=bar test test2.dat
>> > $ openstack object show test test2.dat
>> > +----------------+----------------------------------+
>> > | Field          | Value                            |
>> > +----------------+----------------------------------+
>> > | account        | v1                               |
>> > | container      | test                             |
>> > | content-length | 507904                           |
>> > | content-type   | binary/octet-stream              |
>> > | etag           | 03e8a398f343ade4e1e1d7c81a66e400 |
>> > | last-modified  | Tue, 27 Nov 2018 13:56:50 GMT    |
>> > | object         | test2.dat                        | <= Metadata is absent
>> > +----------------+----------------------------------+
>>
>> Semi-related: I've seen some interesting things when mucking around with
>> a single container/bucket while switching APIs, when it comes to
>> container properties and metadata. For example, if you set a public read
>> ACL on an S3 bucket, the the corresponding Swift container is also
>> publicly readable but its read ACL looks empty (i.e. private) when you
>> ask via the Swift API.
>>
>> Cheers,
>> Florian
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux