Hi Florian,
Thanks for the help. I did further testing and narrowed it down to objects that have been uploaded when the bucket has versioning enabled.
Objects created before that are not affected: all metadata operations are still possible.
Here is a simple way to reproduce this: http://paste.openstack.org/show/736713/
And here is the snippet to easily turn on/off S3 versioning on a given bucket: https://gist.github.com/Miouge1/b8ae19b71411655154e74e609b61f24e
Cheers,
Maxime
On Fri, 30 Nov 2018 at 22:28 Florian Haas <florian@xxxxxxxxxxxxxx> wrote:
On 28/11/2018 19:06, Maxime Guyot wrote:
> Hi Florian,
>
> You assumed correctly, the "test" container (private) was created with
> the "openstack container create test", then I am using the S3 API to
> enable/disable object versioning on it.
> I use the following Python snippet to enable/disable S3 bucket versioning:
>
> import boto, boto.s3, boto.s3.connection
> conn = conn = boto.connect_s3(aws_access_key_id='***',
> aws_secret_access_key='***', host='***', port=8080,
> calling_format=boto.s3.connection.OrdinaryCallingFormat())
> bucket = conn.get_bucket('test')
> bucket.configure_versioning(True) # Or False to disable S3 bucket versioning
> bucket.get_versioning_status()
Thanks for making this so easy to reproduce! I must confess upfront that
I've found myself unable to reproduce your problem, but I've retraced
your steps and maybe you'll find this useful to develop a hypothesis as
to what's happening in your case.
$ openstack object show -f shell foo bar
account="AUTH_5ed51981f4a8468292bf2c578806ebf7"
container="foo"
content_length="12"
content_type="text/plain"
last_modified="Thu, 22 Nov 2018 15:02:57 GMT"
object="bar"
properties="S3cmd-Attrs='atime:1542629253/ctime:1542629253/gid:1000/gname:florian/md5:6f5902ac237024bdd0c176cb93063dc4/mode:33204/mtime:1542629253/uid:1000/uname:florian'"
See the properties that are set there? These are obviously not
properties ever set through the Swift API, but instead they were set
when I uploaded this object into the corresponding bucket, using the S3 API.
I can double check that property with boto:
>>> foo = conn.get_bucket('foo')
>>> bar = bucket.get_key('bar')
>>> bar.metadata
{'s3cmd-attrs':
u'atime:1542629253/ctime:1542629253/gid:1000/gname:florian/md5:6f5902ac237024bdd0c176cb93063dc4/mode:33204/mtime:1542629253/uid:1000/uname:florian'}
Now I enable versioning:
>>> foo.configure_versioning(True)
True
>>> foo.get_versioning_status()
{'Versioning': 'Enabled'}
Check if the metadata is still there:
>>> bar.metadata
{'s3cmd-attrs':
u'atime:1542629253/ctime:1542629253/gid:1000/gname:florian/md5:6f5902ac237024bdd0c176cb93063dc4/mode:33204/mtime:1542629253/uid:1000/uname:florian'}
Refetch object to be sure:
>>> bar = bucket.get_key('bar')
{'s3cmd-attrs':
u'atime:1542629253/ctime:1542629253/gid:1000/gname:florian/md5:6f5902ac237024bdd0c176cb93063dc4/mode:33204/mtime:1542629253/uid:1000/uname:florian'}
Disable versioning again:
>>> foo.configure_versioning(False)
True
>>> foo.get_versioning_status()
{'Versioning': 'Suspended'}
Now add a property using the Swift API:
$ openstack object set --property spam=eggs foo bar
And read it back:
$ openstack object show -f shell foo bar
account="AUTH_5ed51981f4a8468292bf2c578806ebf7"
container="foo"
content_length="12"
content_type="text/plain"
last_modified="Wed, 28 Nov 2018 19:52:48 GMT"
object="bar"
properties="Spam='eggs'"
Notice that not only has the property been set, it has *overwritten* the
S3 properties that were set before. I am not sure if this is meant to be
this way, i.e. if native Swift acts this way too, but it appears to be
how radosgw does it.
However, now that have the "spam" property set, I go ahead and re-enable
versioning:
>>> foo.configure_versioning(True)
True
>>> foo.get_versioning_status()
{'Versioning': 'Enabled'}
And then I re-query my object:
$ openstack object show -f shell foo bar
account="AUTH_5ed51981f4a8468292bf2c578806ebf7"
container="foo"
content_length="12"
content_type="text/plain"
last_modified="Thu, 29 Nov 2018 11:47:41 GMT"
object="bar"
properties="Spam='eggs'"
So as you can see, in my case the "spam" property, when defined with the
Swift API, did get preserved even across enabling versioning.
I can also use s3cmd to check that the Swift meta header looks like an
x-amz-meta header when querying the same object via S3:
$ s3cmd info s3://foo/bar
s3://foo/bar (object):
File size: 12
Last mod: Thu, 29 Nov 2018 11:47:41 GMT
MIME type: text/plain
Storage: STANDARD
MD5 sum: 6f5902ac237024bdd0c176cb93063dc4
SSE: none
Policy: none
CORS: none
x-amz-meta-spam: eggs
So could it be that there is *something* you're doing that just
overwrites your metadata?
>> Semi-related: I've seen some interesting things when mucking around with
>> a single container/bucket while switching APIs, when it comes to
>> container properties and metadata. For example, if you set a public read
>> ACL on an S3 bucket, the the corresponding Swift container is also
>> publicly readable but its read ACL looks empty (i.e. private) when you
>> ask via the Swift API.
>
> This can definitely become a problem if Swift API says "private" but
> data is actually publicly available.
> Since the doc says "S3 and Swift APIs share a common namespace, so you
> may write data with one API and retrieve it with the other", it might be
> useful to document this kind of limitations somewhere.
I agree, but as an aside I believe that in your particular case you're
highlighting one of the inherent limitations of the dual-API capability.
S3 bucket versioning works, as far as I understand, quite differently
from versioned objects in Swift, so making radosgw behave correctly for
versioning in both APIs — as soon as it's enabled in only one — strikes
me as really, really difficult. Similar to what Yehuda mentioned about
ACL differences.
Not sure if this helps?
Cheers,
Florian
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
