Ups... thanks for your efforts, Ben!
This could explain some bit's. Still I have lot's of question as it seems different S3 tools/clients behaive different. We need to stick on CyberDuck on Windows and s3cms and boto on Linux and many things are not the same with RadosGW :|
And more on my to-test-list ;)
Regards
Anton
Gesendet: Mittwoch, 12. April 2017 um 06:49 Uhr
Von: "Ben Hines" <bhines@xxxxxxxxx>
An: ceph.novice@xxxxxxxxxxxxxxxx
Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx>, "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx>
Betreff: Re: Kraken release and RGW --> "S3 bucket lifecycle API has been added. Note that currently it only supports object expiration."
Von: "Ben Hines" <bhines@xxxxxxxxx>
An: ceph.novice@xxxxxxxxxxxxxxxx
Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx>, "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx>
Betreff: Re: Kraken release and RGW --> "S3 bucket lifecycle API has been added. Note that currently it only supports object expiration."
After much banging on this and reading through the Ceph RGW source, i figured out Ceph RadosGW returns -13 ( EACCES - AcessDenied) if you dont pass in a 'Prefix' in your S3 lifecycle configuration setting. It also returns EACCES if the XML is invalid in any way, which is probably not the most correct / user friendly result.
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTlifecycle.html specifies 'Prefix' as Optional, so i'll put in a bug for this.
-Ben
On Mon, Apr 3, 2017 at 12:14 PM, Ben Hines <bhines@xxxxxxxxx> wrote:
Interesting.I'm wondering what the -13 return code for the op execution in my debug output is (can't find in the source..)I just tried out setting the lifecycle with cyberduck and got this error, which is probably the other bug with AWSv4 auth, http://tracker.ceph.com/issues/17076 Not sure if cyberduck can be forced to use V2.2017-04-03 12:07:15.093235 7f5617024700 10 op=20RGWPutLC_ObjStore_S32017-04-03 12:07:15.093248 7f5617024700 2 req 14:0.000438:s3:PUT /bentest/:put_lifecycle:authorizing.....2017-04-03 12:07:15.093637 7f5617024700 10 delaying v4 auth2017-04-03 12:07:15.093643 7f5617024700 10 ERROR: AWS4 completion for this operation NOT IMPLEMENTED2017-04-03 12:07:15.093652 7f5617024700 10 failed to authorize request2017-04-03 12:07:15.093658 7f5617024700 20 handler->ERRORHANDLER: err_no=-2201 new_err_no=-22012017-04-03 12:07:15.093844 7f5617024700 2 req 14:0.001034:s3:PUT /bentest/:put_lifecycle:op status=02017-04-03 12:07:15.093859 7f5617024700 2 req 14:0.001050:s3:PUT /bentest/:put_lifecycle:http status=5012017-04-03 12:07:15.093884 7f5617024700 1 ====== req done req=0x7f561701e340 op status=0 http_status=501 ======-BenOn Mon, Apr 3, 2017 at 7:16 AM, <ceph.novice@xxxxxxxxxxxxxxxx> wrote:... hmm, "modify" gives no error and may be the option to use, but I don't see anything related to an "expires" meta field[root s3cmd-master]# ./s3cmd --no-ssl --verbose modify s3://Test/INSTALL --expiry-days=365
INFO: Summary: 1 remote files to modify
modify: 's3://Test/INSTALL'
[root s3cmd-master]# ./s3cmd --no-ssl --verbose info s3://Test/INSTALL
s3://Test/INSTALL (object):
File size: 3123
Last mod: Mon, 03 Apr 2017 12:35:28 GMT
MIME type: text/plain
Storage: STANDARD
MD5 sum: 63834dbb20b32968505c4ebe768fc8c4
SSE: none
policy: <?xml version="1.0" encoding="UTF-8"?><ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>Test</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>INSTALL</Key><LastModified>2017-04-03T12:35:28.533Z</LastModified><ETag>"63834dbb20b32968505c4ebe768fc8c4"</ETag><Size>3123</Size><StorageClass>STANDARD</StorageClass><Owner><ID>666</ID><DisplayName>First User</DisplayName></Owner></Contents><Contents><Key>README.TXT</Key><LastModified>2017-03-31T22:36:38.380Z</LastModified><ETag>"708efc3b9184c8b112e36062804aca1e"</ETag><Size>88</Size><StorageClass>STANDARD</StorageClass><Owner><ID>666</ID><DisplayName>First User</DisplayName></Owner></Contents></ListBucketResult>
cors: none
ACL: First User: FULL_CONTROL
x-amz-meta-s3cmd-attrs: atime:1491218263/ctime:1490998096/gid:0/gname:root/md5:63834dbb20b32968505c4ebe768fc8c4/mode:33188/mtime:1488021707/uid:0/uname:rootGesendet: Montag, 03. April 2017 um 14:13 Uhr
Von: ceph.novice@xxxxxxxxxxxxxxxx
An: ceph-users <ceph-users@xxxxxxxxxxxxxx>
Betreff: Re: Kraken release and RGW --> "S3 bucket lifecycle API has been added. Note that currently it only supports object expiration."... additional strange but a bit different info related to the "permission denied"....
[root s3cmd-master]# ./s3cmd --no-ssl put INSTALL s3://Test/ --expiry-days=5
upload: 'INSTALL' -> 's3://Test/INSTALL' [1 of 1]
3123 of 3123 100% in 0s 225.09 kB/s done
[root s3cmd-master]# ./s3cmd info s3://Test/INSTALL
s3://Test/INSTALL (object):
File size: 3123
Last mod: Mon, 03 Apr 2017 12:01:47 GMT
MIME type: text/plain
Storage: STANDARD
MD5 sum: 63834dbb20b32968505c4ebe768fc8c4
SSE: none
policy: <?xml version="1.0" encoding="UTF-8"?><ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>Test</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>INSTALL</Key><LastModified>2017-04-03T12:01:47.745Z</LastModified><ETag>"63834dbb20b32968505c4ebe768fc8c4"</ETag><Size>3123</Size><StorageClass>STANDARD</StorageClass><Owner><ID>666</ID><DisplayName>First User</DisplayName></Owner></Contents><Contents><Key>README.TXT</Key><LastModified>2017-03-31T22:36:38.380Z</LastModified><ETag>"708efc3b9184c8b112e36062804aca1e"</ETag><Size>88</Size><StorageClass>STANDARD</StorageClass><Owner><ID>666</ID><DisplayName>First User</DisplayName></Owner></Contents></ListBucketResult>
cors: none
ACL: First User: FULL_CONTROL
x-amz-meta-s3cmd-attrs: atime:1491218263/ctime:1490998096/gid:0/gname:root/md5:63834dbb20b32968505c4ebe768fc8c4/mode:33188/mtime:1488021707/uid:0/uname:root
[root s3cmd-master]# ./s3cmd --no-ssl expire s3://Test/ --expiry-days=365
ERROR: Access to bucket 'Test' was denied
ERROR: S3 error: 403 (AccessDenied)
[root s3cmd-master]# ./s3cmd --no-ssl expire s3://Test/INSTALL --expiry-days=365
ERROR: Parameter problem: Expecting S3 URI with just the bucket name set instead of 's3://Test/INSTALL'
[root@mucsds26 s3cmd-master]# ./s3cmd --no-ssl expire s3://Test/ --expiry-days=365
ERROR: Access to bucket 'Test' was denied
ERROR: S3 error: 403 (AccessDenied)
[root s3cmd-master]# ./s3cmd --no-ssl la expire s3://Test
2017-04-03 12:01 3123 s3://Test/INSTALL
2017-03-31 22:36 88 s3://Test/README.TXT
################################################
Gesendet: Montag, 03. April 2017 um 12:31 Uhr
Von: ceph.novice@xxxxxxxxxxxxxxxx
An: "Ben Hines" <bhines@xxxxxxxxx>, ceph-users <ceph-users@xxxxxxxxxxxxxx>
Betreff: Re: Kraken release and RGW --> "S3 bucket lifecycle API has been added. Note that currently it only supports object expiration."
Hi Cephers...
I did set the "lifecycle" via Cyberduck.I do also get an error first, then suddenly Cyberduck refreshes the window aand the lifecycle is there.
I see the following when I check it via s3cmd (GitHub master version because the regular installed version doesn't offer the "getlifecycle" option):
[root s3cmd-master]# ./s3cmd getlifecycle s3://Test/README.txt
<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Rule>
<ID>Cyberduck-nVWEhQwE</ID>
<Prefix/>
<Status>Enabled</Status>
<Expiration>
<Days>1</Days>
</Expiration>
</Rule>
</LifecycleConfiguration>
Here is my S3 "user info":
[root ~]# radosgw-admin user info --uid=666
{
"user_id": "666",
"display_name": "First User",
"email": "a.b@xxxx",
"suspended": 0,
"max_buckets": 1000,
"auid": 0,
"subusers": [],
"keys": [
{
"user": "666",
"access_key": "abc ;)",
"secret_key": "abc def ;)"
}
],
"swift_keys": [],
"caps": [],
"op_mask": "read, write, delete",
"default_placement": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"temp_url_keys": [],
"type": "rgw"
}
If someone has a working example how to set lifecycle via the s3cmd, I can try it and send the outcome...
Gesendet: Montag, 03. April 2017 um 01:43 Uhr
Von: "Ben Hines" <bhines@xxxxxxxxx>
An: "Orit Wasserman" <owasserm@xxxxxxxxxx>
Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx>
Betreff: Re: Kraken release and RGW --> "S3 bucket lifecycle API has been added. Note that currently it only supports object expiration."
Hmm, Nope, not using tenants feature. The users/buckets were created on prior ceph versions, perhaps i'll try with a newly created user + bucket.
radosgw-admin user info --uid=foo
{
"user_id": "foo",
"display_name": "foo",
"email": "snip",
"suspended": 0,
"max_buckets": 1000,
"auid": 0,
"subusers": [
{
"id": "foo:swift",
"permissions": "full-control"
}
],
"keys": [
{
"user": "foo:swift",
"access_key": "xxx",
"secret_key": ""
},
{
"user": "foo",
"access_key": "xxx",
"secret_key": "xxxx"
}
],
"swift_keys": [],
"caps": [
{
"type": "buckets",
"perm": "*"
},
{
"type": "metadata",
"perm": "*"
},
{
"type": "usage",
"perm": "*"
},
{
"type": "users",
"perm": "*"
},
{
"type": "zone",
"perm": "*"
}
],
"op_mask": "read, write, delete",
"default_placement": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1024,
"max_size_kb": 0,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1024,
"max_size_kb": 0,
"max_objects": -1
},
"temp_url_keys": [],
"type": "none"
}
On Sun, Apr 2, 2017 at 5:54 AM, Orit Wasserman <owasserm@xxxxxxxxxx[mailto:owasserm@xxxxxxxxxx]> wrote:
I see : acct_user=foo, acct_name=foo,
Are you using radosgw with tenants?
If not it could be the problem
Orit
On Sat, Apr 1, 2017 at 7:43 AM, Ben Hines <bhines@xxxxxxxxx[mailto:bhines@xxxxxxxxx]> wrote:
I'm also trying to use lifecycles (via boto3) but i'm getting permission denied trying to create the lifecycle. I'm bucket owner with full_control and WRITE_ACP for good measure. Any ideas?
This is debug ms=20 debug radosgw=20
2017-03-31 21:28:18.382217 7f50d0010700 2 req 8:0.000693:s3:PUT /bentest:put_lifecycle:verifying op permissions
2017-03-31 21:28:18.382222 7f50d0010700 5 Searching permissions for identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=foo, acct_name=foo, subuser=, perm_mask=15, is_admin=) mask=56
2017-03-31 21:28:18.382232 7f50d0010700 5 Searching permissions for uid=foo
2017-03-31 21:28:18.382235 7f50d0010700 5 Found permission: 15
2017-03-31 21:28:18.382237 7f50d0010700 5 Searching permissions for group=1 mask=56
2017-03-31 21:28:18.382297 7f50d0010700 5 Found permission: 3
2017-03-31 21:28:18.382307 7f50d0010700 5 Searching permissions for group=2 mask=56
2017-03-31 21:28:18.382313 7f50d0010700 5 Permissions for group not found
2017-03-31 21:28:18.382318 7f50d0010700 5 Getting permissions identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=foo, acct_name=foo, subuser=, perm_mask=15, is_admin=) owner=foo perm=8
2017-03-31 21:28:18.382325 7f50d0010700 10 identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=foo, acct_name=foo, subuser=, perm_mask=15, is_admin=) requested perm (type)=8, policy perm=8, user_perm_mask=8, acl perm=8
2017-03-31 21:28:18.382330 7f50d0010700 2 req 8:0.000808:s3:PUT /bentest:put_lifecycle:verifying op params
2017-03-31 21:28:18.382334 7f50d0010700 2 req 8:0.000813:s3:PUT /bentest:put_lifecycle:pre-executing
2017-03-31 21:28:18.382339 7f50d0010700 2 req 8:0.000817:s3:PUT /bentest:put_lifecycle:executing
2017-03-31 21:28:18.382361 7f50d0010700 15 read len=183 data="" xmlns="http://s3.amazonaws.com/doc/2006-03-01/[http://s3.amazonaws.com/doc/2006-03-01/]"><Rule><Status>Enabled</Status><Expiration><Days>1</Days></Expiration><ID>0</ID></Rule></LifecycleConfiguration>
2017-03-31 21:28:18.382439 7f50d0010700 2 req 8:0.000917:s3:PUT /bentest:put_lifecycle:completing
2017-03-31 21:28:18.382594 7f50d0010700 2 req 8:0.001072:s3:PUT /bentest:put_lifecycle:op status=-13
2017-03-31 21:28:18.382620 7f50d0010700 2 req 8:0.001098:s3:PUT /bentest:put_lifecycle:http status=403
2017-03-31 21:28:18.382665 7f50d0010700 1 ====== req done req=0x7f50d000a340 op status=-13 http_status=403 ======
-Ben
On Tue, Mar 28, 2017 at 6:42 AM, Daniel Gryniewicz <dang@xxxxxxxxxx[mailto:dang@xxxxxxxxxx]> wrote:
On 03/27/2017 04:28 PM, ceph.novice@xxxxxxxxxxxxxxxx[mailto:ceph.novice@xxxxxxxxxxxxxxxx] wrote:Hi Cephers.
Couldn't find any special documentation about the "S3 object expiration"
so I assume it should work "AWS S3 like" (?!?) ... BUT ...
we have a test cluster based on 11.2.0 - Kraken and I set some object
expiration dates via CyberDuck and DragonDisk, but the objects are still
there, days after the applied date/time. Do I miss something?
Thanks & regards
It is intended to work like AWS S3, yes. Not every feature of AWS lifecycle is supported, (for example no moving between storage tiers), but deletion works, and is tested in teuthology runs.
Did you somehow turn it off? The config option rgw_enable_lc_threads controls it, but it defaults to "on". Also make sure rgw_lc_debug_interval is not set, and that rgw_lifecycle_work_time isn't set to some interval too small scan your objects...
Daniel
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx[mailto:ceph-users@xxxxxxxxxxxxxx]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com]
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx[mailto:ceph-users@xxxxxxxxxxxxxx]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com]
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com][http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com]]
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com]
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
