Hi Bryan,
This is a bug related to storage classes. Compression does take effect
for requests that specify the storage class via the s3
x-amz-storage-class or swift x-object-storage-class header. But when
this header is absent, we default to the STANDARD storage class without
consulting its compression type. The fix is pending backport to nautilus
in https://tracker.ceph.com/issues/41981.
On 11/6/19 5:54 PM, Bryan Stillwell wrote:
Today I tried enabling RGW compression on a Nautilus 14.2.4 test cluster and found it wasn't doing any compression at all. I figure I must have missed something in the docs, but I haven't been able to find out what that is and could use some help.
This is the command I used to enable zlib-based compression:
# radosgw-admin zone placement modify --rgw-zone=default --placement-id=default-placement --compression=zlib
I then restarted the radosgw process to activate the change (there's only 1 RGW in this test cluster):
# systemctl restart ceph-radosgw@radosgw.$(hostname -s)
I verified it was enabled correctly with:
# radosgw-admin zone get | jq -r '.placement_pools'
[
{
"key": "default-placement",
"val": {
"index_pool": "default.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "default.rgw.buckets.data",
"compression_type": "zlib"
}
},
"data_extra_pool": "default.rgw.buckets.non-ec",
"index_type": 0
}
}
]
Before starting the test I had nothing in the default.rgw.buckets.data pool:
# ceph df | grep default.rgw.buckets.data
default.rgw.buckets.data 16 0 B 0 0 B 0 230 TiB
I then tried uploading a 1GiB file consisting of all 0s from /dev/zero with both S3 (boto) and Swift (swiftclient) and each time they used 1GiB of data on the cluster:
# ceph df -f json | jq -r '.' | grep -A9 default.rgw.buckets.data
"name": "default.rgw.buckets.data",
"id": 16,
"stats": {
"stored": 1073741824,
"objects": 256,
"kb_used": 1048576,
"bytes_used": 1073741824,
"percent_used": 1.4138463484414387e-06,
"max_avail": 253148744646656
}
The same thing was reported by bucket stats:
# radosgw-admin bucket stats --bucket=bs-test | jq -r '.usage'
{
"rgw.main": {
"size": 1073741824,
"size_actual": 1073741824,
"size_utilized": 1073741824,
"size_kb": 1048576,
"size_kb_actual": 1048576,
"size_kb_utilized": 1048576,
"num_objects": 1
}
}
What am I missing?
Thanks,
Bryan
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
