Hi, everyone.
Recently, we did a test as follows:
We enabled cache tier and added a cache pool "vms_back_cache" on top
of the base pool "vms_back". we first created an object, and then
created a snap in the base pool and writing to that object again,
which would make the object be promoted into the cache pool. At this
time, we used "ceph-objectstore-tool" to dump the object, and the
result is as follows:
{
"id": {
"oid": "test.obj.6",
"key": "",
"snapid": -2,
"hash": 750422257,
"max": 0,
"pool": 11,
"namespace": "",
"max": 0
},
"info": {
"oid": {
"oid": "test.obj.6",
"key": "",
"snapid": -2,
"hash": 750422257,
"max": 0,
"pool": 11,
"namespace": ""
},
"version": "5010'5",
"prior_version": "4991'3",
"last_reqid": "client.175338.0:1",
"user_version": 5,
"size": 4194303,
"mtime": "2017-08-23 15:09:03.459892",
"local_mtime": "2017-08-23 15:09:03.461111",
"lost": 0,
"flags": 4,
"snaps": [],
"truncate_seq": 0,
"truncate_size": 0,
"data_digest": 4294967295,
"omap_digest": 4294967295,
"watchers": {}
},
"stat": {
"size": 4194303,
"blksize": 4096,
"blocks": 8200,
"nlink": 1
},
"SnapSet": {
"snap_context": {
"seq": 13,
"snaps": [
13
]
},
"head_exists": 1,
"clones": [
{
"snap": 13,
"size": 4194303,
"overlap": "[0~100,115~4194188]"
}
]
}
}
Then we did cache-flush and cache-evict to flush that object down to
the base pool, and, again, used "ceph-objectstore-tool" to dump the
object in the base pool:
{
"id": {
"oid": "test.obj.6",
"key": "",
"snapid": -2,
"hash": 750422257,
"max": 0,
"pool": 10,
"namespace": "",
"max": 0
},
"info": {
"oid": {
"oid": "test.obj.6",
"key": "",
"snapid": -2,
"hash": 750422257,
"max": 0,
"pool": 10,
"namespace": ""
},
"version": "5015'4",
"prior_version": "4991'2",
"last_reqid": "osd.34.5013:1",
"user_version": 5,
"size": 4194303,
"mtime": "2017-08-23 15:09:03.459892",
"local_mtime": "2017-08-23 15:10:48.122138",
"lost": 0,
"flags": 52,
"snaps": [],
"truncate_seq": 0,
"truncate_size": 0,
"data_digest": 163942140,
"omap_digest": 4294967295,
"watchers": {}
},
"stat": {
"size": 4194303,
"blksize": 4096,
"blocks": 8200,
"nlink": 1
},
"SnapSet": {
"snap_context": {
"seq": 13,
"snaps": [
13
]
},
"head_exists": 1,
"clones": [
{
"snap": 13,
"size": 4194303,
"overlap": "[]"
}
]
}
}
As is shown, the "overlap" field is empty.
In the osd log, we found the following records:
2017-08-23 12:46:36.083014 7f675c704700 20 osd.0 pg_epoch: 19 pg[3.3(
v 15'2 (0'0,15'2] local-les=15 n=2 ec=14 les/c/f 15/15/0 14/14/14)
[0,2,1] r=0 lpr=14 crt=0'0 lcod 15'1 mlcod 15'1 active+clean] got
attrs
2017-08-23 12:46:36.083021 7f675c704700 15
filestore(/home/xuxuehan/github-xxh-fork/ceph/src/dev/osd0) read
3.3_head/#3:dd4db749:test-rados-api-xxh02v.ops.corp.qihoo.net-10886-3::foo:head#
0~8
2017-08-23 12:46:36.083398 7f675c704700 10
filestore(/home/xuxuehan/github-xxh-fork/ceph/src/dev/osd0)
FileStore::read
3.3_head/#3:dd4db749:test-rados-api-xxh02v.ops.corp.qihoo.net-10886-3::foo:head#
0~8/8
2017-08-23 12:46:36.083414 7f675c704700 20 osd.0 pg_epoch: 19 pg[3.3(
v 15'2 (0'0,15'2] local-les=15 n=2 ec=14 les/c/f 15/15/0 14/14/14)
[0,2,1] r=0 lpr=14 crt=0'0 lcod 15'1 mlcod 15'1 active+clean] got
data
2017-08-23 12:46:36.083444 7f675c704700 20 osd.0 pg_epoch: 19 pg[3.3(
v 15'2 (0'0,15'2] local-les=15 n=2 ec=14 les/c/f 15/15/0 14/14/14)
[0,2,1] r=0 lpr=14 crt=0'0 lcod 15'1 mlcod 15'1 active+clean]
cursor.is_complete=0 0 attrs 8 bytes 0 omap header bytes 0 omap data
bytes in 0 keys 0 reqids
2017-08-23 12:46:36.083457 7f675c704700 10 osd.0 pg_epoch: 19 pg[3.3(
v 15'2 (0'0,15'2] local-les=15 n=2 ec=14 les/c/f 15/15/0 14/14/14)
[0,2,1] r=0 lpr=14 crt=0'0 lcod 15'1 mlcod 15'1 active+clean]
dropping ondisk_read_lock
2017-08-23 12:46:36.083467 7f675c704700 15 osd.0 pg_epoch: 19 pg[3.3(
v 15'2 (0'0,15'2] local-les=15 n=2 ec=14 les/c/f 15/15/0 14/14/14)
[0,2,1] r=0 lpr=14 crt=0'0 lcod 15'1 mlcod 15'1 active+clean]
do_osd_op_effects osd.0 con 0x7f67874f0d00
2017-08-23 12:46:36.083478 7f675c704700 15 osd.0 pg_epoch: 19 pg[3.3(
v 15'2 (0'0,15'2] local-les=15 n=2 ec=14 les/c/f 15/15/0 14/14/14)
[0,2,1] r=0 lpr=14 crt=0'0 lcod 15'1 mlcod 15'1 active+clean]
log_op_stats osd_op(osd.0.6:2 3.92edb2bb
test-rados-api-xxh02v.ops.corp
It seems that, when doing "copy-get", no extensive attributes are
copied. We believe that it's the following code that led to this
result:
int ReplicatedPG::getattrs_maybe_cache(ObjectContextRef obc,
map<string, bufferlist> *out,
bool user_only) {
int r = 0;
if (pool.info.require_rollback()) {
if (out)
*out = obc->attr_cache;
} else {
r = pgbackend->objects_get_attrs(obc->obs.oi.soid, out);
}
if (out && user_only) {
map<string, bufferlist> tmp;
for (map<string, bufferlist>::iterator i = out->begin();
i != out->end(); ++i) {
if (i->first.size() > 1 && i->first[0] == '_')
tmp[i->first.substr(1, i->first.size())].claim(i->second);
}
tmp.swap(*out);
}
return r;
}
It seems that when "user_only" is true, extensive attributes without a
'_' as the starting character in its name would be filtered out. Is it
supposed to be doing things in this way?
And we found that there are only two places in the source code that
invoked ReplicatedPG::getattrs_maybe_cache, in both of which
"user_only" is true. Why add this parameter?
By the way, we also found that these codes are added in commit
78d9c0072bfde30917aea4820a811d7fc9f10522, but we don't understand the
purpose of it.
Thank you:-)
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
