Hi there.
I have an issue with cephfs and multiple datapools inside. I have like
SIX datapools inside the cephfs, I control where files are stored using
xattrs in the directories.
The "root" directory only contains directories with "xattrs" requesting
new objects to be stored in different pools. So good so far. The problem
is that "root" datapool has a ghost file per file inside the cephfs,
even when the object is actually stored in a different datapool. Taking
no space at all, but counting against the number of objects in the cephfs.
The files are empty (0 bytes) but they have xattrs saying in what pool
the object is actually stored.
Should this data be stored in the metadata pool?. By comparison, my
metadata pool is 244MB in size but it basically uses the same size when
I had no objects and now, with 1.3 million objects: ~250 MB.
cephfsROOT_data 70 0 0 354G
1332929
cephfsROOT_metadata 71 244M 0.07 354G
1625
black_1 72 944G 52.58 851G
241736
black_2 73 944G 52.59 851G
241744
black_3 74 953G 52.82 851G
243990
black_4 75 934G 52.33 851G
239243
black_5 76 944G 52.59 851G
241814
black_6 77 531G 38.44 851G 136081
Black_* are associated to the cephfs via "ceph fs add_data_pool XXX".
For each object created inside a "black_*", a ghost empty object is
created in "cephfsROOT_data".
That huge number of files is causing other issues, like the warning "1
pools have many more objects per pg than average".
About that warning. I change the value of "mon_pg_warn_max_object_skew"
and The warning is still there. Asking the monitors about it, they show
the new value, but asking from client, it still shows "10":
root@jcea:/var/run/ceph# ceph --admin-daemon ceph-client.jcea.asok --id
jcea config show|grep skew
"mon_timecheck_skew_interval": "30",
"mon_pg_warn_max_object_skew": "10",
I don't know how to proceed.
--
Jesús Cea Avión _/_/ _/_/_/ _/_/_/
jcea@xxxxxxx - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/
Twitter: @jcea _/_/ _/_/ _/_/_/_/_/
jabber / xmpp:jcea@xxxxxxxxxx _/_/ _/_/ _/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
