10+1 is a bad idea for obvious reasons (not enough coding chunks, you will be offline if even one server is offline).
The real problem is that your 20kb files will be split up into 2kb chunks and the metadata overhead and bluestore min alloc size will eat up your disk space.
Paul
2018-07-30 13:44 GMT+02:00 Anton Aleksandrov <anton@xxxxxxxxxxxxxx>:
Hello community,
I am building first cluster for project, that hosts millions of small (from 20kb) and big (up to 10mb) files. Right now we are moving from local 16tb raid storage to cluster of 12 small machines. We are planning to have 11 OSD nodes, use erasure coding pool (10+1) and one host for MDS.
On my local tests I see, that available space decrease unproportionally to the amount of data copied into cluster. With clean cluster I have, for example 100gb available space, but after copying 40gb in - size decreases for about 5-10%. Is that normal?
Is there any term, that would specify cluster's minimal object size?
I also have question if having so many small files (current number is about 50'000'000 files at least) - could have negative impact and where would be our bottleneck? As we don't have money for SSD, we will have WAL/DB on separate simple HDD.
Also - would that help to put Metadata pool on separate disks, away from Data pool drives for CephFS?
Regards,
Anton.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Paul Emmerich
Looking for help with your Ceph cluster? Contact us at https://croit.io
croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90
Looking for help with your Ceph cluster? Contact us at https://croit.io
croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
