Hi,
If we have 12 SATA disks, each 4TB as storage pool.2016-05-10 16:40 GMT+08:00 Yoann Moulin <yoann.moulin@xxxxxxx>:
Hello,
I'd like some advices about the setup of a new ceph cluster. Here the use case :
RadowGW (S3 and maybe swift for hadoop/spark) will be the main usage. Most of
the access will be in read only mode. Write access will only be done by the
admin to update the datasets.
We might use rbd some time to sync data as temp storage (when POSIX is needed)
but performance will not be an issue here. We might use cephfs in the futur if
that can replace a filesystem on rdb.
We gonna start with 16 nodes (up to 24). The configuration of each node is :
CPU : 2 x Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz (12c/48t)
Memory : 128GB
OS Storage : 2 x SSD 240GB Intel S3500 DC (raid 1)
Journal or cache Storage : 2 x SSD 400GB Intel S3300 DC (no Raid)
OSD Disk : 10 x HGST ultrastar-7k6000 6TB
Public Network : 1 x 10Gb/s
Private Network : 1 x 10Gb/s
OS : Ubuntu 16.04
Ceph version : Jewel
The question is : journal or cache tier (read only) on the SD 400GB Intel S3300 DC ?
Each disk is able to write sequentially at 220MB/s. SSDs can write at ~500MB/s.
if we set 5 journals on each SSDs, SSD will still be the bottleneck (1GB/s vs
2GB/s). If we set the journal on OSDs, we can expect a good throughput in read
on the disk (in case of data not in the cache) and write shouldn't be so bad
too, even if we have random read on the OSD during the write ?
SSDs as cache tier seem to be a better usage than only 5 journal on each ? Is
that correct ?
We gonna use an EC pool for big files (jerasure 8+2 I think) and a replicated
pool for small files.
If I check on http://ceph.com/pgcalc/, in this use case
replicated pool: pg_num = 8192 for 160 OSDs but 16384 for 240 OSDs
Ec pool : pg_num = 4096
and pgp_num = pg_num
Should I set the pg_num to 8192 or 16384 ? what is the impact on the cluster if
we set the pg_num to 16384 at the beginning ? 16384 is high, isn't it ?
Thanks for your help
--
Yoann Moulin
EPFL IC-IT
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
