On 2019-10-24 09:46, Janne Johansson wrote:
(Slightly abbreviated) Den tors 24 okt. 2019 kl 09:24 skrev Frank Schilder <frans@xxxxxx <mailto:frans@xxxxxx>>: What I learned are the following: 1) Avoid this work-around too few hosts for EC rule at all cost. 2) Do not use EC 2+1. It does not offer anything interesting for production. Use 4+2 (or 8+2, 8+3 if you have the hosts). 3) If you have no perspective of getting at least 7 servers in the long run (4+2=6 for EC profile, +1 for fail-over automatic rebuild), do not go for EC. 4) Before you start thinking about replicating to a second site, you should have a primary site running solid first. This is collected from my experience. I would do things different now and maybe it helps you with deciding how to proceed. Its basically about what resources can you expect in the foreseeable future and what compromises are you willing to make with regards to sleep and sanity. Amen to all of those points. We did similar-but-not-same mistakes on an EC cluster here. You are going to produce more tears than I/O if you make these mis-designs mentioned above. We could add: 5) Never buy SMR drives, pretend they don't even exist. If a similar technology appears tomorrow for cheap SSD/NVME, skip it.
Amen from my side, too. Luckily, we only made a small fraction of these mistakes (running 4+2 on 6 servers and wondering about funny effects when taking one server offline, while we still were testing the setup, before we finally descided to ask for a 7th server), but this can in parts be extrapolated. Concerning SMR, I learnt that SMR-awareness is on Ceph's roadmap (for host-managed SMR drives). Once that is available, host-managed SMR drives should be a well-working and cheap solution especially for backup / WORM workloads. But as of for now, even disk vendors will tell you to avoid SMR for datacenter setups (unless you have a storage system aware of it and host-managed drives). Cheers, Oliver
-- May the most significant bit of your life be positive. _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
