Thanks a lot for the insightful comments. huxiaoyu@xxxxxxxxxxxx From: Janne Johansson Date: 2021-03-13 11:36 To: huxiaoyu@xxxxxxxxxxxx CC: ceph-users Subject: Re: How big an OSD disk could be? Den fre 12 mars 2021 kl 18:10 skrev huxiaoyu@xxxxxxxxxxxx <huxiaoyu@xxxxxxxxxxxx>: > Dear cephers, > Just wonder how big an OSD disk could be? Currently the biggest HDD has a capacity of 18TB or 20TB. It is suitable for an OSD still? > Is there a limitation of the capacity of a single OSD? Can it be 30TB , 50TB or 100TB a single OSD? What could be the potential limit? I think your patience will be the limit. If you stick 8x or 10x 20TB drives in a box, it will take a very long time to fill them to say, 50%. That would be some 80-100TB to move over the network and a lot of IO spread over only 8-10 drives before it settles, which would be a long time on spinning drives. Unless the objects are huge, 100TB of files/objects would mean many many millions of writes just to fill the OSD and its list of contents. For spinners, it would mean many many millions of head movements just to write it all out, hopefully never encountering a random bad block. After you have filled that up, if such a host crashes or needs maintenance, another 80-100TB will need recreating from the other huge drives. I don't think there are specific limitations on the size itself, but as the single drive becomes larger and larger, just adding a new host or a drive will mean the cluster is rebalancing for days or weeks if not more. At some point you would end up having the cluster almost never in HEALTH_OK state because of normal replacements, expansions and other surprises, which in turn could cause secondary problems with mon DBs and things like that. -- May the most significant bit of your life be positive. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |