Re: How data is stored on EC?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

Thank you very much the explanation.
For me is objectstore, is it different?

Istvan Szabo
Senior Infrastructure Engineer
---------------------------------------------------
Agoda Services Co., Ltd.
e: istvan.szabo@xxxxxxxxx<mailto:istvan.szabo@xxxxxxxxx>
---------------------------------------------------

On 2021. Dec 4., at 3:53, 胡 玮文 <huww98@xxxxxxxxxxx> wrote:


Email received from the internet. If in doubt, don't click any link nor open any attachment !
________________________________
Hi Istvan,

Upper-level applications may chunk data into smaller objects, typically 4M each, e.g. CephFS [1], RBD [2]. However, the max object size enforced by OSD is configurable by osd_max_object_size, which defaults to 128M.

So, as of my understanding, your 100MB file will typically be chunked into 25x4MB objects, then each object will be chunked to 4x1MB + 2x1MB and stored on OSDs.

[1]: https://docs.ceph.com/en/latest/cephfs/file-layouts/#layout-fields
[2]: https://docs.ceph.com/en/latest/man/8/rbd/#cmdoption-rbd-object-size

Weiwen Hu

发件人: Szabo, Istvan (Agoda)<mailto:Istvan.Szabo@xxxxxxxxx>
发送时间: 2021年12月4日 5:09
收件人: Ceph Users<mailto:ceph-users@xxxxxxx>
主题:  How data is stored on EC?

Hi,

Just curious my math is correct or not.

There is 4:2EC
Ssd min alloc size 4kb, due to this the minimum file size that worth to store has to
be bigger than 16KB. Correct?

So if it is 10KB object, it will still be stored on 16kb +2 additional 4KB chunk so 24KB?

If the file is 100MB, it will be chunked to 4x25MB + 2x25MB and the chunks will be stored how? 25MBs or small 4KBs?

Istvan Szabo
Senior Infrastructure Engineer
---------------------------------------------------
Agoda Services Co., Ltd.
e: istvan.szabo@xxxxxxxxx<mailto:istvan.szabo@xxxxxxxxx>
---------------------------------------------------
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux