|
Hey, I recall reading about this somewhere but I can't find it in the docs or list archive and confirmation from a dev or someone who knows for sure would be nice. What I recall is that bluestore has a max 4GB file size limit based on the design of bluestore not the osd_max_object_size setting. The bluestore source seems to suggest that by setting the OBJECT_MAX_SIZE to a 32bit max, giving an error if osd_max_object_size is > OBJECT_MAX_SIZE, and not writing the data if offset+length >= OBJECT_MAX_SIZE. So it seems like the in osd file size int can't exceed 32 bits which is 4GB, like FAT32. Am I correct or maybe I'm reading all this wrong..? If bluestore has a hard 4GB object limit using radosstriper to break up an object would work, but does using an EC pool that breaks up the object to shards smaller than OBJECT_MAX_SIZE have the same effect as radosstriper to get around a 4GB limit? We use rados directly and would like to move to bluestore but we have some large objects <= 13G that may need attention if this 4GB limit does exist and an ec pool doesn't get around it. https://github.com/ceph/ceph/blob/master/src/os/bluestore/BlueStore.cc#L88 #define OBJECT_MAX_SIZE 0xffffffff // 32 bits https://github.com/ceph/ceph/blob/master/src/os/bluestore/BlueStore.cc#L4395 // sanity check(s)
auto osd_max_object_size =
cct->_conf.get_val<Option::size_t>("osd_max_object_size");
if (osd_max_object_size >= (size_t)OBJECT_MAX_SIZE) {
derr << __func__ << " osd_max_object_size >= 0x" << std::hex << OBJECT_MAX_SIZE
<< "; BlueStore has hard limit of 0x" << OBJECT_MAX_SIZE << "." << std::dec << dendl;
return -EINVAL;
}
https://github.com/ceph/ceph/blob/master/src/os/bluestore/BlueStore.cc#L12331
if (offset + length >= OBJECT_MAX_SIZE) {
r = -E2BIG;
} else {
_assign_nid(txc, o);
r = _do_write(txc, c, o, offset, length, bl, fadvise_flags);
txc->write_onode(o);
}
Thanks!
Kevin
-- Kevin Hrpcek NASA SNPP Atmosphere SIPS Space Science & Engineering Center University of Wisconsin-Madison |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
