Well, just from a quick skim, libradosstriper.h has a function
rados_striper_set_object_layout_object_size(rados_striper_t striper, unsigned int object_size)
and libradosstriper.hpp has one in RadosStriper
set_object_layout_object_size(unsigned int object_size);
So I imagine you specify it with those the same way you've set the stripe unit and counts.
On Sat, Oct 7, 2017 at 12:38 PM Alexander Kushnirenko <kushnirenko@xxxxxxxxx> wrote:
Hi, Gregory!It turns out that this error is internal CEPH feature. I wrote standalone program to create 132M object in striper mode. It works only for 4M stripe. If you set stripe_unit = 2M it still creates 4M stripe_unit. Anything bigger than 4M causes crash here:__u32 object_size = layout->object_size;__u32 su = layout->stripe_unit;__u32 stripe_count = layout->stripe_count;assert(object_size >= su); <------------I'm curious where it gets layout->object_size for object that is just been created.As I understod striper mode was created by CERN guys. In there document they recommend 8M stripe_unit. But it does not work in luminous.Created I/O context.Connected to pool backup with rados_striper_createStripe unit OK 8388608Stripe count OK 1/build/ceph-12.2.0/src/osdc/Striper.cc: In function 'static void Striper::file_to_extents(CephContext*, const char*, const file_layout_t*, uint64_t, uint64_t, uint64_t, std::map<object_t, std::vector<ObjectExtent> >&, uint64_t)' thread 7f13bd5c1e00 time 2017-10-07 21:44:58.654778/build/ceph-12.2.0/src/osdc/Striper.cc: 64: FAILED assert(object_size >= su)ceph version 12.2.0 (32ce2a3ae5239ee33d6150705cdb24d43bab910c) luminous (rc)1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7f13b3f3b332]2: (Striper::file_to_extents(CephContext*, char const*, file_layout_t const*, unsigned long, unsigned long, unsigned long, std::map<object_t, std::vector<ObjectExtent, std::allocator<ObjectExtent> >, std::less<object_t>, std::allocator<std::pair<object_t const, std::vector<ObjectExtent, std::allocator<ObjectExtent> > > > >&, unsigned long)+0x1e1e) [0x7f13bce235ee]3: (Striper::file_to_extents(CephContext*, char const*, file_layout_t const*, unsigned long, unsigned long, unsigned long, std::vector<ObjectExtent, std::allocator<ObjectExtent> >&, unsigned long)+0x51) [0x7f13bce23691]4: (libradosstriper::RadosStriperImpl::internal_aio_write(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::intrusive_ptr<libradosstriper::MultiAioCompletionImpl>, ceph::buffer::list const&, unsigned long, unsigned long, ceph_file_layout const&)+0x224) [0x7f13bcda4184]5: (libradosstriper::RadosStriperImpl::write_in_open_object(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ceph_file_layout const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ceph::buffer::list const&, unsigned long, unsigned long)+0x13c) [0x7f13bcda476c]6: (libradosstriper::RadosStriperImpl::write(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ceph::buffer::list const&, unsigned long, unsigned long)+0xd5) [0x7f13bcda4bd5]7: (rados_striper_write()+0xdb) [0x7f13bcd9ba0b]8: (()+0x10fb) [0x55dd87b410fb]9: (__libc_start_main()+0xf1) [0x7f13bc9d72b1]10: (()+0xbca) [0x55dd87b40bca]On Fri, Sep 29, 2017 at 11:46 PM, Gregory Farnum <gfarnum@xxxxxxxxxx> wrote:I haven't used the striper, but it appears to make you specify sizes, stripe units, and stripe counts. I would expect you need to make sure that the size is an integer multiple of the stripe unit. And it probably defaults to a 4MB object if you don't specify one?On Fri, Sep 29, 2017 at 2:09 AM Alexander Kushnirenko <kushnirenko@xxxxxxxxx> wrote:_______________________________________________Hi,I'm trying to use CEPH-12.2.0 as storage for with Bareos-16.2.4 backup with libradosstriper1 support.Libradosstriber was suggested on this list to solve the problem, that current CEPH-12 discourages users from using object with very big size (>128MB). Bareos treat Rados Object as Volume and in CEPH-10 it created objects with very big size (10G and more). CEPH-10 allowed such behaviour, put recovery indeed take very long time. So stripping objects seems to be the right thing to do.Bareos supports libradosstriper and the code seems to work. But for some reason it run only with stripe_unit=4194304, which seems to be typical value for RadosGW for example. I tried several other values for stripe_unit, but the code exit with error.Is there a particular reason why only 4M size works? Can one use some CLI to test different stripe sizes?Basic flow of creating object in Bareos is the following:rados_ioctx_create(m_cluster, m_rados_poolname, &m_ctx);rados_striper_create(m_ctx, &m_striper);rados_striper_set_object_layout_stripe_unit(m_striper, m_stripe_unit);rados_striper_set_object_layout_stripe_count(m_striper, m_stripe_count);.....status = rados_striper_write(m_striper, m_virtual_filename, buffer, count, offset);Alexander
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
