Those are probably large directories; each omap key is a file/subdir in the directory. Normally the mds fragments dirs across several objects, so you shouldn't have a huge number of omap entries in any one single object. Do you have some custom mds config in this area? -- dan On Wed, Aug 25, 2021 at 2:01 PM Frank Schilder <frans@xxxxxx> wrote: > > Hi Dan, > > thanks for looking at this. Here are the lines from health detail and ceph.log: > > [root@gnosis ~]# ceph health detail > HEALTH_WARN 4 large omap objects > LARGE_OMAP_OBJECTS 4 large omap objects > 4 large objects found in pool 'con-fs2-meta1' > Search the cluster log for 'Large omap object found' for more details. > > The search gives: > > 2021-08-25 11:17:00.675474 osd.21 osd.21 192.168.32.77:6846/12302 651 : cluster [WRN] Large omap object found. Object: 12:373fb013:::1000eec35f5.01000000:head PG: 12.c80dfcec (12.6c) Key count: 216000 Size (bytes): 101520000 > 2021-08-25 11:17:06.866726 osd.37 osd.37 192.168.32.77:6850/12306 644 : cluster [WRN] Large omap object found. Object: 12:05982a7e:::1000d7fd167.02800000:head PG: 12.7e5419a0 (12.20) Key count: 2293816 Size (bytes): 1078093520 > 2021-08-25 11:17:11.152671 osd.37 osd.37 192.168.32.77:6850/12306 645 : cluster [WRN] Large omap object found. Object: 12:05da1450:::1000e118c0a.00000000:head PG: 12.a285ba0 (12.20) Key count: 220612 Size (bytes): 103687640 > 2021-08-25 11:17:36.603664 osd.36 osd.36 192.168.32.75:6848/11882 1243 : cluster [WRN] Large omap object found. Object: 12:0b298d19:::1000eec35f7.04e00000:head PG: 12.98b194d0 (12.50) Key count: 657212 Size (bytes): 308889640 > > They are all in the fs meta-data pool. > > Best regards, > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Dan van der Ster <dan@xxxxxxxxxxxxxx> > Sent: 25 August 2021 13:57:44 > To: Frank Schilder > Cc: ceph-users > Subject: Re: LARGE_OMAP_OBJECTS: any proper action possible? > > Hi Frank, > > Which objects are large? (You should see this in ceph.log when the > large obj was detected). > > -- dan > > On Wed, Aug 25, 2021 at 12:27 PM Frank Schilder <frans@xxxxxx> wrote: > > > > Hi all, > > > > I have the notorious "LARGE_OMAP_OBJECTS: 4 large omap objects" warning and am again wondering if there is any proper action one can take except "wait it out and deep-scrub (numerous ceph-users threads)" or "ignore (https://docs.ceph.com/en/latest/rados/operations/health-checks/#large-omap-objects)". Only for RGWs is a proper action described, but mine come from MDSes. Is there any way to ask an MDS to clean up or split the objects? > > > > The disks with the meta-data pool can easily deal with objects of this size. My question is more along the lines: If I can't do anything anyway, why the warning? If there is a warning, I would assume that one can do something proper to prevent large omap objects from being born by an MDS. What is it? > > > > Best regards, > > ================= > > Frank Schilder > > AIT Risø Campus > > Bygning 109, rum S14 > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |