On Fri, 22 Feb 2013, S?bastien Han wrote: > Hi all, > > I finally got a core dump. > > I did it with a kill -SEGV on the OSD process. > > https://www.dropbox.com/s/ahv6hm0ipnak5rf/core-ceph-osd-11-0-0-20100-1361539008 > > Hope we will get something out of it :-). AHA! We have a theory. The pg log isnt trimmed during scrub (because teh old scrub code required that), but the new (deep) scrub can take a very long time, which means the pg log will eat ram in the meantime.. especially under high iops. Can you try wip-osd-log-trim (which is bobtail + a simple patch) and see if that seems to work? Note that that patch shouldn't be run in a mixed argonaut+bobtail cluster, since it isn't properly checking if the scrub is class or chunky/deep. Thanks! sage > -- > Regards, > S?bastien Han. > > > On Fri, Jan 11, 2013 at 7:13 PM, Gregory Farnum <greg@xxxxxxxxxxx> wrote: > > On Fri, Jan 11, 2013 at 6:57 AM, S?bastien Han <han.sebastien@xxxxxxxxx> wrote: > >>> Is osd.1 using the heap profiler as well? Keep in mind that active use > >>> of the memory profiler will itself cause memory usage to increase ? > >>> this sounds a bit like that to me since it's staying stable at a large > >>> but finite portion of total memory. > >> > >> Well, the memory consumption was already high before the profiler was > >> started. So yes with the memory profiler enable an OSD might consume > >> more memory but this doesn't cause the memory leaks. > > > > My concern is that maybe you saw a leak but when you restarted with > > the memory profiling you lost whatever conditions caused it. > > > >> Any ideas? Nothing to say about my scrumbing theory? > > I like it, but Sam indicates that without some heap dumps which > > capture the actual leak then scrub is too large to effectively code > > review for leaks. :( > > -Greg > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
