Re: cosd multi-second stalls cause "wrongly marked me down"

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, 2011-03-11 at 12:09 -0700, Gregory Farnum wrote:
> On Friday, March 11, 2011 at 10:51 AM, Jim Schutt wrote:
> > I'm also confused. Here's the code I ran:
> > 
> >  void put() {
> >  //generic_dout(0) << "RefCountedObject::put " << this << " " << nref.read() << " -> " << (nref.read() - 1) << dendl;
> >  if (nref.dec() == 0) {
> >  utime_t s = g_clock.now();
> >  delete this;
> >  utime_t e = g_clock.now();
> >  if (double(e - s) > 0.5) {
> >  generic_dout(1) << "RefCountedObject::put delete " << this << " took " << double(e - s) << " secs!" << dendl;
> >  }
> >  }
> >  }
> > 
> > I added those double casts, because I had a similar problem with my throttler->put
> > test: without the casts, it was firing but the reported delay was less than
> > 0.25 sec. Adding the casts stopped that - I haven't yet checked into why.
> > 
> > Still checking for what I'm missing...
> > 
> > -- Jim
> 
> Heh -- it turns out that operator double() is a little bit broken -- it divides by the wrong constant! Pushed the (very simple) fix in 3fb4fd8612b7a05f7d89cfd0b48f765c79830f95 to the stable branch.
> This would be putting everything into the wrong order of magnitude. Half second delete issues are a lot more believable (to me at least!) than 5 second ones, and I wouldn't be surprised if they were the fault of the default memory allocator combined with the throttler getting bogged down. (Apparently even tcmalloc will do a better job of memory management if memory is freed from the same thread it's allocated in, and almost none of our memory use is polite in that way.)
> -Greg
> 
Re-running my tests....

-- Jim

> 
> 
> 


--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux