On Tue, Jul 5, 2016 at 10:45 AM, Wido den Hollander <wido@xxxxxxxx> wrote: > >> Op 5 juli 2016 om 19:27 schreef Gregory Farnum <gfarnum@xxxxxxxxxx>: >> >> >> On Tue, Jul 5, 2016 at 2:10 AM, Wido den Hollander <wido@xxxxxxxx> wrote: >> > >> >> Op 5 juli 2016 om 10:56 schreef huang jun <hjwsm1989@xxxxxxxxx>: >> >> >> >> >> >> i see osd timed out many times. >> >> In SimpleMessenger mode, when sending msg, the Pipeconnection will >> >> hold a lock, which maybe hold by other threads, >> >> it's reported before: http://tracker.ceph.com/issues/9921 >> >> >> > >> > Thank you! It surely looks like the same symptoms we are seeing in this cluster. >> > >> > The bug has been marked as resolved, but are you sure it is? >> >> Pretty sure about that bug being done. >> >> The conntrack filling thing sounds vaguely familiar though. Is this >> the latest hammer? I think there were some leaks of messages while >> sending replies that might have blocked up incoming queues that got >> resolved later. > > Keep in mind, it's the conntrack filling up on the client which results in >50% packetloss on that client. > > The cluster is not firewalled and doesn't do any connection tracking. > > This is hammer 0.94.5, if this is fixed in .6 or .7, do you have an idea for which commit I should look? (Simple)Messenger related? If it is one of the op leaks, it'll be in the OSD OpTracker stuff to avoid keeping around message references for tracking purposes and unblocking the client Throttles. -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html