Thanks Greg, That makes sense. Can you also confirm that latency values are always in seconds? I haven't seen any documentation for it and want to be sure before I say it is one way or the other. Dan -----Original Message----- From: Gregory Farnum [mailto:greg@xxxxxxxxxxx] Sent: Friday, May 09, 2014 1:42 PM To: Dan Ryder (daryder) Cc: Haomai Wang; ceph-users at ceph.com Subject: Re: Low latency values The recovery_state "latencies" are all about how long your PGs are in various states of recovery; they're not per-operation latencies. 3 days still seems awfully long, but if you had a lot of data that needed to get recovered and were throttling it tightly enough that could happen. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Fri, May 9, 2014 at 9:44 AM, Dan Ryder (daryder) <daryder at cisco.com> wrote: > Ok, that makes sense for the OSD IO latency values. But I'm confused about the recoverystate_perf latency values. > > For example: > "started_latency": { "avgcount": 296, > "sum": 86047405.517876000}, > > "primary_latency": { "avgcount": 240, > "sum": 53489945.222530000}, > > If these values are in seconds, I don't think the latency should be anywhere near that high. 290700 seconds > 3 days. > > 86047405.517876000 / 296 = 290700.694317 seconds? > > 53489945.222530000 / 240 = 222874.771761 seconds? > > Thanks, > Dan > > -----Original Message----- > From: Haomai Wang [mailto:haomaiwang at gmail.com] > Sent: Friday, May 09, 2014 12:29 PM > To: Dan Ryder (daryder) > Cc: ceph-users at ceph.com > Subject: Re: Low latency values > > yes > > On Sat, May 10, 2014 at 12:19 AM, Dan Ryder (daryder) <daryder at cisco.com> wrote: >> Thanks Haomai, >> >> So are all latency values calculated in seconds? >> >> Dan >> >> -----Original Message----- >> From: Haomai Wang [mailto:haomaiwang at gmail.com] >> Sent: Friday, May 09, 2014 11:20 AM >> To: Dan Ryder (daryder) >> Cc: ceph-users at ceph.com >> Subject: Re: Low latency values >> >> 178/184229=0.00097 s = 0.97ms >> >> On Fri, May 9, 2014 at 10:49 PM, Dan Ryder (daryder) <daryder at cisco.com> wrote: >>> Hi, >>> >>> >>> >>> I?m seeing really low latency values, to the extent that they don?t >>> seem realistic. >>> >>> >>> >>> Snippet from the latest perf dump for this OSD: >>> >>> >>> >>> "op_r_latency": { "avgcount": 184229, >>> >>> "sum": 178.077710000}, >>> >>> >>> >>> Long run avg = 178.07771/184229 = 0.00097 ms? Is it correct that >>> latency values have milliseconds as units? >>> >>> If so, this number seems too small. >>> >>> >>> >>> Any help would be appreciated. >>> >>> >>> >>> Thanks, >>> >>> Dan >>> >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users at lists.ceph.com >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >> >> >> >> -- >> Best Regards, >> >> Wheat > > > > -- > Best Regards, > > Wheat > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |