Re: Blocked requests during and after CephFS delete

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



[ Re-added the list since I don't have log files. ;) ]

On Mon, Dec 9, 2013 at 5:52 AM, Oliver Schulz <oschulz@xxxxxxxxxx> wrote:
> Hi Greg,
>
> I'll send this privately, maybe better not to post log-files, etc.
> to the list. :-)
>
>
>> Nobody's reported it before, but I think the CephFS MDS is sending out
>> too many delete requests. [...]
>>
>> That's all speculation on my part though; can you go sample the slow
>> requests and see what their makeup looked like? Do you have logs from
>> the MDS or OSDs during that time period?
>
>
> Uh - how do I sample the requests?

I believe the slow requests should have been logged in the monitor's
central log. That's a file sitting in the mon directory, and is
probably accessible via other means I can't think of off-hand. Go see
if it describes what the slow OSD requests are (eg, are they a bunch
of MDS deletes with some other stuff sprinkled in, or all other stuff,
or whatever).

> Concerning logs - you mean the regular ceph daemon log files?
> Sure - I'm attaching a tarball of all daemon logs from the
> relevant time interval (please don't publish them ;-) ).
> It's 13.2 MB, I hope it goes through by email.
>
>
> I also dumped "ceph health" every minute during the test.
>
> * 15:34:34 to 15:48:37 is the effect from my first mass delete. I aborted
>   that one before it could finished, to see if emperor would to better

By abort, you mean you stopped deleting all the things you intended to?
<snip>
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux