On Tue, Sep 30, 2014 at 1:30 PM, Micha Krause <micha at krausam.de> wrote:
> Hi,
>
>> > > That's strange. 3.13 is way before any changes that could have had
>> any
>>>>
>>>> such effect. Can you by any chance try with older kernels to see where
>>>> it starts misbehaving for you? 3.12? 3.10? 3.8?
>>>
>>>
>> If I have to compile Kernels anyway I will test 3.16.3 as well :-/.
>
>
> Debian has released Kernel 3.16.3 a few days ago, so I tested it, and it
> also hangs:
>
> osdc shows:
>
> 3628783 osd21 37.b0474ca0 rb.0.167614e.2ae8944a.000000038b0d
> ??? write
> 3628784 osd21 37.b0474ca0 rb.0.167614e.2ae8944a.000000038b0d
> ??? write
> 3628785 osd21 37.b0474ca0 rb.0.167614e.2ae8944a.000000038b0d
> ??? write
> 3628788 osd25 37.63cdc62f rb.0.1676178.2ae8944a.000000017ff5
> read
> 3628789 osd26 37.6bc418b7 rb.0.1689cf4.238e1f29.000000000288
> ??? write
> 3628790 osd19 37.256efded rb.0.1689cf4.238e1f29.00000000029a
> ??? write
> 3628791 osd26 37.268309ac rb.0.1689cf4.238e1f29.00000000029b
> ??? write
> 3628792 osd22 37.b3b9e9bd rb.0.1689cf4.238e1f29.00000000029d
> ??? write
> 3628793 osd21 37.4eec30a0 rb.0.1689cf4.238e1f29.000000001fff
> ??? write
> 3628794 osd19 37.8a60bdc7 rb.0.1689cf4.238e1f29.0000000020d4
> ??? write
> 3628795 osd18 37.c904eca5 rb.0.1689cf4.238e1f29.0000000020d5
> ??? write
> 3628796 osd18 37.b71458a6 rb.0.1689cf4.238e1f29.000000005212
> ??? write
> 3628797 osd29 37.5e179f39 rb.0.1345def.2ae8944a.0000000c8011
> ??? write
> 3628798 osd23 37.3e6fc4fb rb.0.1345def.2ae8944a.0000000c8013
> ??? write
> 3628799 osd26 37.8467da30 rb.0.1345def.2ae8944a.0000000c8957
> ??? write
> 3628800 osd16 37.d720935f rb.0.1345def.2ae8944a.0000000c895a
> ??? write
> 3628801 osd28 37.7925c9e4 rb.0.1345def.2ae8944a.0000000c8a79
> ??? write
> 3628802 osd20 37.bcdf4c74 rb.0.1345def.2ae8944a.0000000c8a7c
> ??? write
> 3628803 osd26 37.ebc514ac rb.0.1345def.2ae8944a.0000000dc031
> ??? write
> 3628804 osd27 37.77bd6435 rb.0.1345def.2ae8944a.0000000dd74e
> ??? write
> 3628805 osd28 37.b724973a rb.0.1345def.2ae8944a.0000000dd751
> ??? write
> 3628806 osd26 37.14b308b0 rb.0.1345def.2ae8944a.0000000efff4
> ??? write
> 3628807 osd23 37.adf44248 rb.0.1345def.2ae8944a.0000000f001a
> ??? write
> 3628808 osd26 37.629f422c rb.0.1345def.2ae8944a.0000000f0ccf
> ??? write
> 3628809 osd20 37.90d5ce6b rb.0.1345def.2ae8944a.0000000f0cd3
> ??? write
> 3628810 osd27 37.11918e9b rb.0.1345def.2ae8944a.000000103ff3
> ??? write
> 3628811 osd29 37.286d56b9 rb.0.1345def.2ae8944a.000000104023
> ??? write
> 3628812 osd28 37.191f0724 rb.0.1345def.2ae8944a.00000010459f
> ??? write
> 3628813 osd18 37.69e6111d rb.0.1345def.2ae8944a.0000001045a1
> ??? write
>
> this output does not change.
>
> dmesg shows hung task stuff again, but no rbd related lines.
Hi Micha,
Do you have any osd logs from this occurence? They are usually in
/var/log/ceph.
Thanks,
Ilya
