Calling iput inside OSD dispatcher thread might cause deadlock?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,
Recently I was running into a hung task on kernel client.
After investigating the environment, I think there might be a deadlock
caused by calling iput inside OSD dispatch(ceph-msgr) thread

My kernel cpeh client version is 5.15
The call stack for the kernel ceph-msgr thread is:

[<0>] wait_on_page_bit_common+0x106/0x300
[<0>] truncate_inode_pages_range+0x381/0x6a0
[<0>] ceph_evict_inode+0x4a/0x200 [ceph]
[<0>] evict+0xc6/0x190
[<0>] ceph_put_wrbuffer_cap_refs+0xdf/0x1d0 [ceph]
[<0>] writepages_finish+0x2c4/0x440 [ceph]
[<0>] handle_reply+0x5be/0x6d0 [libceph]
[<0>] dispatch+0x49/0xa60 [libceph]
[<0>] ceph_con_workfn+0x10fa/0x24b0 [libceph]
[<0>] worker_run_work+0xb8/0xd0
[<0>] process_one_work+0x1d3/0x3c0
[<0>] worker_thread+0x4d/0x3e0
[<0>] kthread+0x12d/0x150
[<0>] ret_from_fork+0x1f/0x30

And the following messages are from osdc in ceph debugfs

2774296 osd30 4.1ea6f009 4.9 [30,14]/30 [30,14]/30 e18281
200006dabea.00000003 0x40002c 92 write
...
2774578 osd30 4.1ea6f009 4.9 [30,14]/30 [30,14]/30 e18281
200006dabea.00000003 0x400014 1 read
2774579 osd30 4.1ea6f009 4.9 [30,14]/30 [30,14]/30 e18281
200006dabea.00000003 0x400014 1 read
2774580 osd30 4.1ea6f009 4.9 [30,14]/30 [30,14]/30 e18281
200006dabea.00000003 0x400014 1 read
2774581 osd30 4.1ea6f009 4.9 [30,14]/30 [30,14]/30 e18281
200006dabea.00000003 0x400014 1 read
2774582 osd30 4.1ea6f009 4.9 [30,14]/30 [30,14]/30 e18281
200006dabea.00000003 0x400014 1 read
2774583 osd30 4.1ea6f009 4.9 [30,14]/30 [30,14]/30 e18281
200006dabea.00000003 0x400014 1 read
2774584 osd30 4.1ea6f009 4.9 [30,14]/30 [30,14]/30 e18281
200006dabea.00000003 0x400014 1 read
...

We can see that kernel client has sent both write and multiple read
requests on object 200006dabea.00000003.
The iput_final waits for truncate_inode_pages_range to finish which in
turn waits for page bit.

[Index of Archives]     [CEPH Users]     [Ceph Large]     [Ceph Dev]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux