@John-Paul Robinson: I’ve also experienced nfs being blocked when serving rbd devices (XFS system). In my scenario I had rbd device mapped on an OSD host and nfs exported (lab scenario). Log entries below.. Running Centos 7 w/ 3.10.0-229.14.1.el7.x86_64. Next step for me is to compile 3.18.22 and test nfs and scst (iscsi / fc). Oct 22 13:30:01 osdhost01 systemd: Started Session 14 of user root. Oct 22 13:37:04 osdhost01 kernel: INFO: task nfsd:12672 blocked for more than 120 seconds. Oct 22 13:37:04 osdhost01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Oct 22 13:37:04 osdhost01 kernel: nfsd D ffff880627c73680 0 12672 2 0x00000080 Oct 22 13:37:04 osdhost01 kernel: ffff880bda763b08 0000000000000046 ffff880be73af1c0 ffff880bda763fd8 Oct 22 13:37:04 osdhost01 kernel: ffff880bda763fd8 ffff880bda763fd8 ffff880be73af1c0 ffff880627c73f48 Oct 22 13:37:04 osdhost01 kernel: ffff880c3ff98ae8 0000000000000002 ffffffff811562e0 ffff880bda763b80 Oct 22 13:37:04 osdhost01 kernel: Call Trace: Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811562e0>] ? wait_on_page_read+0x60/0x60 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81609b3d>] io_schedule+0x9d/0x130 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811562ee>] sleep_on_page+0xe/0x20 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81607910>] __wait_on_bit+0x60/0x90 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81156076>] wait_on_page_bit+0x86/0xb0 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81098280>] ? autoremove_wake_function+0x40/0x40 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811561b1>] filemap_fdatawait_range+0x111/0x1b0 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81157edf>] filemap_write_and_wait_range+0x3f/0x70 Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa04db706>] xfs_file_fsync+0x66/0x1f0 [xfs] Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811f6cad>] vfs_fsync_range+0x1d/0x30 Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa06db659>] nfsd_commit+0xb9/0xe0 [nfsd] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa06e8b97>] nfsd4_commit+0x57/0x60 [nfsd] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa06ea257>] nfsd4_proc_compound+0x4d7/0x7f0 [nfsd] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa06d5e1b>] nfsd_dispatch+0xbb/0x200 [nfsd] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa069bb53>] svc_process_common+0x453/0x6f0 [sunrpc] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa069bef3>] svc_process+0x103/0x170 [sunrpc] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa06d57a7>] nfsd+0xe7/0x150 [nfsd] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa06d56c0>] ? nfsd_destroy+0x80/0x80 [nfsd] Oct 22 13:37:04 osdhost01 kernel: [<ffffffff8109727f>] kthread+0xcf/0xe0 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff810971b0>] ? kthread_create_on_node+0x140/0x140 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff816142d8>] ret_from_fork+0x58/0x90 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff810971b0>] ? kthread_create_on_node+0x140/0x140 Oct 22 13:37:04 osdhost01 kernel: INFO: task kworker/u50:81:15660 blocked for more than 120 seconds. Oct 22 13:37:04 osdhost01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Oct 22 13:37:04 osdhost01 kernel: kworker/u50:81 D ffff880c3fc73680 0 15660 2 0x00000080 Oct 22 13:37:04 osdhost01 kernel: Workqueue: writeback bdi_writeback_workfn (flush-252:0) Oct 22 13:37:04 osdhost01 kernel: ffff88086deeb738 0000000000000046 ffff880beb6796c0 ffff88086deebfd8 Oct 22 13:37:04 osdhost01 kernel: ffff88086deebfd8 ffff88086deebfd8 ffff880beb6796c0 ffff880c3fc73f48 Oct 22 13:37:04 osdhost01 kernel: ffff88061aec0fc0 ffff880c1bb2dea0 ffff88061aec0ff0 ffff88061aec0fc0 Oct 22 13:37:04 osdhost01 kernel: Call Trace: Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81609b3d>] io_schedule+0x9d/0x130 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff812ac785>] get_request+0x1b5/0x780 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81098240>] ? wake_up_bit+0x30/0x30 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff812aea66>] blk_queue_bio+0xc6/0x390 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff812aa8c2>] generic_make_request+0xe2/0x130 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff812aa981>] submit_bio+0x71/0x150 Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa04d0813>] xfs_submit_ioend_bio.isra.12+0x33/0x40 [xfs] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa04d090f>] xfs_submit_ioend+0xef/0x130 [xfs] Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa04d16ca>] xfs_vm_writepage+0x36a/0x5d0 [xfs] Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811610a3>] __writepage+0x13/0x50 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81161bc1>] write_cache_pages+0x251/0x4d0 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81161090>] ? global_dirtyable_memory+0x70/0x70 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81161e8d>] generic_writepages+0x4d/0x80 Oct 22 13:37:04 osdhost01 kernel: [<ffffffffa04d0ea3>] xfs_vm_writepages+0x43/0x50 [xfs] Oct 22 13:37:04 osdhost01 kernel: [<ffffffff81162f3e>] do_writepages+0x1e/0x40 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811f0340>] __writeback_single_inode+0x40/0x220 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811f103e>] writeback_sb_inodes+0x25e/0x420 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811f129f>] __writeback_inodes_wb+0x9f/0xd0 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811f1ae3>] wb_writeback+0x263/0x2f0 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff811f301c>] bdi_writeback_workfn+0x1cc/0x460 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff8108f0cb>] process_one_work+0x17b/0x470 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff8108fe9b>] worker_thread+0x11b/0x400 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff8108fd80>] ? rescuer_thread+0x400/0x400 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff8109727f>] kthread+0xcf/0xe0 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff810971b0>] ? kthread_create_on_node+0x140/0x140 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff816142d8>] ret_from_fork+0x58/0x90 Oct 22 13:37:04 osdhost01 kernel: [<ffffffff810971b0>] ? kthread_create_on_node+0x140/0x140 |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
