I'm seeing some hangs when testing with TLS in v6.7-rc8. This kernel has the patch I sent this morning to get rid of nfsd_put, but otherwise is stock v6.7-rc8: [ 2125.174937] run fstests generic/126 at 2024-01-03 09:46:39 [ 2129.793577] run fstests generic/127 at 2024-01-03 09:46:44 [ 3199.661565] run fstests generic/128 at 2024-01-03 10:04:33 [ 3204.502354] run fstests generic/129 at 2024-01-03 10:04:38 [ 3208.111189] RPC: Could not send backchannel reply error: -110 [ 3384.487762] INFO: task looptest:92512 blocked for more than 120 seconds. [ 3384.491396] Not tainted 6.7.0-rc8-g6c5361baaf84 #74 [ 3384.494103] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3384.497215] task:looptest state:D stack:0 pid:92512 tgid:92512 ppid:92333 flags:0x00004002 [ 3384.499471] Call Trace: [ 3384.500129] <TASK> [ 3384.500717] __schedule+0x3c4/0xad0 [ 3384.501652] schedule+0x31/0xd0 [ 3384.502455] io_schedule+0x42/0x70 [ 3384.503344] folio_wait_bit_common+0x121/0x330 [ 3384.504442] ? __pfx_wake_page_function+0x10/0x10 [ 3384.505624] folio_wait_writeback+0x27/0x80 [ 3384.506639] __filemap_fdatawait_range+0x79/0xe0 [ 3384.507343] filemap_write_and_wait_range+0x81/0xb0 [ 3384.508043] nfs_wb_all+0x21/0x120 [nfs] [ 3384.508739] nfs4_file_flush+0x6e/0xb0 [nfsv4] [ 3384.509499] filp_flush+0x30/0x70 [ 3384.510045] filp_close+0xf/0x30 [ 3384.510639] put_files_struct+0x78/0xd0 [ 3384.511222] do_exit+0x345/0xb10 [ 3384.511764] ? handle_mm_fault+0x9e/0x360 [ 3384.512364] ? preempt_count_add+0x47/0xa0 [ 3384.513012] do_group_exit+0x2d/0x80 [ 3384.513592] __x64_sys_exit_group+0x14/0x20 [ 3384.514252] do_syscall_64+0x3f/0xf0 [ 3384.514809] entry_SYSCALL_64_after_hwframe+0x6e/0x76 [ 3384.515581] RIP: 0033:0x7f564911595d [ 3384.516186] RSP: 002b:00007ffecd703338 EFLAGS: 00000202 ORIG_RAX: 00000000000000e7 [ 3384.517306] RAX: ffffffffffffffda RBX: 00007f5649211fa8 RCX: 00007f564911595d [ 3384.518368] RDX: 00000000000000e7 RSI: ffffffffffffff28 RDI: 0000000000000000 [ 3384.519473] RBP: 00007ffecd703390 R08: 00007ffecd7032d8 R09: 00007ffecd70323f [ 3384.520543] R10: 00007ffecd703190 R11: 0000000000000202 R12: 0000000000000001 [ 3384.521597] R13: 0000000000000000 R14: 0000000000000000 R15: 00007f5649211fc0 [ 3384.522647] </TASK> [root@kdevops-nfs-tls ~]# cat /sys/kernel/debug/sunrpc/rpc_clnt/*/tasks 6 5285 0 0x0 0x0 0 nfs41_sequence_ops [nfsv4] nfsv4 SEQUENCE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1460 c805 -11 0x5 0x2a2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1461 c805 -11 0x5 0x2b2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1462 c805 -11 0x5 0x2c2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1463 c805 -11 0x5 0x2d2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1464 c805 -11 0x5 0x302113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1465 c805 -11 0x5 0x2e2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1466 c805 -11 0x5 0x2f2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1467 c805 -11 0x5 0x312113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1468 c805 -11 0x5 0x322113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1469 c805 -11 0x5 0x332113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1470 c805 -11 0x5 0x342113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1471 c805 -11 0x5 0x352113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1472 c805 -11 0x5 0x362113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1473 c805 -11 0x5 0x372113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1474 c805 -11 0x5 0x382113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1475 c805 -11 0x5 0x392113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1476 c805 -11 0x5 0x3a2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1477 c805 -11 0x5 0x3b2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1478 c805 -11 0x5 0x3c2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1479 c805 -11 0x5 0x3d2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1480 c805 -11 0x5 0x3e2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1481 c805 -11 0x5 0x3f2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1482 c805 -11 0x5 0x402113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1483 c805 -11 0x5 0x412113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1484 c805 -11 0x5 0x422113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending 1485 c005 -11 0x5 0x432113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending 1486 c005 -11 0x5 0x442113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending 1487 c005 -11 0x5 0x452113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending 1488 c005 -11 0x5 0x462113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending 1489 c005 -11 0x5 0x472113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending 1490 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1491 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1492 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1493 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1494 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1495 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1496 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1497 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1498 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1499 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1500 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1501 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1502 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1503 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1504 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1505 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1506 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1507 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1508 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1509 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1510 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1511 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1512 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1513 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1514 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1515 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1516 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1517 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1518 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1519 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1520 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1521 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1522 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1523 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1524 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1525 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1526 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1527 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1528 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1529 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1530 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1531 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1532 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1533 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1534 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1535 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1536 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1537 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1538 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1539 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1540 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1541 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1542 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1543 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1544 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1545 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1546 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1547 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1548 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1549 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1550 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1551 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1552 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1553 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1554 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1555 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1556 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1557 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1558 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1559 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1560 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1561 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1562 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1563 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1564 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1565 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1566 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1567 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1568 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1569 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1570 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1571 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1572 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1573 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1574 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1575 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1576 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1577 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1578 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1579 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1580 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1581 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1582 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1583 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1584 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1585 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1586 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1587 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1588 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1589 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1590 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1591 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1592 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1593 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1594 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1595 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1596 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1597 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table 1598 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table In the server, I see this in the log around the same time: [ 2344.590202] rpc-srv/tcp: nfsd: got error -90 when sending 112 bytes - shutting down socket [ 2349.182087] rpc-srv/tcp: nfsd: got error -74 when sending 112 bytes - shutting down socket [ 2352.685424] rpc-srv/tcp: nfsd: got error -90 when sending 112 bytes - shutting down socket [ 3583.897575] rpc-srv/tcp: nfsd: got error -104 when sending 112 bytes - shutting down socket /proc/fs/nfsd/clients is empty however, so the client has been timed out at this point. On the wire, it looks like the client is just not communicating at all with the server. Turning up tracepoints and rpcdebug info doesn't show any activity whatsoever. It's like the client has just stopped trying to do anything. I was going to start digging more into the client to figure out what's going on, but figured I'd send this in case others had seen something similar. Anyone? -- Jeff Layton <jlayton@xxxxxxxxxx>