I'm seeing some hangs when testing with TLS in v6.7-rc8. This kernel has
the patch I sent this morning to get rid of nfsd_put, but otherwise is
stock v6.7-rc8:
[ 2125.174937] run fstests generic/126 at 2024-01-03 09:46:39
[ 2129.793577] run fstests generic/127 at 2024-01-03 09:46:44
[ 3199.661565] run fstests generic/128 at 2024-01-03 10:04:33
[ 3204.502354] run fstests generic/129 at 2024-01-03 10:04:38
[ 3208.111189] RPC: Could not send backchannel reply error: -110
[ 3384.487762] INFO: task looptest:92512 blocked for more than 120 seconds.
[ 3384.491396] Not tainted 6.7.0-rc8-g6c5361baaf84 #74
[ 3384.494103] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3384.497215] task:looptest state:D stack:0 pid:92512 tgid:92512 ppid:92333 flags:0x00004002
[ 3384.499471] Call Trace:
[ 3384.500129] <TASK>
[ 3384.500717] __schedule+0x3c4/0xad0
[ 3384.501652] schedule+0x31/0xd0
[ 3384.502455] io_schedule+0x42/0x70
[ 3384.503344] folio_wait_bit_common+0x121/0x330
[ 3384.504442] ? __pfx_wake_page_function+0x10/0x10
[ 3384.505624] folio_wait_writeback+0x27/0x80
[ 3384.506639] __filemap_fdatawait_range+0x79/0xe0
[ 3384.507343] filemap_write_and_wait_range+0x81/0xb0
[ 3384.508043] nfs_wb_all+0x21/0x120 [nfs]
[ 3384.508739] nfs4_file_flush+0x6e/0xb0 [nfsv4]
[ 3384.509499] filp_flush+0x30/0x70
[ 3384.510045] filp_close+0xf/0x30
[ 3384.510639] put_files_struct+0x78/0xd0
[ 3384.511222] do_exit+0x345/0xb10
[ 3384.511764] ? handle_mm_fault+0x9e/0x360
[ 3384.512364] ? preempt_count_add+0x47/0xa0
[ 3384.513012] do_group_exit+0x2d/0x80
[ 3384.513592] __x64_sys_exit_group+0x14/0x20
[ 3384.514252] do_syscall_64+0x3f/0xf0
[ 3384.514809] entry_SYSCALL_64_after_hwframe+0x6e/0x76
[ 3384.515581] RIP: 0033:0x7f564911595d
[ 3384.516186] RSP: 002b:00007ffecd703338 EFLAGS: 00000202 ORIG_RAX: 00000000000000e7
[ 3384.517306] RAX: ffffffffffffffda RBX: 00007f5649211fa8 RCX: 00007f564911595d
[ 3384.518368] RDX: 00000000000000e7 RSI: ffffffffffffff28 RDI: 0000000000000000
[ 3384.519473] RBP: 00007ffecd703390 R08: 00007ffecd7032d8 R09: 00007ffecd70323f
[ 3384.520543] R10: 00007ffecd703190 R11: 0000000000000202 R12: 0000000000000001
[ 3384.521597] R13: 0000000000000000 R14: 0000000000000000 R15: 00007f5649211fc0
[ 3384.522647] </TASK>
[root@kdevops-nfs-tls ~]# cat /sys/kernel/debug/sunrpc/rpc_clnt/*/tasks
6 5285 0 0x0 0x0 0 nfs41_sequence_ops [nfsv4] nfsv4 SEQUENCE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1460 c805 -11 0x5 0x2a2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1461 c805 -11 0x5 0x2b2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1462 c805 -11 0x5 0x2c2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1463 c805 -11 0x5 0x2d2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1464 c805 -11 0x5 0x302113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1465 c805 -11 0x5 0x2e2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1466 c805 -11 0x5 0x2f2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1467 c805 -11 0x5 0x312113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1468 c805 -11 0x5 0x322113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1469 c805 -11 0x5 0x332113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1470 c805 -11 0x5 0x342113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1471 c805 -11 0x5 0x352113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1472 c805 -11 0x5 0x362113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1473 c805 -11 0x5 0x372113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1474 c805 -11 0x5 0x382113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1475 c805 -11 0x5 0x392113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1476 c805 -11 0x5 0x3a2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1477 c805 -11 0x5 0x3b2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1478 c805 -11 0x5 0x3c2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1479 c805 -11 0x5 0x3d2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1480 c805 -11 0x5 0x3e2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1481 c805 -11 0x5 0x3f2113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1482 c805 -11 0x5 0x402113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1483 c805 -11 0x5 0x412113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1484 c805 -11 0x5 0x422113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
1485 c005 -11 0x5 0x432113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
1486 c005 -11 0x5 0x442113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
1487 c005 -11 0x5 0x452113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
1488 c005 -11 0x5 0x462113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
1489 c005 -11 0x5 0x472113be 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
1490 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1491 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1492 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1493 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1494 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1495 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1496 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1497 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1498 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1499 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1500 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1501 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1502 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1503 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1504 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1505 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1506 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1507 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1508 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1509 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1510 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1511 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1512 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1513 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1514 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1515 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1516 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1517 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1518 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1519 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1520 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1521 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1522 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1523 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1524 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1525 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1526 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1527 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1528 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1529 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1530 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1531 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1532 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1533 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1534 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1535 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1536 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1537 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1538 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1539 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1540 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1541 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1542 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1543 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1544 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1545 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1546 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1547 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1548 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1549 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1550 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1551 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1552 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1553 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1554 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1555 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1556 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1557 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1558 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1559 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1560 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1561 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1562 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1563 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1564 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1565 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1566 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1567 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1568 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1569 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1570 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1571 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1572 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1573 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1574 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1575 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1576 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1577 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1578 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1579 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1580 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1581 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1582 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1583 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1584 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1585 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1586 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1587 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1588 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1589 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1590 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1591 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1592 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1593 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1594 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1595 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1596 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1597 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
1598 c005 0 0x5 0x0 0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
In the server, I see this in the log around the same time:
[ 2344.590202] rpc-srv/tcp: nfsd: got error -90 when sending 112 bytes - shutting down socket
[ 2349.182087] rpc-srv/tcp: nfsd: got error -74 when sending 112 bytes - shutting down socket
[ 2352.685424] rpc-srv/tcp: nfsd: got error -90 when sending 112 bytes - shutting down socket
[ 3583.897575] rpc-srv/tcp: nfsd: got error -104 when sending 112 bytes - shutting down socket
/proc/fs/nfsd/clients is empty however, so the client has been timed out
at this point.
On the wire, it looks like the client is just not communicating at all
with the server. Turning up tracepoints and rpcdebug info doesn't show
any activity whatsoever. It's like the client has just stopped trying to
do anything.
I was going to start digging more into the client to figure out what's
going on, but figured I'd send this in case others had seen something
similar. Anyone?
--
Jeff Layton <jlayton@xxxxxxxxxx>
