hangs during fstests testing with TLS

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I'm seeing some hangs when testing with TLS in v6.7-rc8. This kernel has
the patch I sent this morning to get rid of nfsd_put, but otherwise is
stock v6.7-rc8:

[ 2125.174937] run fstests generic/126 at 2024-01-03 09:46:39
[ 2129.793577] run fstests generic/127 at 2024-01-03 09:46:44
[ 3199.661565] run fstests generic/128 at 2024-01-03 10:04:33
[ 3204.502354] run fstests generic/129 at 2024-01-03 10:04:38
[ 3208.111189] RPC: Could not send backchannel reply error: -110
[ 3384.487762] INFO: task looptest:92512 blocked for more than 120 seconds.
[ 3384.491396]       Not tainted 6.7.0-rc8-g6c5361baaf84 #74
[ 3384.494103] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3384.497215] task:looptest        state:D stack:0     pid:92512 tgid:92512 ppid:92333  flags:0x00004002
[ 3384.499471] Call Trace:
[ 3384.500129]  <TASK>
[ 3384.500717]  __schedule+0x3c4/0xad0
[ 3384.501652]  schedule+0x31/0xd0
[ 3384.502455]  io_schedule+0x42/0x70
[ 3384.503344]  folio_wait_bit_common+0x121/0x330
[ 3384.504442]  ? __pfx_wake_page_function+0x10/0x10
[ 3384.505624]  folio_wait_writeback+0x27/0x80
[ 3384.506639]  __filemap_fdatawait_range+0x79/0xe0
[ 3384.507343]  filemap_write_and_wait_range+0x81/0xb0
[ 3384.508043]  nfs_wb_all+0x21/0x120 [nfs]
[ 3384.508739]  nfs4_file_flush+0x6e/0xb0 [nfsv4]
[ 3384.509499]  filp_flush+0x30/0x70
[ 3384.510045]  filp_close+0xf/0x30
[ 3384.510639]  put_files_struct+0x78/0xd0
[ 3384.511222]  do_exit+0x345/0xb10
[ 3384.511764]  ? handle_mm_fault+0x9e/0x360
[ 3384.512364]  ? preempt_count_add+0x47/0xa0
[ 3384.513012]  do_group_exit+0x2d/0x80
[ 3384.513592]  __x64_sys_exit_group+0x14/0x20
[ 3384.514252]  do_syscall_64+0x3f/0xf0
[ 3384.514809]  entry_SYSCALL_64_after_hwframe+0x6e/0x76
[ 3384.515581] RIP: 0033:0x7f564911595d
[ 3384.516186] RSP: 002b:00007ffecd703338 EFLAGS: 00000202 ORIG_RAX: 00000000000000e7
[ 3384.517306] RAX: ffffffffffffffda RBX: 00007f5649211fa8 RCX: 00007f564911595d
[ 3384.518368] RDX: 00000000000000e7 RSI: ffffffffffffff28 RDI: 0000000000000000
[ 3384.519473] RBP: 00007ffecd703390 R08: 00007ffecd7032d8 R09: 00007ffecd70323f
[ 3384.520543] R10: 00007ffecd703190 R11: 0000000000000202 R12: 0000000000000001
[ 3384.521597] R13: 0000000000000000 R14: 0000000000000000 R15: 00007f5649211fc0
[ 3384.522647]  </TASK>

[root@kdevops-nfs-tls ~]# cat /sys/kernel/debug/sunrpc/rpc_clnt/*/tasks
    6 5285      0 0x0 0x0        0 nfs41_sequence_ops [nfsv4] nfsv4 SEQUENCE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1460 c805    -11 0x5 0x2a2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1461 c805    -11 0x5 0x2b2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1462 c805    -11 0x5 0x2c2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1463 c805    -11 0x5 0x2d2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1464 c805    -11 0x5 0x302113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1465 c805    -11 0x5 0x2e2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1466 c805    -11 0x5 0x2f2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1467 c805    -11 0x5 0x312113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1468 c805    -11 0x5 0x322113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1469 c805    -11 0x5 0x332113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1470 c805    -11 0x5 0x342113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1471 c805    -11 0x5 0x352113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1472 c805    -11 0x5 0x362113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1473 c805    -11 0x5 0x372113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1474 c805    -11 0x5 0x382113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1475 c805    -11 0x5 0x392113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1476 c805    -11 0x5 0x3a2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1477 c805    -11 0x5 0x3b2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1478 c805    -11 0x5 0x3c2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1479 c805    -11 0x5 0x3d2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1480 c805    -11 0x5 0x3e2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1481 c805    -11 0x5 0x3f2113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1482 c805    -11 0x5 0x402113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1483 c805    -11 0x5 0x412113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1484 c805    -11 0x5 0x422113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_connect_status [sunrpc] q:xprt_sending
 1485 c005    -11 0x5 0x432113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
 1486 c005    -11 0x5 0x442113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
 1487 c005    -11 0x5 0x452113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
 1488 c005    -11 0x5 0x462113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
 1489 c005    -11 0x5 0x472113be        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:call_transmit_status [sunrpc] q:xprt_sending
 1490 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1491 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1492 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1493 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1494 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1495 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1496 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1497 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1498 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1499 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1500 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1501 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1502 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1503 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1504 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1505 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1506 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1507 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1508 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1509 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1510 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1511 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1512 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1513 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1514 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1515 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1516 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1517 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1518 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1519 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1520 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1521 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1522 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1523 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1524 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1525 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1526 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1527 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1528 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1529 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1530 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1531 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1532 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1533 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1534 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1535 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1536 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1537 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1538 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1539 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1540 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1541 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1542 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1543 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1544 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1545 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1546 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1547 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1548 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1549 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1550 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1551 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1552 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1553 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1554 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1555 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1556 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1557 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1558 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1559 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1560 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1561 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1562 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1563 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1564 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1565 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1566 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1567 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1568 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1569 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1570 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1571 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1572 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1573 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1574 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1575 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1576 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1577 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1578 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1579 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1580 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1581 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1582 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1583 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1584 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1585 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1586 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1587 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1588 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1589 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1590 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1591 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1592 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1593 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1594 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1595 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1596 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1597 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table
 1598 c005      0 0x5 0x0        0 nfs_pgio_common_ops [nfs] nfsv4 WRITE a:rpc_prepare_task [sunrpc] q:ForeChannel Slot table

In the server, I see this in the log around the same time:

[ 2344.590202] rpc-srv/tcp: nfsd: got error -90 when sending 112 bytes - shutting down socket
[ 2349.182087] rpc-srv/tcp: nfsd: got error -74 when sending 112 bytes - shutting down socket
[ 2352.685424] rpc-srv/tcp: nfsd: got error -90 when sending 112 bytes - shutting down socket
[ 3583.897575] rpc-srv/tcp: nfsd: got error -104 when sending 112 bytes - shutting down socket

/proc/fs/nfsd/clients is empty however, so the client has been timed out
at this point.

On the wire, it looks like the client is just not communicating at all
with the server. Turning up tracepoints and rpcdebug info doesn't show
any activity whatsoever. It's like the client has just stopped trying to
do anything.

I was going to start digging more into the client to figure out what's
going on, but figured I'd send this in case others had seen something
similar. Anyone?
-- 
Jeff Layton <jlayton@xxxxxxxxxx>





[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux