Hi Anna,
This trace on the client shows that it was requested to use *nfl_util
0x400000*
filelayout_decode_layout: set_layout_map Begin
Sep 13 09:20:10 svl-marcrh-node-1 kernel: nfs4_print_deviceid: device
id= [3000035a0b38c07df0465]
Sep 13 09:20:10 svl-marcrh-node-1 kernel: filelayout_decode_layout:
*nfl_util 0x400000* num_fh 1 fsi 0 po 0
Sep 13 09:20:10 svl-marcrh-node-1 kernel: DEBUG:
filelayout_decode_layout: fh len 61
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_check_layout
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_check_layout
returns 0
This trace on the DS show that it reads 0x48000 before skipping to 0x880000
as you can see it reads 0x48000 from DS 10.11.56.194 before starting to
read from 10.11.56.195
and I also show it on the server side that revived those reads.
Sep 13 09:20:10 svl-marcrh-node-1 kernel:
pnfs_generic_layout_insert_lseg:Begin
Sep 13 09:20:10 svl-marcrh-node-1 kernel:
pnfs_generic_layout_insert_lseg: inserted lseg 00000000867a0461 iomode 1
offset 0 length 18446744073709551615 at tail
Sep 13 09:20:10 svl-marcrh-node-1 kernel:
pnfs_generic_layout_insert_lseg:Return
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_find_alloc_layout Begin
ino=000000003c8009bb layout=00000000f5222588
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_update_layout: inode
0:51/88064 pNFS layout segment found for (read-only, offset: 0, length:
18446744073709551615)
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_read_pagelist
ino 88064 pgbase 0 req 1048576@0
Sep 13 09:20:10 svl-marcrh-node-1 kernel: filelayout_read_pagelist USE
DS: {10.11.56.194:2049,} cl_count 2
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_find_alloc_layout Begin
ino=000000003c8009bb layout=00000000f5222588
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_update_layout: inode
0:51/88064 pNFS layout segment found for (read-only, offset: 0, length:
18446744073709551615)
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_read_pagelist
ino 88064 pgbase 0 req 524288@1048576
Sep 13 09:20:10 svl-marcrh-node-1 kernel: filelayout_read_pagelist USE
DS: {10.11.56.194:2049,} cl_count 1
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_find_alloc_layout Begin
ino=000000003c8009bb layout=00000000f5222588
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_update_layout: inode
0:51/88064 pNFS layout segment found for (read-only, offset: 0, length:
18446744073709551615)
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_read_pagelist
ino 88064 pgbase 0 req 1048576@1572864
Sep 13 09:20:10 svl-marcrh-node-1 kernel: filelayout_read_pagelist USE
DS: {10.11.56.194:2049,} cl_count 2
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_find_alloc_layout Begin
ino=000000003c8009bb layout=00000000f5222588
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_update_layout: inode
0:51/88064 pNFS layout segment found for (read-only, offset: 0, length:
18446744073709551615)
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_read_pagelist
ino 88064 pgbase 0 req 524288@2621440
Sep 13 09:20:10 svl-marcrh-node-1 kernel: filelayout_read_pagelist USE
DS: {10.11.56.194:2049,} cl_count 1
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_find_alloc_layout Begin
ino=000000003c8009bb layout=00000000f5222588
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_update_layout: inode
0:51/88064 pNFS layout segment found for (read-only, offset: 0, length:
18446744073709551615)
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_read_pagelist
ino 88064 pgbase 0 req 524288@3145728
Sep 13 09:20:10 svl-marcrh-node-1 kernel: filelayout_read_pagelist USE
DS: {10.11.56.194:2049,} cl_count 2
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_find_alloc_layout Begin
ino=000000003c8009bb layout=00000000f5222588
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_update_layout: inode
0:51/88064 pNFS layout segment found for (read-only, offset: 0, length:
18446744073709551615)
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_read_pagelist
ino 88064 pgbase 0 req 1048576@3670016
Sep 13 09:20:10 svl-marcrh-node-1 kernel: filelayout_read_pagelist USE
DS: {10.11.56.194:2049,} cl_count 1
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_find_alloc_layout Begin
ino=000000003c8009bb layout=00000000f5222588
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_update_layout: inode
0:51/88064 pNFS layout segment found for (read-only, offset: 0, length:
18446744073709551615)
Sep 13 09:20:10 svl-marcrh-node-1 kernel: --> filelayout_read_pagelist
ino 88064 pgbase 0 req 524288@4718592
Sep 13 09:20:10 svl-marcrh-node-1 kernel: filelayout_read_pagelist USE
DS: {10.11.56.195:2049,} cl_count 2
Sep 13 09:20:10 svl-marcrh-node-1 kernel: pnfs_find_alloc_layout Begin
ino=000000003c8009bb layout=00000000f5222588
So it is skipping *0x400000 *but why was the first chuck 0x48000
Also way some read are 1048576 and most are 524288
And this is on the server side.
gpfsRead enter: gnP 0xFFFF91F810942688 flags 0x1 uioP 0xFFFFAED0C47878B0
vinfoP 0xFFFF91F81352FA30 off 0 len 1048576
gpfsRead enter: gnP 0xFFFF91F810942688 flags 0x1 uioP 0xFFFFAED0C47878B0
vinfoP 0xFFFF91F81352FA30 off 1048576 len 524288
gpfsRead enter: gnP 0xFFFF91F810942688 flags 0x1 uioP 0xFFFFAED0CA01F8B0
vinfoP 0xFFFF91F81352FA30 off 1572864 len 1048576
gpfsRead enter: gnP 0xFFFF91F810942688 flags 0x1 uioP 0xFFFFAED0CA5FB8B0
vinfoP 0xFFFF91F81352FA30 off 2621440 len 524288
gpfsRead enter: gnP 0xFFFF91F810942688 flags 0x1 uioP 0xFFFFAED0CA01F8B0
vinfoP 0xFFFF91F81352FA30 off 3145728 len 524288
gpfsRead enter: gnP 0xFFFF91F810942688 flags 0x1 uioP 0xFFFFAED0C47878B0
vinfoP 0xFFFF91F81352FA30 off 3670016 len 1048576
gpfsRead enter: gnP 0xFFFF91F810942688 flags 0x1 uioP 0xFFFFAED0CA01F8B0
vinfoP 0xFFFF91F81352FA30 off 8912896 len 524288
Thanks, Marc.
On 9/16/24 1:33 PM, Anna Schumaker wrote:
Hi Marc,
On 9/16/24 4:27 PM, marc eshel wrote:
Hi,
Who is the current maintainer of the Linux NFS client?
That would be me and Trond.
I have an issue with the way pNFS client is distributing the reads among the DSs and I can provide the traces if anyone cares.
What is the problem? And sure, send along the traces! I'll try to take a look as soon as I can.
Anna
Thanks, Marc.
