Hi Krzysztof, > On Jan 8, 2023, at 08:25, Krzysztof Kozlowski <krzysztof.kozlowski@xxxxxxxxxx> wrote: > > [You don't often get email from krzysztof.kozlowski@xxxxxxxxxx. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification] > > On 07/01/2023 16:44, Krzysztof Kozlowski wrote: >> Hi, >> >> Bisect identified commit 7fd461c47c6c ("NFSv4.2: Change the default >> KConfig value for READ_PLUS") as one leading to NULL pointer exception >> when mounting NFS root on NFSv4 client: >> >> [ 25.739003] systemd[1]: Set hostname to <odroidhc1>. >> [ 25.771714] systemd[1]: Failed to bump fs.file-max, ignoring: Invalid >> argument >> [ 26.199478] 8<--- cut here --- >> [ 26.201366] Unable to handle kernel NULL pointer dereference at >> virtual address 00000004 >> ... >> [ 26.555522] mmiocpy from xdr_inline_decode+0xec/0x16c >> [ 26.560628] xdr_inline_decode from nfs4_xdr_dec_read_plus+0x178/0x358 >> [ 26.567130] nfs4_xdr_dec_read_plus from call_decode+0x204/0x304 >> >> Full OOPS attached. Full log available here: >> https://krzk.eu/#/builders/21/builds/3901/steps/15/logs/serial0 >> >> Disabling NFS_V4_2_READ_PLUS fixes the issue, so obviously the commit is >> not the cause, but rather making it default caused the regression. >> >> I did not make the bisect yet which commit introduced it, if every >> config includes NFS_V4_2_READ_PLUS. > > When every kernel is built with NFS_V4_2_READ_PLUS, bisect pointed to: > d3b00a802c84 ("NFS: Replace the READ_PLUS decoding code") > > commit d3b00a802c845a6021148ce2e669b5a0b5729959 > Author: Anna Schumaker <Anna.Schumaker@xxxxxxxxxx> > Date: Thu Jul 21 14:21:34 2022 -0400 > > NFS: Replace the READ_PLUS decoding code > > We now take a 2-step process that allows us to place data and hole > segments directly at their final position in the xdr_stream without > needing to do a bunch of redundant copies to expand holes. Due to the > variable lengths of each segment, the xdr metadata might cross page > boundaries which I account for by setting a small scratch buffer so > xdr_inline_decode() won't fail. > > Signed-off-by: Anna Schumaker <Anna.Schumaker@xxxxxxxxxx> > Signed-off-by: Trond Myklebust <trond.myklebust@xxxxxxxxxxxxxxx> > > With a trace: > [ 25.898462] systemd[1]: Set hostname to <odroidhc1>. > [ 25.933746] systemd[1]: Failed to bump fs.file-max, ignoring: Invalid > argument > [ 25.986237] random: crng init done > [ 26.264564] 8<--- cut here --- > [ 26.266823] Unable to handle kernel NULL pointer dereference at > virtual address 00000fe8 > ... > [ 26.597263] nfs4_xdr_dec_read_plus from call_decode+0x204/0x304 > [ 26.603222] call_decode from __rpc_execute+0xd0/0x890 > [ 26.608328] __rpc_execute from rpc_async_schedule+0x1c/0x34 > [ 26.613960] rpc_async_schedule from process_one_work+0x294/0x790 > [ 26.620030] process_one_work from worker_thread+0x54/0x518 > [ 26.625570] worker_thread from kthread+0xf4/0x128 > [ 26.630336] kthread from ret_from_fork+0x14/0x2c > Is this test being run against a 6.2-rc2 server, or is it an older server platform? We know there were bugs in older server implementations, so the question is whether this might be a problem with handling a bad/corrupt RPC reply from the server, or whether it is happening against code that is supposed to have been fixed? Thanks Trond _________________________________ Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@xxxxxxxxxxxxxxx
