"git bisect" led to a result. The offending merge commit is f91f2ee54a21404fbc633550e99d69d14c2478f2 "Merge branch 'akpm' (rest of patches from Andrew)". The first bad commit in the merged series of commits is https://github.com/torvalds/linux/commit/6b4c9f4469819a0c1a38a0a4541337e0f9bf6c11 . a75d4c33377277b6034dd1e2663bce444f952c14, the commit before 6b4c9f44, is good. I've also verified 5.1.21 and 5.3.8 (from https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/) without 6b4c9f4469819a0c1a38a0a4541337e0f9bf6c11 and both builds are good. (The 5.1.21 and 5.3.7 builds from Ubuntu were bad, so I haven't cross- checked "vanilla" 5.1.21 and 5.3.8 kernel builds.) Recap symptoms: - `mlockall(MCL_CURRENT)` hangs - shutdown/reboot hangs when it reaches "shutdown->reboot" - `cat /proc/$(pidof test)/smaps` shows "Locked" w/ odd values, which are equal to "Pss" Affected: - `cryptsetup luksOpen` hangs (when it tries to lock memory) - Apache Cassandra hangs during startup (when it performs an `mlockall(MCL_CURRENT)`) git checkout v5.1.21 # revert the "comment-only" commit (no need to test this one) # "filemap: add a comment about FAULT_FLAG_RETRY_NOWAIT behavior" git revert 8b0f9fa2e02dc95216577c3387b0707c5f60fbaf # "filemap: drop the mmap_sem for all blocking operations" git revert 6b4c9f4469819a0c1a38a0a4541337e0f9bf6c11 --> GOOD git checkout v5.3.8 # revert the "comment-only" commit (no need to test this one) # "filemap: add a comment about FAULT_FLAG_RETRY_NOWAIT behavior" git revert 8b0f9fa2e02dc95216577c3387b0707c5f60fbaf # "filemap: drop the mmap_sem for all blocking operations" git revert 6b4c9f4469819a0c1a38a0a4541337e0f9bf6c11 --> GOOD Bisect log: git bisect start # bad: [9e98c678c2d6ae3a17cb2de55d17f69dddaa231b] Linux 5.1-rc1 git bisect bad 9e98c678c2d6ae3a17cb2de55d17f69dddaa231b # good: [1c163f4c7b3f621efff9b28a47abb36f7378d783] Linux 5.0 git bisect good 1c163f4c7b3f621efff9b28a47abb36f7378d783 # good: [e266ca36da7de45b64b05698e98e04b578a88888] Merge tag 'staging- 5.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging git bisect good e266ca36da7de45b64b05698e98e04b578a88888 # good: [36011ddc78395b59a8a418c37f20bcc18828f1ef] Merge tag 'gfs2- 5.1.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/gfs2/linux- gfs2 git bisect good 36011ddc78395b59a8a418c37f20bcc18828f1ef # good: [6bc3fe8e7e172d5584e529a04cf9eec946428768] tools: mark 'test_vmalloc.sh' executable git bisect good 6bc3fe8e7e172d5584e529a04cf9eec946428768 # good: [dc2535be1fd547fbd56aff091370280007b0a1af] Merge tag 'clk-for- linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux git bisect good dc2535be1fd547fbd56aff091370280007b0a1af # bad: [2b9c272cf5cd81708e51b4ce3e432ce9566cfa47] Merge tag 'fbdev- v5.1' of git://github.com/bzolnier/linux git bisect bad 2b9c272cf5cd81708e51b4ce3e432ce9566cfa47 # good: [9bc446100334dbbc14eb3757274ef08746c3f9bd] Merge tag 'microblaze-v5.1-rc1' of git://git.monstr.eu/linux-2.6-microblaze git bisect good 9bc446100334dbbc14eb3757274ef08746c3f9bd # bad: [5160bcce5c3c80de7d8722511c144d3041409657] Merge tag 'f2fs-for- 5.1' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs git bisect bad 5160bcce5c3c80de7d8722511c144d3041409657 # good: [240a59156d9bcfabceddb66be449e7b32fb5dc4a] f2fs: fix to add refcount once page is tagged PG_private git bisect good 240a59156d9bcfabceddb66be449e7b32fb5dc4a # good: [9352ca585b2ac7b67d2119b9386573b2a4c0ef4b] Merge tag 'pm-5.1- rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm git bisect good 9352ca585b2ac7b67d2119b9386573b2a4c0ef4b # good: [f261c4e529dac5608a604d3dd3ae1cd2adf23c89] Merge branch 'akpm' (patches from Andrew) git bisect good f261c4e529dac5608a604d3dd3ae1cd2adf23c89 # good: [aadcef64b22f668c1a107b86d3521d9cac915c24] f2fs: fix to avoid deadlock in f2fs_read_inline_dir() git bisect good aadcef64b22f668c1a107b86d3521d9cac915c24 # bad: [8b0f9fa2e02dc95216577c3387b0707c5f60fbaf] filemap: add a comment about FAULT_FLAG_RETRY_NOWAIT behavior git bisect bad 8b0f9fa2e02dc95216577c3387b0707c5f60fbaf # bad: [6b4c9f4469819a0c1a38a0a4541337e0f9bf6c11] filemap: drop the mmap_sem for all blocking operations git bisect bad 6b4c9f4469819a0c1a38a0a4541337e0f9bf6c11 # bad: [a75d4c33377277b6034dd1e2663bce444f952c14] filemap: kill page_cache_read usage in filemap_fault git bisect good a75d4c33377277b6034dd1e2663bce444f952c14 All kernels built with make oldconfig # accept the defaults make bindeb-pkg On Fri, 2019-10-25 at 17:58 +0200, Robert Stupp wrote: > On Fri, 2019-10-25 at 16:00 +0200, Michal Hocko wrote: > > And one more thing. Considering that you are able to reproduce and > > you > > have a working kernel, could you try to bisect this? > > Yikes - running self-built kernels brings back a lot of memories ;) > > Anyway, going this route (using the `config` from Ubuntu 5.1.x as a > base and accepting the defaults for `make oldconfig`): > > git checkout v5.1-rc1 > git bisect start > git bisect bad > git bisect good v5.0 > > ... first try @ e266ca36da7de45b64b05698e98e04b578a88888 is a `git > bisect good` > > Will report back, when I've got a result... >
