On 3/6/20 10:00 AM, Jens Axboe wrote: > On 3/6/20 9:44 AM, Paul E. McKenney wrote: >> On Fri, Mar 06, 2020 at 04:36:20PM +0100, Jann Horn wrote: >>> On Fri, Mar 6, 2020 at 4:34 PM Jens Axboe <axboe@xxxxxxxxx> wrote: >>>> On 3/6/20 7:57 AM, Jann Horn wrote: >>>>> +paulmck >>>>> >>>>> On Wed, Mar 4, 2020 at 3:40 PM Jens Axboe <axboe@xxxxxxxxx> wrote: >>>>>> On 3/4/20 12:59 AM, Dmitry Vyukov wrote: >>>>>>> On Fri, Feb 7, 2020 at 9:14 AM syzbot >>>>>>> <syzbot+e017e49c39ab484ac87a@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote: >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> syzbot found the following crash on: >>>>>>>> >>>>>>>> HEAD commit: 4c7d00cc Merge tag 'pwm/for-5.6-rc1' of git://git.kernel.o.. >>>>>>>> git tree: upstream >>>>>>>> console output: https://syzkaller.appspot.com/x/log.txt?x=12fec785e00000 >>>>>>>> kernel config: https://syzkaller.appspot.com/x/.config?x=e162021ddededa72 >>>>>>>> dashboard link: https://syzkaller.appspot.com/bug?extid=e017e49c39ab484ac87a >>>>>>>> compiler: clang version 10.0.0 (https://github.com/llvm/llvm-project/ c2443155a0fb245c8f17f2c1c72b6ea391e86e81) >>>>>>>> >>>>>>>> Unfortunately, I don't have any reproducer for this crash yet. >>>>>>>> >>>>>>>> IMPORTANT: if you fix the bug, please add the following tag to the commit: >>>>>>>> Reported-by: syzbot+e017e49c39ab484ac87a@xxxxxxxxxxxxxxxxxxxxxxxxx >>>>>>> >>>>>>> +io_uring maintainers >>>>>>> >>>>>>> Here is a repro: >>>>>>> https://gist.githubusercontent.com/dvyukov/6b340beab6483a036f4186e7378882ce/raw/cd1922185516453c201df8eded1d4b006a6d6a3a/gistfile1.txt >>>>>> >>>>>> I've queued up a fix for this: >>>>>> >>>>>> https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-5.6&id=9875fe3dc4b8cff1f1b440fb925054a5124403c3 >>>>> >>>>> I believe that this fix relies on call_rcu() having FIFO ordering; but >>>>> <https://www.kernel.org/doc/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.html#Callback%20Registry> >>>>> says: >>>>> >>>>> | call_rcu() normally acts only on CPU-local state[...] It simply >>>>> enqueues the rcu_head structure on a per-CPU list, >> >> Indeed. For but one example, if there was a CPU-to-CPU migration between >> the two call_rcu() invocations, it would not be at all surprising for >> the two callbacks to execute out of order. >> >>>>> Is this fix really correct? >>>> >>>> That's a good point, there's a potentially stronger guarantee we need >>>> here that isn't "nobody is inside an RCU critical section", but rather >>>> that we're depending on a previous call_rcu() to have happened. Hence I >>>> think you are right - it'll shrink the window drastically, since the >>>> previous callback is already queued up, but it's not a full close. >>>> >>>> Hmm... >>> >>> You could potentially hack up the semantics you want by doing a >>> call_rcu() whose callback does another call_rcu(), or something like >>> that - but I'd like to hear paulmck's opinion on this first. >> >> That would work! >> >> Or, alternatively, do an rcu_barrier() between the two calls to >> call_rcu(), assuming that the use case can tolerate rcu_barrier() >> overhead and latency. > > If the nested call_rcu() works, that seems greatly preferable to needing > the rcu_barrier(), even if that would not be a showstopper for me. The > nested call_rcu() is just a bit odd, but with a comment it should be OK. > > Incremental here I'm going to test, would just fold in of course. Been running for a few minutes just fine, I'm going to leave the reproducer beating on it for a few hours. But here's the folded in final: https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-5.6&id=fae702294a6a0774ceb3cf250be79e7fe207250a -- Jens Axboe
