Re: BPF regression in 5.10.168 and 5.15.93 impacting Cilium

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Jun 14, 2023 at 2:48 PM Alexei Starovoitov
<alexei.starovoitov@xxxxxxxxx> wrote:
>
> On Wed, Jun 14, 2023 at 1:57 PM Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx> wrote:
> >
> > On Wed, Jun 14, 2023 at 11:23:52AM -0700, Robert Kolchmeyer wrote:
> > > Hi all,
> > >
> > > I believe 5.10.168 and 5.15.93 introduced a regression that impacts
> > > the Cilium project. Some information on the nature of the regression
> > > is available at https://github.com/cilium/cilium/issues/25500. The
> > > primary symptom seems to be the error `BPF program is too large.`
> > >
> > > My colleague has found that reverting the following two commits:
> > >
> > > 8de8c4a "bpf: Support <8-byte scalar spill and refill"
> > > 9ff2beb "bpf: Fix incorrect state pruning for <8B spill/fill"
> > >
> > > resolves the regression.
> > >
> > > If we revert these in the stable tree, there may be a few changes that
> > > depend on those that also need to be reverted, but I'm not sure yet.
> > >
> > > Would it make sense to revert these changes (and any dependent ones)
> > > in the 5.10 and 5.15 trees? If anyone has other ideas, I can help test
> > > possible solutions.
> >
> > Can you actually test if those reverts work properly for you and if
> > there are other dependencies involved?
> >
> > And is this issue also in 6.1.y and Linus's tree?  If not, why not, are
> > we just missing a commit?  We can't revert something from a stable
> > release if you are going to hit the same issue when moving to a new
> > release, right?
> >
> > thanks,
> >
> > greg k-h
>
> Before jumping to reverts..
> how is it fixed in 6.0+ kernels?
>
> "BPF program is too large" can probably be worked around on the cilium side.
> The kernel cannot guarantee that a particular program will
> always be verifiable. We find safety bugs in the verifier and often
> enough the fixes to such issues make the verifier work harder to prove
> the safety of the program.
> This is one of such cases. These two commits are necessary.
> Reverting them will prevent loading of valid programs.
> So reverts is a dangerous path.
> The best is to identify the other patches from 6.0+ and backport them.
> The second best path is to bump 1M limit to something higher to
> mitigate "more work by the verifier".

Thanks all for the pointers, this is super helpful.

I'm working on getting more details, and I'll post here when I have them.

-Robert




[Index of Archives]     [Linux Kernel]     [Kernel Development Newbies]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite Hiking]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux