On Thu, Apr 13, 2023 at 7:47 PM Daniel Borkmann <daniel@xxxxxxxxxxxxx> wrote: > > On 4/13/23 4:53 AM, Yafang Shao wrote: > > In our container environment, we are using EDT-bpf to limit the egress > > bandwidth. EDT-bpf can be used to limit egress only, but can't be used > > to limit ingress. Some of our users also want to limit the ingress > > bandwidth. But after applying EDT-bpf, which is based on clsact qdisc, > > it is impossible to limit the ingress bandwidth currently, due to some > > reasons, > > 1). We can't add ingress qdisc > > The ingress qdisc can't coexist with clsact qdisc as clsact has both > > ingress and egress handler. So our traditional method to limit ingress > > bandwidth can't work any more. > > I'm not following, the latter is a super set of the former, why do you > need it to co-exist? > It seems that I have a misunderstanding. > > 2). We can't redirect ingress packet to ifb with bpf > > By trying to analyze if it is possible to redirect the ingress packet to > > ifb with a bpf program, we find that the ifb device is not supported by > > bpf redirect yet. > > You actually can: Just let BPF program return TC_ACT_UNSPEC for this > case and then add a matchall with higher prio (so it runs after bpf) > that contains an action with mirred egress redirect that pushes to ifb > dev - there is no change needed. > Ah, indeed, it works. Many thanks for your help. > > This patch tries to resolve it by supporting redirecting to ifb with bpf > > program. > > > > Ingress bandwidth limit is useful in some scenarios, for example, for the > > TCP-based service, there may be lots of clients connecting it, so it is > > not wise to limit the clients' egress. After limiting the server-side's > > ingress, it will lower the send rate of the client by lowering the TCP > > cwnd if the ingress bandwidth limit is reached. If we don't limit it, > > the clients will continue sending requests at a high rate. > > Adding artificial queueing for the inbound traffic, aren't you worried > about DoS'ing your node? Yes, we worried about it, but we haven't observed it in our data center. > If you need to tell the sender to slow down, > have you looked at hbm (https://lpc.events/event/4/contributions/486/, > samples/bpf/hbm_out_kern.c) which uses ECN CE marking to tell the TCP > sender to slow down? (Fwiw, for UDP https://github.com/cloudflare/rakelimit > would be an option.) > We're considering using ECN. Thanks for your information, I will analyze it. -- Regards Yafang
