On 4/13/23 4:53 AM, Yafang Shao wrote:
In our container environment, we are using EDT-bpf to limit the egress bandwidth. EDT-bpf can be used to limit egress only, but can't be used to limit ingress. Some of our users also want to limit the ingress bandwidth. But after applying EDT-bpf, which is based on clsact qdisc, it is impossible to limit the ingress bandwidth currently, due to some reasons, 1). We can't add ingress qdisc The ingress qdisc can't coexist with clsact qdisc as clsact has both ingress and egress handler. So our traditional method to limit ingress bandwidth can't work any more.
I'm not following, the latter is a super set of the former, why do you need it to co-exist?
2). We can't redirect ingress packet to ifb with bpf By trying to analyze if it is possible to redirect the ingress packet to ifb with a bpf program, we find that the ifb device is not supported by bpf redirect yet.
You actually can: Just let BPF program return TC_ACT_UNSPEC for this case and then add a matchall with higher prio (so it runs after bpf) that contains an action with mirred egress redirect that pushes to ifb dev - there is no change needed.
This patch tries to resolve it by supporting redirecting to ifb with bpf program. Ingress bandwidth limit is useful in some scenarios, for example, for the TCP-based service, there may be lots of clients connecting it, so it is not wise to limit the clients' egress. After limiting the server-side's ingress, it will lower the send rate of the client by lowering the TCP cwnd if the ingress bandwidth limit is reached. If we don't limit it, the clients will continue sending requests at a high rate.
Adding artificial queueing for the inbound traffic, aren't you worried about DoS'ing your node? If you need to tell the sender to slow down, have you looked at hbm (https://lpc.events/event/4/contributions/486/, samples/bpf/hbm_out_kern.c) which uses ECN CE marking to tell the TCP sender to slow down? (Fwiw, for UDP https://github.com/cloudflare/rakelimit would be an option.) Thanks, Daniel
