Cgroup is a key technology in container-based computing. It achieves resource control. In recent years, we have seen BPF being used extensively in many use cases. But in areas like security auditing [1] and performance monitoring, BPF can still be improved to provide better cgroup granularity stats. In this LSF/MM/BPF, I would like to drive a discussion on potential new features in BPF for better observability: 1. Interface to export per-cgroup stats. There are two considerations. a. BPF iter is a great way to export kernel state. Iter objects pinned in bpffs allow users to read stats as files. Cgroup-parameterized iter [2] would be a great addition that generates one stats file for each cgroup. This creates a better organized interface for cgroup stats. b. Containerized applications may also create their cgroups. They may also want to read the stats of their cgroups. This requirement needs us to make BPF based cgroup stats visible to the processes inside the container. But BPF based stats are exposed through bpffs, unlike the cgroup native stats which are exported through cgroup file system. Mounting the whole bpffs inside the container isn't a good idea as it exposes other task's stats to the container. It would be great to have the ability to mount only a bpffs directory into a container. 2. Stats collection for cgroups. a. Cgroups are hierarchical. For most of the time, the users want stats not of a single cgroup, but the aggregated stats of a cgroup tree. For example, in k8s, a Pod contains one or more containers. There is a cgroup for the Pod and a child cgroup for each container in the pod. When we read the stats for the Pod, we would want the stats to be aggregated stats of the child cgroups. Therefore aggregation is a natural requirement for cgroup stats collection. The proposal of rstat map [3] utilizes the rstat framework in the kernel for this purpose. Above are some topics in my mind, I am looking forward to any feedback. Thanks, Hao [1] Secure Namespaced Kernel Audit for Containers https://tfjmp.org/publications/2021-socc.pdf [2] Extend cgroup interface with bpf https://lwn.net/Articles/886292/ [3] Hierarchical Cgroup Stats Collection Using BPF https://www.spinics.net/lists/linux-mm/msg288283.html
