Hi Hao Xiang, On Mon, Jul 11, 2022 at 3:50 PM Hao Xiang . <hao.xiang@xxxxxxxxxxxxx> wrote: > > Ping... > > Can someone please help to shed some light on this? > > Thanks, Hao > > On Sun, Jul 3, 2022 at 3:33 PM Hao Xiang . <hao.xiang@xxxxxxxxxxxxx> wrote: > > > > Hi everyone, > > > > I am super new to bpf and the open source community in general. Please > > bear with me asking some basic questions. > > We are working on a bpf monitoring tool to track the CPU and memory > > usage for all bpf programs loaded in the system. We were able to get > > CPU usage per bpf program with the BPF_OBJ_GET_INFO_BY_ID syscall on a > > bpf prog object. We are trying to do the same on a map object to query > > for per map memory usage. The information returned from bpf_map_info > > only contains things like max_entries, key_size, value_size, which can > > be used to calculate estimated memory allocation size. But we are also > > interested in knowing how much memory is actually being used by our > > program. For instance, one of our bpf program uses a map with type > > hashtable. The hashtable is created with a chunk of pre-allocated > > memory based on the max_entres, key_size and value size. The > > pre-allocated size is useful information to know but so is the current > > number of entries in the hashtable. We used to run into a performance > > issue where our bpf map's max_entries is set to be too small and we > > end up totally exhausting the pre-allocated memory. So knowing things > > like current entry count VS max entry count of a hashtable is useful > > information for us. > > With that being said, we have a few questions and hopefully we can get > > some help from the community. > > 1) We couldn't find anything in bpf_map_info to give us the current > > entry count of a hashtable. I read that bpf_map_info returns > > information about a map object in general. So it makes total sense to > > not have information of a particular map type. But is there an > > existing place we can get the per map type information (eg, the > > current entry count of a hashtable, the number of elements pushed to a > > stack, etc)? cc more BPF experts for their comments. I agree with you that knowing the current space usage of a map is quite helpful. In my understanding, a naive and inefficient way to estimate space usage is iterating the map and counting the map elements. That's doable, but may not be the best method though. Regarding auto-adjusting map size, I remember Andrii talked about resizable hash maps, maybe he can tell you more [1]. [1] http://vger.kernel.org/bpfconf2022_material/lsfmmbpf2022-algs.pdf > > 2) If there isn't an existing place to return map type specific > > information, would it make sense to extend the structure bpf_map_info > > with a union at the end and have that union to contain per map type > > specific information? > > > > Thanks, Hao
