Hi Hao, I am not talking to myself :-) Thanks for addressing the question. Please see inline. On Mon, Jul 11, 2022 at 4:35 PM Hao Luo <haoluo@xxxxxxxxxx> wrote: > > Hi Hao Xiang, > > On Mon, Jul 11, 2022 at 3:50 PM Hao Xiang . <hao.xiang@xxxxxxxxxxxxx> wrote: > > > > Ping... > > > > Can someone please help to shed some light on this? > > > > Thanks, Hao > > > > On Sun, Jul 3, 2022 at 3:33 PM Hao Xiang . <hao.xiang@xxxxxxxxxxxxx> wrote: > > > > > > Hi everyone, > > > > > > I am super new to bpf and the open source community in general. Please > > > bear with me asking some basic questions. > > > We are working on a bpf monitoring tool to track the CPU and memory > > > usage for all bpf programs loaded in the system. We were able to get > > > CPU usage per bpf program with the BPF_OBJ_GET_INFO_BY_ID syscall on a > > > bpf prog object. We are trying to do the same on a map object to query > > > for per map memory usage. The information returned from bpf_map_info > > > only contains things like max_entries, key_size, value_size, which can > > > be used to calculate estimated memory allocation size. But we are also > > > interested in knowing how much memory is actually being used by our > > > program. For instance, one of our bpf program uses a map with type > > > hashtable. The hashtable is created with a chunk of pre-allocated > > > memory based on the max_entres, key_size and value size. The > > > pre-allocated size is useful information to know but so is the current > > > number of entries in the hashtable. We used to run into a performance > > > issue where our bpf map's max_entries is set to be too small and we > > > end up totally exhausting the pre-allocated memory. So knowing things > > > like current entry count VS max entry count of a hashtable is useful > > > information for us. > > > With that being said, we have a few questions and hopefully we can get > > > some help from the community. > > > 1) We couldn't find anything in bpf_map_info to give us the current > > > entry count of a hashtable. I read that bpf_map_info returns > > > information about a map object in general. So it makes total sense to > > > not have information of a particular map type. But is there an > > > existing place we can get the per map type information (eg, the > > > current entry count of a hashtable, the number of elements pushed to a > > > stack, etc)? > > cc more BPF experts for their comments. > > > I agree with you that knowing the current space usage of a map is > > quite helpful. In my understanding, a naive and inefficient way to > > estimate space usage is iterating the map and counting the map > > elements. That's doable, but may not be the best method though. > > Regarding auto-adjusting map size, I remember Andrii talked about > > resizable hash maps, maybe he can tell you more [1]. > > I agree it's doable to get the map count by iterating through the map. I just > looked through the code and realized that it's quite heavy doing it that way. > Basically we will have to make N bpf syscalls where N = used count. Our monitor > program wants to retrieve the current usage in a higher frequency and the underlying > cost looks pretty significant. Other than that, is it expected that a user process who > can do BPF_OBJ_GET_INFO_BY_FD requires the same privilege as someone > who can do BPF_MAP_GET_NEXT_KEY? I would expect that only the process > talking to the bpf prog can update/delete/get keys of a map but a process who only > wants to monitor the prog/map requires less privilege? > > > [1] http://vger.kernel.org/bpfconf2022_material/lsfmmbpf2022-algs.pdf > > > > 2) If there isn't an existing place to return map type specific > > > information, would it make sense to extend the structure bpf_map_info > > > with a union at the end and have that union to contain per map type > > > specific information? > > > > > > Thanks, Hao
