On Mon, Nov 11, 2019 at 10:37 AM Jakub Kicinski <jakub.kicinski@xxxxxxxxxxxxx> wrote: > > On Sat, 9 Nov 2019 00:06:30 -0800, Andrii Nakryiko wrote: > > With BPF_F_MMAPABLE array allocating data in separate chunk of memory, > > array_map_gen_lookup has to accomodate these changes. For non-memory-mapped > > there are no changes and no extra instructions. For BPF_F_MMAPABLE case, > > pointer to where array data is stored has to be dereferenced first. > > > > Generated code for non-memory-mapped array: > > > > ; p = bpf_map_lookup_elem(&data_map, &zero); > > 22: (18) r1 = map[id:19] > > 24: (07) r1 += 408 /* array->inline_data offset */ > > 25: (61) r0 = *(u32 *)(r2 +0) > > 26: (35) if r0 >= 0x3 goto pc+3 > > 27: (67) r0 <<= 3 > > 28: (0f) r0 += r1 > > 29: (05) goto pc+1 > > 30: (b7) r0 = 0 > > > > Generated code for memory-mapped array: > > > > ; p = bpf_map_lookup_elem(&data_map, &zero); > > 22: (18) r1 = map[id:27] > > 24: (07) r1 += 400 /* array->data offset */ > > 25: (79) r1 = *(u64 *)(r1 +0) /* extra dereference */ > > 26: (61) r0 = *(u32 *)(r2 +0) > > 27: (35) if r0 >= 0x3 goto pc+3 > > 28: (67) r0 <<= 3 > > 29: (0f) r0 += r1 > > 30: (05) goto pc+1 > > 31: (b7) r0 = 0 > > Would it not be possible to overallocate the memory and align the start > of the bpf_map in case of BPF_F_MMAPABLE so that no extra dereference > is needed? So let's say if sizeof(struct bpf_array) is 300, then I'd have to either: - somehow make sure that I allocate 4k (for data) + 300 (for struct bpf_array) in such a way that those 4k of data are 4k-aligned. Is there any way to do that? - assuming there isn't, then another way would be to allocate entire 4k page for struct bpf_array itself, but put it at the end of that page, so that 4k of data is 4k-aligned. While wasteful, the bigger problem is that pointer to bpf_array is not a pointer to allocated memory anymore, so we'd need to remember that and adjust address before calling vfree(). Were you suggesting #2 as a solution? Or am I missing some other way to do this?
