On 4/28/20 11:56 PM, Andrii Nakryiko wrote:
On Mon, Apr 27, 2020 at 1:19 PM Yonghong Song <yhs@xxxxxx> wrote:A new bpf command BPF_ITER_CREATE is added. The anonymous bpf iterator is seq_file based. The seq_file private data are referenced by targets. The bpf_iter infrastructure allocated additional space at seq_file->private after the space used by targets to store some meta data, e.g., prog: prog to run session_id: an unique id for each opened seq_file seq_num: how many times bpf programs are queried in this session has_last: indicate whether or not bpf_prog has been called after all valid objects have been processed A map between file and prog/link is established to help fops->release(). When fops->release() is called, just based on inode and file, bpf program cannot be located since target seq_priv_size not available. This map helps retrieve the prog whose reference count needs to be decremented. Signed-off-by: Yonghong Song <yhs@xxxxxx> --- include/linux/bpf.h | 3 + include/uapi/linux/bpf.h | 6 ++ kernel/bpf/bpf_iter.c | 162 ++++++++++++++++++++++++++++++++- kernel/bpf/syscall.c | 27 ++++++ tools/include/uapi/linux/bpf.h | 6 ++ 5 files changed, 203 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 4fc39d9b5cd0..0f0cafc65a04 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1112,6 +1112,8 @@ struct bpf_link *bpf_link_get_from_fd(u32 ufd); int bpf_obj_pin_user(u32 ufd, const char __user *pathname); int bpf_obj_get_user(const char __user *pathname, int flags); +#define BPF_DUMP_SEQ_NET_PRIVATE BIT(0) + struct bpf_iter_reg { const char *target; const char *target_func_name; @@ -1133,6 +1135,7 @@ int bpf_iter_run_prog(struct bpf_prog *prog, void *ctx); int bpf_iter_link_attach(const union bpf_attr *attr, struct bpf_prog *prog); int bpf_iter_link_replace(struct bpf_link *link, struct bpf_prog *old_prog, struct bpf_prog *new_prog); +int bpf_iter_new_fd(struct bpf_link *link); int bpf_percpu_hash_copy(struct bpf_map *map, void *key, void *value); int bpf_percpu_array_copy(struct bpf_map *map, void *key, void *value); diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index f39b9fec37ab..576651110d16 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -113,6 +113,7 @@ enum bpf_cmd { BPF_MAP_DELETE_BATCH, BPF_LINK_CREATE, BPF_LINK_UPDATE, + BPF_ITER_CREATE, }; enum bpf_map_type { @@ -590,6 +591,11 @@ union bpf_attr { __u32 old_prog_fd; } link_update; + struct { /* struct used by BPF_ITER_CREATE command */ + __u32 link_fd; + __u32 flags; + } iter_create; + } __attribute__((aligned(8))); /* The description below is an attempt at providing documentation to eBPF diff --git a/kernel/bpf/bpf_iter.c b/kernel/bpf/bpf_iter.c index fc1ce5ee5c3f..1f4e778d1814 100644 --- a/kernel/bpf/bpf_iter.c +++ b/kernel/bpf/bpf_iter.c @@ -2,6 +2,7 @@ /* Copyright (c) 2020 Facebook */ #include <linux/fs.h> +#include <linux/anon_inodes.h> #include <linux/filter.h> #include <linux/bpf.h> @@ -19,6 +20,19 @@ struct bpf_iter_link { struct bpf_iter_target_info *tinfo; }; +struct extra_priv_data { + struct bpf_prog *prog; + u64 session_id; + u64 seq_num; + bool has_last; +}; + +struct anon_file_prog_assoc { + struct list_head list; + struct file *file; + struct bpf_prog *prog; +}; + static struct list_head targets; static struct mutex targets_mutex; static bool bpf_iter_inited = false; @@ -26,6 +40,50 @@ static bool bpf_iter_inited = false; /* protect bpf_iter_link.link->prog upddate */ static struct mutex bpf_iter_mutex; +/* Since at anon seq_file release function, the prog cannot + * be retrieved since target seq_priv_size is not available. + * Keep a list of <anon_file, prog> mapping, so that + * at file release stage, the prog can be released properly. + */ +static struct list_head anon_iter_info; +static struct mutex anon_iter_info_mutex; + +/* incremented on every opened seq_file */ +static atomic64_t session_id; + +static u32 get_total_priv_dsize(u32 old_size) +{ + return roundup(old_size, 8) + sizeof(struct extra_priv_data); +} + +static void *get_extra_priv_dptr(void *old_ptr, u32 old_size) +{ + return old_ptr + roundup(old_size, 8); +} + +static int anon_iter_release(struct inode *inode, struct file *file) +{ + struct anon_file_prog_assoc *finfo; + + mutex_lock(&anon_iter_info_mutex); + list_for_each_entry(finfo, &anon_iter_info, list) { + if (finfo->file == file) {I'll look at this and other patches more thoroughly tomorrow with clear head, but this iteration to find anon_file_prog_assoc is really unfortunate. I think the problem is that you are allowing seq_file infrastructure to call directly into target implementation of seq_operations without intercepting them. If you change that and put whatever extra info is necessary into seq_file->private in front of target's private state, then you shouldn't need this, right?
Yes. This is true. The idea is to minimize the target change. But maybe this is not a good goal by itself. You are right, if I intercept all seq_ops(), I do not need the above change, I can tailor seq_file private_data right before calling target one and restore after the target call. Originally I only have one interception, show(), now I have stop() too to call bpf at the end of iteration. Maybe I can interpret all four, I think. This way, I can also get ride of target feature.
This would also make each target's logic a bit simpler because you can: - centralize creation and initialization of bpf_iter_meta (session_id, seq, seq_num will be set up once in this generic code); - loff_t pos increments; - you can extract and centralize bpf_iter_get_prog() call in show() implementation as well. I think with that each target's logic will be simpler and you won't need to maintain anon_file_prog_assocs. Are there complications I'm missing? [...]
