BPF Single Step Debugging ========================= Developers have long used single-step debugging for some user-space tricky bugs. But for bpf programs running inside the kernel, true single-step is not really an option. Possible solutions would be: (1). maybe we can run the bpf programs on a VM, and we can single-step bpf program by repeatedly freezing and waking up VMs. (2). maybe we can resign bpf_prog_test_run() interface to at least allow single steps for programs in bpf_prog_test_run() environment. (3). maybe we can simulate single steps in production environment by attaching kprobes all instructions in a program. Each of the above approaches has some limitations. Furthermore, source level single stepping is much more preferred than optimized code (-O2) single stepping as the former prints out in much user friendly formats. We would like to examine the above three different options as well as how far we are away from source level debugging and what we could do about it. -- Yonghong