Thanks Andrii. I am changing the email format to plain text. Hopefully this will work with the vger mailing list. >> I don't think the Linux side can/should work like that. Note that the proposal I made for Windows in previous email is *effectively* the same as the following: 1. Load a bpf program that reads data from a user supplied map and then writes the data into a ring buffer. 2. User space app populates the data map and then invoke the program using bpf_prog_test. Assuming this approach would be considered a valid use of eBPF, I think we can implement the API on Windows as proposed below. I will be happy to work with you to build a solution on Linux that is acceptable to you. Thanks, Shankar শংকর শীল From: Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> Sent: Wednesday, June 26, 2024 4:44 PM To: Shankar Seal <Shankar.Seal@xxxxxxxxxxxxx> Cc: Shankar Seal <Shankar.Seal=40microsoft.com@xxxxxxxxxxxxxx>; dthaler1968=40googlemail.com@xxxxxxxxxxxxxx; bpf@xxxxxxxx; bpf@xxxxxxxxxxxxxxx Subject: Re: [Bpf] Re: [EXTERNAL] RE: Re: Writing into a ring buffer map from user space >> On Mon, Jun 24, 2024 at 8:50 PM Shankar Seal <mailto:Shankar.Seal@xxxxxxxxxxxxx> wrote: >> Here is a brief overview of what we intend to do in the eBPF for Windows code: >> The user space app will not directly write into the underlying ring buffer of the eBPF map. Instead, the user app (via the libbpf API) will send the data via an IOCTL[1] to the eBPF core (a Windows Kernel Driver[2]) that manages the ring buffer map. The driver will internally invoke the same code that implements the bpf_ringbuf_output helper function to write the user provided data buffer into the ring buffer map. >>I am not aware of how the ring buffer map is implemented in the Linux kernel. But presumably a similar approach could be taken in Linux as well? >> [1] https://learn.microsoft.com/en-us/windows/win32/devio/device-input-and-output-control-ioctl- >> [2] https://learn.microsoft.com/en-us/windows/win32/devio/device-input-and-output-control-ioctl- I don't think the Linux side can/should work like that. Also, keep in mind that your HTML-based messages are not reaching mailto:bpf@xxxxxxxxxxxxxxx. So please fix your HTML set up and continue conversation over mailto:bpf@xxxxxxxxxxxxxxx. Thanks, Shankar শংকর শীল ________________________________________ From: Andrii Nakryiko <mailto:andrii.nakryiko@xxxxxxxxx> Sent: Monday, June 24, 2024 8:36 PM To: Shankar Seal <mailto:Shankar.Seal@xxxxxxxxxxxxx> Cc: Shankar Seal <Shankar.Seal=mailto:40microsoft.com@xxxxxxxxxxxxxx>; dthaler1968=mailto:40googlemail.com@xxxxxxxxxxxxxx <dthaler1968=mailto:40googlemail.com@xxxxxxxxxxxxxx>; mailto:bpf@xxxxxxxx <mailto:bpf@xxxxxxxx>; mailto:bpf@xxxxxxxxxxxxxxx <mailto:bpf@xxxxxxxxxxxxxxx> Subject: Re: [Bpf] Re: [EXTERNAL] RE: Re: Writing into a ring buffer map from user space On Thu, Jun 20, 2024 at 11:49 PM Shankar Seal <mailto:Shankar.Seal@xxxxxxxxxxxxx> wrote: Since I have not heard back on this topic, I am assuming that there are no strong oppositions to this idea. So I am sharing the signature of the proposed user API. /** * @brief Write data into the ring buffer map from user space. * * @param ring_buffer_map_fd ring buffer map file descriptor. * @param data Pointer to data to be written. * @param data_length Length of data to be written. * @retval 0 The operation was successful. * @retval <0 An error occured, and errno was set. */ int ring_buffer_user__write( fd_t ring_buffer_map_fd, const void* data, size_t data_length); Please let me know if you have any questions about this API. I think the devil will be in the details. API itself makes sense (you can't simplify it further or make it much different), in the end, you are just sending an array of bytes into ringbuf. But the implementation details are what matters. How the notification works. How user space won't break kernel even if intentionally trying, etc. It's not clear where you intend to implement this, etc. Thanks, Shankar শংকর শীল ________________________________________ From: Shankar Seal <Shankar.Seal=mailto:40microsoft.com@xxxxxxxxxxxxxx> Sent: Wednesday, June 5, 2024 10:01 PM To: dthaler1968=mailto:40googlemail.com@xxxxxxxxxxxxxx <dthaler1968=mailto:40googlemail.com@xxxxxxxxxxxxxx>; 'Andrii Nakryiko' <mailto:andrii.nakryiko@xxxxxxxxx> Cc: mailto:bpf@xxxxxxxx <mailto:bpf@xxxxxxxx>; mailto:bpf@xxxxxxxxxxxxxxx <mailto:bpf@xxxxxxxxxxxxxxx> Subject: [Bpf] Re: [EXTERNAL] RE: Re: Writing into a ring buffer map from user space You don't often get email from shankar.seal=mailto:40microsoft.com@xxxxxxxxxxxxxx. https://aka.ms/LearnAboutSenderIdentification Thanks Dave and Andrii. Per https://lwn.net/Articles/907056/, the API that you mentioned "provides single-user-space-producer / single-kernel-consumer semantics over a ring buffer." But this is not the desired behavior for our case. We want both bpf programs in kernel mode and user application to be able to write to the same ring buffer, which can be consumed by a (potentially different) user application. Assuming no such API exists, do you see any strong reason against writing such an API? If not, we would like to implement one in https://github.com/microsoft/ebpf-for-windows and eventually provide a Linux implementation as well. Thanks, Shankar শংকর শীল ________________________________________ From: dthaler1968=mailto:40googlemail.com@xxxxxxxxxxxxxx <dthaler1968=mailto:40googlemail.com@xxxxxxxxxxxxxx> Sent: Tuesday, May 28, 2024 10:42 AM To: 'Andrii Nakryiko' <mailto:andrii.nakryiko@xxxxxxxxx>; Shankar Seal <mailto:Shankar.Seal@xxxxxxxxxxxxx> Cc: mailto:bpf@xxxxxxxx <mailto:bpf@xxxxxxxx>; mailto:bpf@xxxxxxxxxxxxxxx <mailto:bpf@xxxxxxxxxxxxxxx> Subject: [EXTERNAL] RE: [Bpf] Re: Writing into a ring buffer map from user space [You don't often get email from dthaler1968=mailto:40googlemail.com@xxxxxxxxxxxxxx. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Andrii Nakryiko <mailto:andrii.nakryiko@xxxxxxxxx> wrote: > On Tue, May 28, 2024 at 9:32 AM Shankar Seal > <Shankar.Seal=mailto:40microsoft.com@xxxxxxxxxxxxxx> wrote: > > > > Adding mailto:bpf@xxxxxxxxxxxxxxx > > > > A common use case of an BPF ring buffer map to use as a queue of > > events generated by BPF programs that can be read in-order by user > > space applications. I have a scenario requirement for a user space > > application to write into a ring buffer (or similar) map, such that > > events by BPF programs in kernel and user space applications are > > interleaved in the order they were generated, that can be consumed by > > another user space application > > > > I would like to implement this new feature in the > https://github.com/microsoft/ebpf-for-windows project. But before I go ahead with > the implementation, I wanted to check if there is any way to accomplish this in > Linux today? If not, is there any reason why this should not be done? > > Yes, there is. See user_ring_buffer ([0], [1]). > > [0] > https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/prog_tests/ > user_ringbuf.c > [1] > https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/user_ > ringbuf_success.c Both of those links go to GPL code so I suspect Shankar cannot use those links. I think the answer is that BPF_MAP_TYPE_USER_RINGBUF is defined for this purpose and Shankar can read https://lwn.net/Articles/907056/ Thanks, Dave
