On 21/02/22 11:41PM, Toke Høiland-Jørgensen wrote: > "Brian G. Merrell" <brian.g.merrell@xxxxxxxxx> writes: > > On 21/02/18 05:20PM, Toke Høiland-Jørgensen wrote: > >> No, I think the main difference is that in the model you described, > >> you're assuming that your orchestration system would install the XDP > >> program on behalf of the application as well as launch the userspace > >> bits. > > > > Yes, that's right. This is the model we are implementing. > > > >> Whereas I'm assuming that an application that uses XDP will start > >> in userspace (launched by systemd, most likely), and will then load its > >> own XDP program after possibly doing some initialisation first (e.g., > >> pre-populating maps, that sort of thing). > >> > >> From what I've understood from what you explained about your setup, your > >> model could work with both models as well; so why are you assuming that > >> applications won't want to install their own XDP programs? :) > > > > I would just say that in our organizations network and administration > > environment, we ideally want a centralized orchestration tooling and > > control plane that is used for all XDP (and tc) programs running on our > > machines with our model described above. > > Right, sure, I'm not disputing this model is useful as well, I'm just > wondering about how you envision the details working. Say your > orchestration system installs an XDP program on behalf of an application > and then launches the userspace component (assuming one exists). How is > that userspace program supposed to obtain a file descriptor for the > map(s) used by the XDP program in order to communicate with it? OK, so this part is admittedly a little hand-wavy and a work in progress. We're literally working on design and proof of concepts right now, but this is basically what we're envisioning: 1. Orchestration tool gets all its JSON config data, which includes remote paths for BPF programs and any respective userspace programs. 2. Orchestration tool downloads BPF programs and loads them (using Go libxdp when it's available). Then (and this is where I'm going to start waving my hands) the orchestrator will need to gather any necessary map names/ids/fds information to be send to the userspace program. I'm just not exactly sure how easy/hard/possible this part is. 3. We start the userspace programs as separate processes and communicate with them via RPC (there's a nice Go plugin system for this[1]). Each userspace program implements an interface and we communicate the map info (among other things) over RPC to the userspace program when it starts. I'm going to continue researching and fleshing out the details, but are there any obvious problems with this approach? A backup plan is to have the userspace programs do the loading of the BPF program, but it's not obvious to me how that would be easier to obtain the file descriptor for the map(s) vs. having the orchestrator figure it out and send it to the userspace process. If it works out that the orchestrator can load the BPF programs on behalf of the userspace programs, then I think the primary benefit is that the developer of the userspace program doesn't need to follow some boilerplate to load the appropriate way--we've done all that for them. It seems nice that the orchestrator could be the one interface with libxdp (for the XDP case) without every userspace program needing to doing it's own adding/removing (and thus dispatcher swapping), though I would guess that's not really a problem at all. I feel like I've gone out of the scope of libxdp in this e-mail, but you did ask :) And I do appreciate any feedback or raising of red flags. Thanks, Brian [1] https://github.com/hashicorp/go-plugin
|