"Brian G. Merrell" <brian.g.merrell@xxxxxxxxx> writes: > On 21/02/18 05:20PM, Toke Høiland-Jørgensen wrote: >> "Brian G. Merrell" <brian.g.merrell@xxxxxxxxx> writes: > >> >> > We explicitly do not want defaults set by program authors. We want that >> >> > policy to be completely in the hands of the orchestration environment. >> >> >> >> Right, OK. How does the admin configuring the orchestration system >> >> figure out which order to run programs in, BTW? Is this obvious from the >> >> nature of the programs, or do you document it out of band somewhere, or >> >> something like that? >> > >> > We're a pretty huge organization... lots of DCs, public cloud, private >> > cloud, different kernel versions, sister companies, hundreds of >> > applications, etc. We want anyone to be able to write cool BPF >> > programs and userspace applications without needing awareness of >> > what's running before or after or if that order might change in the >> > future. I'm sure the desired order will be more obvious for some >> > programs than others, but we have administrators that can analyze the >> > BPF programs, compose multiple BPF programs together, and order and >> > reorder them. We have a team of people that can work with teams to >> > resolve any interdependencies if necessary. >> > >> > As an example, we've done something similar for HTTP ingress and >> > egress Lua plugins in the past. We have dozens of teams that write Lua >> > code to do custom L7 things with HTTP requests and responses, and then >> > we have a UI where admins/ops folks can literally drag and drog the >> > plugins into the desired order. We wouldn't want teams making >> > assumptions about what order plugins should run in, either. >> >> >> See, so this is the part that's actually analogous to what we want to do >> as a distro. Except the people writing the cool BPF programs are >> different software vendors and open source projects, not different >> divisions within the same sprawling org. But in a sense the situation is >> quite similar. >> >> So thinking a bit more about the difference between your orchestration >> system and the model I've been working from, I think the biggest >> difference is not that you are assuming control of a system with an >> orchestration system. In a sense a distro is also an orchestration >> system bringing together different software from different sources. >> >> No, I think the main difference is that in the model you described, >> you're assuming that your orchestration system would install the XDP >> program on behalf of the application as well as launch the userspace >> bits. > > Yes, that's right. This is the model we are implementing. > >> Whereas I'm assuming that an application that uses XDP will start >> in userspace (launched by systemd, most likely), and will then load its >> own XDP program after possibly doing some initialisation first (e.g., >> pre-populating maps, that sort of thing). >> >> From what I've understood from what you explained about your setup, your >> model could work with both models as well; so why are you assuming that >> applications won't want to install their own XDP programs? :) > > I would just say that in our organizations network and administration > environment, we ideally want a centralized orchestration tooling and > control plane that is used for all XDP (and tc) programs running on our > machines with our model described above. Right, sure, I'm not disputing this model is useful as well, I'm just wondering about how you envision the details working. Say your orchestration system installs an XDP program on behalf of an application and then launches the userspace component (assuming one exists). How is that userspace program supposed to obtain a file descriptor for the map(s) used by the XDP program in order to communicate with it? > That said, I do see your point about the possibility of using some > other application that runs its own XDP programs, and then, yes, we > would definitely want some way to control the priority. Ideally, the > application would have its own configuration to set priorities, but I > do think the system configuration file is a good way to ensure that > the sysadmin does have the power to override if necessary. > > I think you're right that both models should be able to be used. > Thanks for the good discussion. Yeah, you too! Always good to have someone to bounce ideas off of :) -Toke
|