"Brian G. Merrell" <brian.g.merrell@xxxxxxxxx> writes: >> Yeah, so what I was thinking was whether it would be useful to, for >> instance, define a "bundle" format that contains the config data that >> libxdp will understand. Could just be a JSON schema containing keys for >> priority, chain call actions and a filename, so you can just point >> xdp_program__open_file() at that and it will do the rest. >> >> However, I'm still not quite sure I'm convinced that this will be >> generally useful. As you say, you can just as well just set the values >> programmatically after loading the file, and I suspect that different >> deployments will end up having too much custom stuff around this that >> they'll bother using such a facility anyway. WDYT? > > Yeah, my hunch is that different deployments will have custom config > data and then getting the data into libxdp format will just be another > data conversion step. I could be wrong. > > If it is decided to create a data format, then the JSON schema you > described is pretty much exactly what I had in mind, too. For whatever > that is worth! Right, I'll mull this over some more but let's defer it for now. >> > I know in a previous e-mail you mentioned having a config file with >> > priority overrides. That's just not a use case that our team would want >> > to use. And, my opinion would be that the program using libxdp should be >> > the one to implement that sort of policy; it keeps libxdp more simple >> > without needing to worry about parsing config files (and handling config >> > version changes in the code and the spec). For example, xdp-loader could >> > have a config file with priority overrides and people could use that >> > code if they wanted to do something similar. >> >> Yeah, that's totally what would make sense for your deployment case. The >> design where libxdp reads a config file comes from my distro >> perspective: We want to build a system whereby different applications >> can each incorporate XDP functionality and co-exist; and the goal is to >> make libxdp the synchronisation point between them. I.e., we can say to >> application authors "just use libxdp when writing your application and >> it'll work", while at the same time empowering sysadmins to change the >> default application ordering. > > Ah, yes, that perspective helps a lot; I understand much better why you > would want a centralized configuration file. My mind is actually kind of > melting now thinking about all the use cases. > > Is the idea that the configuration file would have the final word? For > example, if there are multiple applications using libxdp (and possibly > even programatically overriding their priorities), could a sysadmin then > write a config file to dictate what they actually want? That would made > sense to me. If that's the goal, then I guess I would take back my > opinion about that policy not belonging in libxdp. I think application > writers would be less likely to use the configuration format (for > reasons we discussed above), but it does seem like a necessary mechanism > for sysadmins to orchestrate multiple applications that don't know about > each other. Yeah, that would be the idea. > For my team's use case, we have the luxury of being the One > Application to rule all BPF programs. Yup, obviously being in full control of the system gives you some options we don't quite have as a distro. But see below - I still think there are some similarities. >> By having that configuration be part of the library, applications can be >> free to use either the command-line loader or include the loading into >> their own user-space binary. >> >> But since you are (notionally) both the application developer and system >> owner, that is less of a concern for you as you control the whole stack. >> >> > Hopefully I'm even making sense, but like I said, I don't have strong >> > feelings about the format, as long as we are able to achieve our >> > required use case of programmatically setting the run configuration >> > values from a libxdp user program. >> >> Sure, that you can certainly achieve with implementing what libxdp >> includes today. I'm just trying to make sure we explore any >> opportunities for standardising something useful so others can benefit >> from it as well; so I hope you'll forgive my probing :) > > 8< snip > >> > We explicitly do not want defaults set by program authors. We want that >> > policy to be completely in the hands of the orchestration environment. >> >> Right, OK. How does the admin configuring the orchestration system >> figure out which order to run programs in, BTW? Is this obvious from the >> nature of the programs, or do you document it out of band somewhere, or >> something like that? > > We're a pretty huge organization... lots of DCs, public cloud, private > cloud, different kernel versions, sister companies, hundreds of > applications, etc. We want anyone to be able to write cool BPF > programs and userspace applications without needing awareness of > what's running before or after or if that order might change in the > future. I'm sure the desired order will be more obvious for some > programs than others, but we have administrators that can analyze the > BPF programs, compose multiple BPF programs together, and order and > reorder them. We have a team of people that can work with teams to > resolve any interdependencies if necessary. > > As an example, we've done something similar for HTTP ingress and > egress Lua plugins in the past. We have dozens of teams that write Lua > code to do custom L7 things with HTTP requests and responses, and then > we have a UI where admins/ops folks can literally drag and drog the > plugins into the desired order. We wouldn't want teams making > assumptions about what order plugins should run in, either. See, so this is the part that's actually analogous to what we want to do as a distro. Except the people writing the cool BPF programs are different software vendors and open source projects, not different divisions within the same sprawling org. But in a sense the situation is quite similar. So thinking a bit more about the difference between your orchestration system and the model I've been working from, I think the biggest difference is not that you are assuming control of a system with an orchestration system. In a sense a distro is also an orchestration system bringing together different software from different sources. No, I think the main difference is that in the model you described, you're assuming that your orchestration system would install the XDP program on behalf of the application as well as launch the userspace bits. Whereas I'm assuming that an application that uses XDP will start in userspace (launched by systemd, most likely), and will then load its own XDP program after possibly doing some initialisation first (e.g., pre-populating maps, that sort of thing). >From what I've understood from what you explained about your setup, your model could work with both models as well; so why are you assuming that applications won't want to install their own XDP programs? :) -Toke
|