On Wednesday, November 27, 2024 11:51:57 AM EST Gregory Farnum wrote: > On Wed, Nov 27, 2024 at 8:14 AM John Mulligan <phlogistonjohn@xxxxxxxxxxxxx> > wrote: > > On Wednesday, November 27, 2024 6:48:03 AM EST Neeraj Pratap Singh wrote: > > > Hey everyone, > > > > > > Recently, we came across a ticket which mentions about xml format not > > > working properly for some of ceph commands. > > > > > > After looking into the code, I found that we are not following any > > > > standard > > > > > to use the formats for the ceph commands across manager modules, and > > > especially xml format is highly ignored. Though xml and xml-pretty is > > > > being > > > > > accepted by 'src/ceph.in' , the ceph's command management tool. > > > > > > Apart from this, each and every module have their own way of using the > > > formats, some have hardcoded for json, plain etc. and rest in other way. > > > > > > I too encountered a file named `object_format.py` which resides in > > > `src/pybind/mgr/` but this also had been used by a few modules > > > (mgr/nfs). > > > > > > Thoughts? > > > > Hi Neeraj, you're right that every ceph mgr module can do things > > independently > > with the format value. The C++ mgr modules might be a tad bit more > > consistent > > with XML handling but I don't know for sure. Certainly, many of the python > > modules are probably not handling xml at all. The `ceph` command line > > tool > > can send various format strings to the mgr module backend but the backend > > is > > free to ignore them. > > > > The object_format.py code is currently in use by the nfs module and the > > smb > > module (in main). We have had discussions of moving the cephadm module to > > use > > it with some frequency but we have not done so yet. I'd love to see more > > modules adapt the object formatter approach. It is designed to try and > > separate concerns so that most python code in the mgr can be written in a > > typical python style and only a very limited set of functions (typically > > in > > the module class) need to be aware it's part of the mgr. However, I have > > not > > had the time or energy to convert "random" mgr module so it's up to the > > mgr > > module maintainers to adopt the object formatter style if desired. > > (Also, I'm sure there are cases it doesn't cover - I am happy to discuss > > design tweaks and features if you'd like to use it but it's not covering > > your > > use cases) > > > > What are you looking at in terms of XML support? With XML vs JSON/YAML > > it's > > much more typical to have a pre-defined xml schema versus the free use of > > lists > > and maps. Therefore most of our objects can be automatically mapped to > > JSON/ > > YAML without much additional work. > > We could come up with our own schema or adopt something out there... or we > > could create some sort of JSON-but-in-XML, but I am unclear how much value > > that would be. > > I'm also curious if the existing JSON/YAML support is not covering a use > > case > > you have? > > This isn't so much something Neeraj is having trouble with. The problem is: > * Our CLI tools and documentation suggest that you can choose to get > responses in plain text, JSON, or XML formats. > * Our C++ implementations use a Formatter class that can output each of > those (and there is some special handling for "plain text" v "programmatic" > in places) > * Our manager's Python implementations are a random grab bag and pretty > much all support only plain text and JSON.[1] > > Our downstream QE filed a bug against mgr/volumes for not outputting XML > when it was expected, but this is going to be true of just about every mgr > module's commands if anybody gets around to running them. > > So from my perspective we need to make a choice. Either > 1) We stop supporting XML, since we...don't actually support it > 2) We start supporting XML robustly in the mgr. > > (2) probably means adopting something like our C++ Formatter infrastructure > (presumably via object_format.py?) to make sure we're consistent about it. > I'm down with this plan but we need to agree on it as a community. (1) has > the advantage of not requiring work from every mgr component, and since > lack of XML support hasn't come up before it doesn't seem likely that > people are actually using it? > -Greg > [1]: This has also led to bugs, where one output schema provides different > data than the other due to calling different variables! Sounds good. I'm happy to help work with folks on the object_format.py code and would love to see it more widely adopted (assuming it helps). As for the question about what to do if xml is requested when the backend does not really support it, I have a slight preference for rejecting it with an error, but I would be ok with supporting some sort of generic xml too. Giving something other than what was requested is pretty bad and I 100% support not doing that! One other thing we might consider would be to extend the json returned by get_command_descriptions to include an (optional?) list of supported format types. then the ceph command line tool could "know" what formats a command expects to support. _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
