Erwan,
You can't "extract" the complete JSON schema from each command. A JSON
schema can have keys like: required, uniqueitems, minitems, oneof,
additionalProperties, and minimum that precisely defines the output.
We could require all all json output to have programmer generated
schemas in the "doc" directory. During run-standalone.sh we can
validate the schemas or in teuthology if we install the schemas. It is
still possible that a mistake in the schema won't be detected if the
test doesn't happen to generate conflicting JSON.
I talked with Noah about this, pointing to my current mechanism for
scrub results output. As the scrub output has evolved I made
corresponding changes to the schemas. So a given schema is only valid
for the version of the code it came from.
See "doc/rados/command/" for 2 schemas for list-inconsistent-* JSON
formatted output. Output is validated using jsonschema by the tests
qa/standalone/scrub/osd-scrub-repair.sh and
qa/standalone/scrub/osd-scrub-snaps.sh. But that only happens when run
locally in the build tree when using run-standalone.sh
David
On 11/7/18 7:48 AM, John Spray wrote:
On Wed, Nov 7, 2018 at 3:11 PM Erwan Velu <evelu@xxxxxxxxxx> wrote:
Hi list,
I'm working on a tool that reads the json output of several ceph commands.
To ease the parsing, I've been choosing the JSON format which guarantee
a parseable output.
I'm using the Unmarshall feature of golang to map this output in an
internal data structure so every member of this JSON output is easily
reachable from the code.
That works pretty well except that I have to "anticipate" what will the
be members and their types.
To do that, I've been transforming the sample output of my Ceph cluster
(luminous) into a data struct with https://mholt.github.io/json-to-go/
That works fine unless that I would be a complete output to get every
possible combination of the json output.
So instead of reversing the json output, and that per version of Ceph as
versions can change the format, how can I extract the complete JSON
schema from each command ?
The short answer is that you can't, because there aren't defined
schemas. It's not straightforward to define them, because the output
of commands is coming from so many different places in the code
(pretty much anything with a ::dump method might show up in a CLI at
some point).
However, Noah has looked at this (see thread "exported data schema
validation") with a view to defining schemas for a subset of key data
structures. Note that even when some schemas exist, that won't
necessarily make it safe to rely on them across versions --
introducing versioning to the output of our CLIs would be a whole
other question (and the answer would probably be to create an API
rather than bolting that onto the CLI).
John
Erwan,
