On Fri, Nov 9, 2018 at 11:41 AM Noah Watkins <nwatkins@xxxxxxxxxx> wrote:
>
> On Fri, Nov 9, 2018 at 1:46 AM Erwan Velu <evelu@xxxxxxxxxx> wrote:
> >
> > - I was wondering if you generated this schema manually or if you
> > created a tool to make it ?
>
> I created them manually
>
> > - I also noticed that some structures I see on the cli side are feed on
> > the fly and depending on the state of the cluster, so items pop in/out.
> > Does your approach cover that ? Are we sure we don't forgot any
> > field/value ?
>
> I've never encountered any output that the JSON schema standard can't
> handle, even when items pop in/out dynamically at runtime, or when
> items have different value types. That doesn't mean it it's easy to
> use instances of those schemas, which seems like what you are seeing
> with the example below. As for missing field/values, there is that
> chance. But that is why it is useful to build intermediate structures
> that are used for the sole purpose of representing the output. That
> should make it easy "guarantee" the output schema.
>
> > - There is already some implementation which are almost impossible to
> > map to a data structure.
> >
> > Services struct {
> > RbdMirror struct {
> > Daemons struct {
> > Num4467 struct {
> >
> > That Num4467 structure is random while it should be an array of a known
> > type.
>
> I'm a little confused about what you mean by its random, but should be
> an array. I'm guessing you mean that that struct corresponds to an
> item in a json serialization that changes. The schema handles that,
> but in this case, language support for dynamically inspecting the
> contents is useful. I'm guessing Go doesn't make that easy / possible?
>
> Perhaps..
>
> >
> > There is also examples where the key is a hostname.
> >
> > Rgw struct {
> > Daemons struct {
> > Summary string `json:"summary"`
> > CephRgw0 struct {
> >
> > Some tooling like ceph-nano or ceph-ansible workaround this, but I'm
> > also facing this.
> >
> > It would be nice if we can avoid this by having only pre-defined data
> > structures. I'm pretty sure if your PR goes on this item you will face a
> > show stopper with it as we can't validate this as a schema.
> >
> > Changing this will break the output and any users. But I do think we'll
> > have to make it at some point for the sanity of the output.
>
> I'm having a problem mapping these structs to the corresponding
> sources in Ceph. Could you maybe paste some of the JSON serializations
> or point to a CLI and how to generate the challenging JSON?
>
> Moving forward, I'm wondering if it would make sense to think about
> how to evolve the existing interfaces to provide more stability.
> Something like `-f json-stable` that has very strict standards and
> then eventually replacing existing `-f json`.
I really don't see anything like this being feasible, for the reasons
that John already discussed. When we change underlying behavior, we're
going to need to change the json structures we output.
That doesn't mean we can't make it easier for tools to reference the
data structures they do understand, and that are going to be reliably
available (such as some kind of instance ID). But any tool for looking
at Ceph behavior either needs to be tailored to a specific Ceph
version, or be able to handle receiving data it doesn't understand
without going crazy.
If for instance the fact that RGW instances are "indexed" by a GUID,
maybe we could output multiple "RGWInstance" json structs which have a
GUID member in addition to the struct they currently output. Would
that be easier to parse? My recollection is that sort of thing would
be an easy change to make in the source. Similarly, I remember a
previous conversation led us to try and output fields even when they
don't actually provide data to human users (because they're empty or
similar), but I don't know how consistent we've been about it.
But it might be necessary to redo some of how the Formatter stuff we
use works as well — look at something like Monitor::get_mon_status for
an example of how it works; we use a pretty naive formula for turning
those JSON structs into text output when that's what the user actually
wants and we need to avoid making the CLI output uglier, in addition
to making the JSON easier to work with.
-Greg
