On Tue, 13 Nov 2018, John Spray wrote:
> On Tue, Nov 13, 2018 at 3:28 PM Erwan Velu <evelu@xxxxxxxxxx> wrote:
> >
> > Le 09/11/2018 à 22:48, Gregory Farnum a écrit :
> > > If for instance the fact that RGW instances are "indexed" by a GUID,
> > > maybe we could output multiple "RGWInstance" json structs which have a
> > > GUID member in addition to the struct they currently output. Would
> > > that be easier to parse? My recollection is that sort of thing would
> > > be an easy change to make in the source. Similarly, I remember a
> > > previous conversation led us to try and output fields even when they
> > > don't actually provide data to human users (because they're empty or
> > > similar), but I don't know how consistent we've been about it.
> >
> > I can share with you my expectations on this side.
> >
> > When you have a complex json output as ceph can generates like 10K chars
> > and various nested types, you need to map this output to an internal
> > representation of your tooling.
> >
> > The golang language but also many other offer function to load a json
> > into a structure type of a the language.
> >
> > In my case it looks like the following :
> >
> > I define a structure like :
> >
> > type OSD struct {
> > ID int `json:"id"`
> > Arch string `json:"arch"`
> > BackAddr string `json:"back_addr"`
> > BackIface string `json:"back_iface"`
> >
> > Hostname string `json:"hostname"`
> >
> > ....
> >
> > }
> >
> > and then I do :
> >
> > var osds []OSD
> > stdout, err := exec.Command("ceph", "osd", "metadata", "-f",
> > "json").Output()
> > if err == nil {
> > err = json.Unmarshal(stdout, &osds)
>
> Clearly if you're calling Unmarshal on an array of OSDs, then you're
> going to want the input in an array rather than a dictionary.
> However, what's to stop you just using a map of strings to OSDs in the
> golang code, and unmarshalling to that? I'd be kind of surprised if
> the golang JSON parser couldn't handle it.
>
> This question of arrays vs. dicts seems to be a distraction from the
> underlying issue that you're wanting a stable external API, which is
> not something that Ceph currently offers (but of course everyone would
> very much like to have). Defining a stable API would be great,
> although I wouldn't use today's CLI as the starting point.
+1
This direction does not feel like a good return on investment--lots of
work and minimal value. I liked Noah's original intent (well, as I
understood it at least): to create an automated way to call attention to
schema changes with minimal tooling. That's a relatively small investment
and simply makes it harder to change the CLI accidentally, without going
down the rabbit hole of overspecifying something that is not meant to be a
stable external interface.
sage
>
> John
>
> >
> > Once I've done that, I can assume :
> >
> > - the JSON was well formatted against that "model" aka the datastructure
> >
> >
> > - I now have a representation of OSDs in this language.
> >
> >
> >
> > So I can make a kind of
> >
> > for each osd in osds { if ("HP" in osd.hostname & "eth0" in BackIface)
> > .... }
> >
> >
> > That internal representation is very useful and makes my life much more
> > easier at developping 3rd party tooling using ceph.
> >
> >
> > The fact that some JSON keys have a random name, I cannot map them this
> > way as I can't anticipate the name of the Keys. That means, that If I
> > want to support the current output, I have to manually parse the whole
> > json and extract every field to map them into a custom data structure.
> > That's a lot of useless work while the unmarshal was invented to do so.
> >
> > So my expectation here is that we don't put unpredicatable strings as
> > keys in such json output but instead create an array where each item
> > have a name field that feature that value.
> >
> >
> > Please consider the other part of the json does it nicely like mons :
> >
> > "mons":[
> > {
> > "rank":0,
> > "name":"ceph-mon0",
> > "addr":"192.168.1.10:6789/0",
> > "public_addr":"192.168.1.10:6789/0"
> > },
> > {
> > "rank":1,
> > "name":"ceph-mon1",
> > "addr":"192.168.1.11:6789/0",
> > "public_addr":"192.168.1.11:6789/0"
> > },
> > {
> > "rank":2,
> > "name":"ceph-mon2",
> > "addr":"192.168.1.12:6789/0",
> > "public_addr":"192.168.1.12:6789/0"
> > }
> > ]
> > },
> >
> >
> > So I'd be cool if the servicemap can change its output from :
> >
> > "services":{
> > "rbd-mirror":{
> > "daemons":{
> > "summary":"",
> > "4467":{
> >
> >
> > to
> >
> > "services":{
> > "rbd-mirror":{
> > "daemons": [
> >
> > {"summary":"",
> > name: "4467"
> >
> > ...
> >
> > }
> >
> > ]
> >
>
>
