Re: [RFC] Introducing yamldt, a yaml to dtb compiler

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On Mon, Jul 31, 2017 at 8:11 AM, David Gibson
<david@xxxxxxxxxxxxxxxxxxxxx> wrote:
> On Fri, Jul 28, 2017 at 10:07:10AM -0500, Rob Herring wrote:
>> On Fri, Jul 28, 2017 at 7:23 AM, Pantelis Antoniou
>> <pantelis.antoniou@xxxxxxxxxxxx> wrote:
>> > Hi Rob,
>> >
>> > On Thu, 2017-07-27 at 21:12 -0500, Rob Herring wrote:
>> >> On Thu, Jul 27, 2017 at 7:51 PM, Tom Rini <trini@xxxxxxxxxxxx> wrote:
>> >> > On Thu, Jul 27, 2017 at 06:00:00PM -0500, Rob Herring wrote:
>> >> >> On Thu, Jul 27, 2017 at 4:46 PM, Pantelis Antoniou
>> >> >> <pantelis.antoniou@xxxxxxxxxxxx> wrote:
>> >> >> > Hi Frank,
>> >> >> >
>> >> >> > On Thu, 2017-07-27 at 13:22 -0700, Frank Rowand wrote:
>> >> >> >> Hi Pantelis,
>> >> >> >>
>> >> >> >> Keep in mind one of the reasons Linus says he is very direct is to
>> >> >> >> avoid leading a developer on, so that they don't waste a lot of time
>> >> >> >> trying to resolve the maintainer's issues instead of realizing that
>> >> >> >> the maintainer is saying "no". Please read my current answer as being
>> >> >> >> "no, not likely to ever be accepted", not "no, not in the current form".
>> >> >> >>
>> >> >> >> My first reaction is: no, this is not a good idea for the Linux kernel.
>> >> >> >>
>> >> >> >
>> >> >> > This has nothing to do with the kernel. It spits out valid DTBs that the
>> >> >> > kernel (or anything else) may use.
>> >> >>
>> >> >> Let me rephrase Frank's statement: this is not a good idea for the
>> >> >> main repository of dts files.
>> >> >>
>> >> >> But sure, DTS is already not the only source of DTBs. It comes from
>> >> >> firmware on Power systems.
>> >> >
>> >> > Yes, but unless they're generated from something other than a (at the
>> >> > time) normal DTS, that's not a good example, IMHO.
>> >>
>> >> They aren't. I'm talking about IBM systems. The firmware has its own
>> >> representation and flattens that to a DTB is how I understand it.
>> >>
>> >> >> If you want to create and maintain your own
>> >> >> source format, then that is perfectly fine. But based on the current
>> >> >> understanding, I'm not seeing a reason we'd convert DTS files to YAML.
>> >> >
>> >> > Can I propose one?  To borrow a phrase, Validation, Validation,
>> >> > Validation.  Let me point to fe496e23b748 in the kernel for a moment.  I
>> >> > found that as part of helping a new engineer come up to speed on doing
>> >> > device tree work.  What I found was a case where:
>> >> > - The binding doc gives one value for compatible as the required value.
>> >> > - The code accepts only a single, different value.
>> >> > - A few in-kernel dts files have different still values.
>> >> >
>> >> > If the common dts source file was in yaml, binding docs would be written
>> >> > so that we could use them as validation and hey, the above wouldn't ever
>> >> > have happened.  And I'm sure this is not the only example that's in-tree
>> >> > right now.  These kind of problems create an artificially high barrier
>> >> > to entry in a rather important area of the kernel (you can't trust the
>> >> > docs, you have to check around the code too, and of course the code
>> >> > might have moved since the docs were written).
>> >>
>> >> I'm all for validation, but the binding doc or schema and files that
>> >> describe platforms (aka DTS files) are not the same thing. The schema
>> >> is what are the constraints for a binding. Maybe some bindings are
>> >> fixed where there's only one valid binding implementation, but that's
>> >> the easy case (we could use DTS for that). I'll take YAML for binding
>> >> docs yesterday. Believe me, I'm tired of reviewing free form binding
>> >> docs. If that's where you want to go, reply to my reply that went
>> >> unanswered on Matt Porter's YAML proposal from 2 years ago (or maybe 3
>> >> now). I had the whole binding doc tree converted over to an initial
>> >> YAML schema. We just need to agree on the schema. Or we can keep
>> >> waiting for Grant to publish what he started on...
>> >>
>> >
>> > The way I see it there's a validation hierarchy.
>> >
>> > There are the bindings that describe the schema of the resulting source
>> > files. The bindings must be validated against a binding schema.
>> >
>> > For the source files, at first they must be valid against the core
>> > language (i.e. DTS or DT YAML variant) schema.
>> >
>> > Next for each node that a binding exists in a valid format, it must be
>> > validated against it. I.e. if an interrupt property exist it must point
>> > to valid interrupt node etc.
>> >
>> > Up next a number of per-platform/configuration validation passes.
>> > I.e. for a complete source file which is using a specific SoC family
>> > i.e. "ti,am33xx" the pass may verify that for the given peripherals
>> > their configuration is correct, i.e. that the interrupt numbers for a
>> > given peripheral are the correct ones for the target board etc.
>> > This may be possible by having a golden master configuration when those
>> > number can be retrieved and compared against.
>> >
>> > Finally you could have a per-application/vendor/end-user final rule
>> > check, i.e. the regulators may be configured in a manner that the power
>> > consumption is under some specified threshold, etc. This is something
>> > that is completely out of the kernel scope, but may have have to
>> > vendors.
>> >
>> > Why don't you share what you've been working on and see what we can do
>> > using it as a base?
>>
>> I did. 2 years ago:
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git/log/?h=dt-yaml-v2
>>
>> It's very rough, but I was at the point of wanting feedback on the
>> schema format. Only the crickets gave me any.
>>
>> It doesn't validate anything, but is purely binding docs mass
>> converted to YAML using DTS files as input.
>
> Efforts on schemas have started and petered out several times :(.
>
> I think the fundamental problem is that there just isn't critical mass
> of people with the time to work on it.  Lots of people want better
> validation, but not enough to put significant time and effort into
> it.  I hope someone proves me wrong about that.

I would say part of the problem is the validation plans are always
just too grand. We need to start small. Just having a machine
parseable documentation format alone would be a win. The validation
can come later IMO. It's going to come later anyway if we do nothing.

> Can I suggest (again) that one approach might be to add more pieces to
> dtc's "checks" system to at least look for the more common errors.
> It's not nearly a complete solution, but it gets you something with
> much less difficulty than defining a whole schema system.  Some
> rudimentary checking of unit addresses has been added relatively
> recently, but not a lot else in the way of semantic checks.

Yes, I obviously agree. And at least for me, it's in a language I
already know. I've thought of a few more checks to add in the course
of this thread. Perhaps we should start a todo list if you have any
specific ideas. I've focused on things I repeatedly catch in binding
reviews (if only I could have a check for needing more specific
compatible strings :) ). Where we hit limits with the checks is when
we need specific compatible(s) to key checks on. This is any binding
that lacks a "class" property like gpio-controller or
interrupt-controller. For example I2C or SPI controllers and buses.
Certainly every common binding with a #*-cells property could have
some level of checks.

Rob
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Device Tree Compilter]     [Device Tree Spec]     [Linux Driver Backports]     [Video for Linux]     [Linux USB Devel]     [Linux PCI Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [XFree86]     [Yosemite Backpacking]


  Powered by Linux