On Mon, Sep 28, 2015 at 10:36:59AM +0100, Graham Whaley wrote:
> On Tue, 2015-09-22 at 16:03 -0300, Danilo Cesar Lemes de Paula wrote:
> > On 09/22/2015 07:22 AM, Graham Whaley wrote:
> > > On Wed, Sep 02, 2015 at 02:50:52PM +0100, Graham Whaley wrote:
> > > > > (RFC/test - not for merging)
> > > > > The below is a test of moving the large HTML KMS properties
> > > > > table
> > > > > out
> > > > > to markdown style in the appropriate files.
> > > > > In the test we only use the first few rows of the existing KMS
> > > > > table
> > > > > an example.
> > > > > We use a fixed width table as the other styles of table
> > > > > supported
> > > > > by
> > > > > pandoc markdown do not support multi-column cells.
> > > > >
> > > > > The test shows a couple of issues:
> > > > > 1) double quote characters are being expanded in the fixed
> > > > > width
> > > > > table
> > > > > which then breaks the table alignment and leaves html style
> > > > > <quote> tags
> > > > > in the text
> > >
> > > Further to this item:
> > > Before I continue working on splitting the tables and converting
> > > to
> > > markdown (which btw Daniel does look feasible so far...), I thought
> > > we
> > > should understand what was going on with the markdown table quote
> > > breakage.
> > > I think I know what is happenig.
> > > The problem revolves around highlight expansion in the kenrel-doc
> > > script. In the output_highlight function we can see that first the
> > > script does highlight expansion (with the eval of @dohighlight),
> > > and
> > > then it invokes the markdown processing.
> > > Done that way around what happens is:
> > > - @dohighlight expands any "X" to html/xml quote tag sequences
> > > - which can push the text beyond the table column widths,
> > > - and then the markdown tries to split the text over columns, and
> > > manages to put the break in the middle of a tag, and thus breaks
> > > the
> > > tag formatting (see below)
> > >
> > > At least that is what I think is happening.
> > > I had thought maybe we could swap the markdown and highlight
> > > processing
> > > order, but then that presents a different issue - the markdown
> > > table
> > > contains metadata with quoted items (such as cols="5"), which the
> > > highligh processing then expands into tags, and hence breaks the
> > > table
> > > format metadata.
> > >
> > > As an example, using the following table !Pinclude'd into drm.tmpl:
> > > /**
> > > * DOC: DRM generic
> > > *
> > > * : DRM generic properties
> > > *
> > > * +----------+--------+--------------------+---------+------------
> > > ----
> > > --------+
> > > * |Property |Type |Property Values |Object
> > > |Description/Restrictions|
> > > *
> > > +==========+========+====================+=========+===============
> > > ====
> > > =====+
> > > * |"rotation"|BITMASK |{ 0, "rotate-0" }, {|CRTC, |rotate
> > > -(degrees)
> > > rotates|
> > > * +----------+--------+--------------------+---------+------------
> > > ----
> > > --------+
> > > *
> > > */
> > >
> > > The post-highlight pre-markdown text captured as the $orig_context
> > > in
> > > the markdown_to_docbook function is:
> > >
> > > +----------+--------+--------------------+---------+---------------
> > > ----
> > > -----+
> > > > Property |Type |Property Values |Object
> > > |Description/Restrictions|
> > > +==========+========+====================+=========+===============
> > > ====
> > > =====+
> > > > <quote>rotation</quote>|BITMASK |{ 0, <quote>rotate-0</quote> },
> > > {|CRTC, |rotate-(degrees) rotates|
> > > +----------+--------+--------------------+---------+---------------
> > > ----
> > > -----+
> > >
> > > which when processed (and you can do this by hand with 'pandoc -
> > > -columns=80 -f markdown -t docbook ...' generates the broken tags
> > > example:
> > >
> > > ...
> > > <tbody>
> > > <row>
> > > <entry>
> > > <para>
> > > <quote>rota
> > > </para>
> > > </entry>
> > > <entry>
> > > <para>
> > > tion</quo
> > > </para>
> > > </entry>
> > > <entry>
> > > <para>
> > > te>|BITMASK |{ 0, <qu
> > > </para>
> > > </entry>
> > > <entry>
> > > <para>
> > > ote>rotate
> > > </para>
> > > </entry>
> > > <entry>
> > > <para>
> > > -0</quote> }, {|CRTC, |rotate-(degrees) rotates
> > > </para>
> > > ...
> > >
> > > where you can see the quote tag processing has gone horribly wrong.
> > >
> > > I believe we'll have the same problem for the other 'highlight'
> > > processed items from kern-doc as well, meaning:
> > > funcname()
> > > $ENVVAR
> > > &struct_name
> > > @parameter
> > > %CONST
> > >
> > > As the kern-doc processing has no knowledge of when it is about to
> > > process a markdown table I can't think of an obvious way around
> > > this.
> > > At present I think the implicit rule is 'no highlight/expansion
> > > items
> > > allowed in markdown tables', which means all those quoted strings
> > > for
> > > the DRM properties cannot currently be migrated to look like
> > > strings.
> > >
> > > Danilo, or anybody, any ideas?
> >
> > Your debugging is pretty precise. The "+=====+===" format is rigid. I
> > don't know if it's a bug in pandoc or something from the spec itself.
> >
> > May I suggest another format?
> >
> > > Property | Type |
> > > ------------|:-------:|
> > > "rotation" | BITMASK |
> >
> > This format looks to work pretty well even if "rotation" is replaced
> > by
> > some very long string (which seems to be the problem). I didn't test
> > it
> > with kernel-doc, but pandoc looks to work fine with it.
> >
> > I'm not sure if we can call it a proper fix, but it should allow you
> > to
> > keep the ball rolling.
> >
> > --
> > Danilo Cesar
>
> Hi all,
> I had a look at the 'pipe' table type - and the other table types
> available in pandoc_markdown. I've consolidated my thoughts into the
> following analysis. Note, this thread is not presently on the docs-list
> - we should pull that in as well if we think it may help.
>
> Let's say there are 5 table types we can use in pandoc_markdown, if we
> add in the use of the 'codeblock' type to the four formal
> pandoc_markdown tables (see table later...).
>
> My hypothesis' are:
>
> 1) We need to have a table format that supports multi-line rows, as
> some rows in the DRM table are way too big to be putting on a single
> line in the source file (for instance, the 'rotation' example I have
> been using totals somewhere in the region of 349 characters as a single
> line)
I think that requirement should probably be dropped, squeezing full-blown
semantics of properties into these tables just doesn't work really. For
complext stuff I except that the table just lists the propers with short
descriptions, and a bit of prose below explains the detailed
semantics/fallbacks/interactions. E.g. with color manager we'll have a
total of 5 properties, but need some _very_ detailed explanations of how
they work and interact. That won't even fit in 20 lines ;-)
> 2) We need to support the kernel-doc highlight types in the tables as:
> 2a) We need to use double quotes for strings in the DRM table at least
> 2b) And I think the other highlight types will be useful in the future
> 3) We need a table type that allows the highlight tags to be
> subsequently processed into the final document format (otherwise the
> highlight tag expansions get left as raw tags in the final text).
> 4) Because (2) 'expands' a line width when it is processing/tagging the
> highlights we need a table type that supports variable length lines,
> and is not based on fixed width column processing.
>
> If we summarise that in a table (ironically :-):
>
> tabletype | multiline | variable | processed | suitable?
> --------------------------------------------------------
> simple | No | No | Yes | No
> multi | Yes | No | Yes | No
> grid | Yes | No | Yes | No
> pipe | No | Yes | Yes | No
> codeblock | Yes | Yes | No | No
>
> What we are looking for is a Yes in the first three columns. And thus,
> presently it looks to me like none of the table formats are wholly
> suitable :-(
>
> I've still not thought of a way of tweaking the kernel-doc and pandoc
> processing to work around this either, as they are done as different
> passes/phases that neither has knowledge about the others
> requirements.
>
> As it stands, I'm failing to find a method to break out the DRM table
> into markdown tables that I believe works, fundamentally due to this
> 'incompatibility' between the kernel-doc and pandoc_markdown processing
> phases around the highlight processing.
Yeah, 1:1 translation won't work, but I think the current table for
properties doesn't work either. Pulling the details out of the table into
prose means we only need a Yes for "variable" and "processed", so I think
we should be covered. Of course if we could somehow resolve this conflict
between kerneldoc and pandoc that would be even better. One idea I have
(but it's a bit nuts) is to only build up a list of all the replacements
we'd like to do, then send things to pandoc and then apply them on the
output. That way we'll make sure that we don't accidentally replace some
of the markup pandoc generated. Would that be compatible with the ""
parsing issues?
-Daniel"
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@xxxxxxxxxxxxxxxxxxxxx
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
