On Fri, May 4, 2012 at 2:46 PM, Konrad Eisele <eiselekd@xxxxxxxxx> wrote:
>
> Nice to hear this.
> When I talk about macro dependency I mean not only the
> macro expansion trace. I mean:
> 1. The #if (and #include) nestings (with dependencies
> pointing to the macros used in the proprocessor line)
> 2. The macro expansion trace
> 3. The connection 1+2 into the AST.
> Your macro_expand() hook addresses (2) only, but I cant
> see how all the extra context for each token can be saved
> in that sheme.
That is much better. There is two separate problem here.
One is keep track of all the macro expand history so you can
trace back the token back to the original form. I believe my
description of the macro_expand hook should take care of that.
Now how to connect the AST tree with those information is a
very good question. Notice the symbol->aux pointer? That is
the place to attach extra context or back end related data
to symbols.
Because each symbol has "pos" and "endpos". If the symbol
is expand from macro, using the previous scheme, the pos
should point to a line in the "<pre-processor>" stream.
However, if the macro expand is happen between "pos" and
"endpos", you will not able to access the token that contain
the macro expand "pos" easily.
For that, we could, just thinking it out loud, add a parser
hook for declares when a symbol is complete building.
That would a very small and straight forward change.
If the hook is not NULL, the call back function will be call
with the symbol that just get defined, and the start and end
token of that symbol.
So your dependence program just need to register the
symbol parsing hook. In side the call back function, walk
the token from start to end. Look up macro expand information
is needed. Build up the dependency struct and store that in
symbol->aux.
BTW, unrelated to this patch, I can see other program might
be able to use the same parser hook to perform source code
transformations as well.
Make sense? In this way, you don't even need the hash
table to attach a context into the token. You can get it directly
from symbol->aux.
> In my patch I have modeled (2) using 2 structs:
> struct macro_expansion {
> int nargs;
> struct symbol *sym;
> struct token *m;
> struct arg args[0];
> };
> struct tok_macro_dep {
> struct macro_expansion *m;
> unsigned int argi;
> unsigned int isbody : 1;
> unsigned int visited : 1;
> };
> Each token from a macro expansion gets tagged with
> tok_macro_dep. If it is an macro argument, <argi> shows the
> index, if it is from the macro body <isbody> is 1.
> Now, I didnt already think about special cases like
> token concaternation, even more data is needed to
> model this. Also when an macro argument is again used as an
> macro argument inside the body expansion, then I kindof
> loose the chain: I would also need a "token *dup_of" pointer
> to point to the original token that the token is a copy
> of (when arguments are created...) etc.
>
> I have read your macro_expand() hook idea, however
> when I understand it right you want to reuse position.stream and
> position.line as a kind of pointer (to save the extra 4 bytes).
> (Your goal is to minimize codebase change, however I wonder
> weather you dont change semantic of struct position and then
> need to change the code that uses struct position anyway...)
Nope, because the position.stream change is only happen on
your dependency analyse program. It is the dependency program
register the hook to it. This behaviour is private to the dependency
analyse program. Other program that use sparse library don't see
it at all, because they don't register macro_expand hooks to perform
those stream manipulations. It will receive the exact AST as before.
> Maybe it is possible like this...I doubt it, where should
> all the extra context, that each token has, be saved and
> extracted from? using that sheme...
Two places, one is symbol->aux. Also the macro_expand
can be lookup by pos->line. That will index into the macro_expand
array which store the context.
Having this two should be enough to put the exact same
dependency result as you are doing right now.
> Maybe it is possible but I dont want to have as a design
> goal to save 4 bytes (I'd use the void *custom sheme to
> save all my extra data, also the pointers to tokens to
> "sit around") and adujust everything else to
> that. The consequence is that the code-complexity would
> grow on the other end.
It is not only about saving 4 bytes. It is about other program
don't have to suck in the full token struct if they don't need to.
It is about re-usable macro hooks and parser hooks that
external program can do more fancy stuff like source code transformations
without impacting the other user of the sparse lib.
> Here is my compromise then:
> Keep the orignial "pos". But still grant me for
> each struct a "void *custom" pointer that I can use
> to store extradata i.e. pointer to token.
symbol->aux.
Chris
--
To unsubscribe from this list: send the line "unsubscribe linux-sparse" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
