On Tue, 2009-10-27 at 00:11 +0800, Wu Zhangjin wrote:
> Hi,
>
> On Mon, 2009-10-26 at 11:13 -0400, Steven Rostedt wrote:
> [...]
> > > +
> > > + /* get the code at "ip" */
> > > + code = *(unsigned int *)ip;
> >
> > Probably want to put the above in an asm with exception handling.
> >
>
> Seems that exception handling in an asm is really "awful"(un-readable)
> and the above ip is what we have got from the ftrace_graph_caller, it
> should be okay. but if exception handling is necessary, I will send a
> new patch for the places(including the following one) which need it.
Yeah, and probably not as important in the mips world, as it is used
more with embedded devices than desktops. We must always take the
"paranoid" approach for tracing. At least in PPC and x86, we assume
everything is broken ;-) And we want to be as robust as possible. If
something goes wrong, we want to detect it ASAP and report it. And keep
the system from crashing.
At least with MIPS we don't need to worry about crashing Linus's
desktop. With is the #1 priority we have on x86 ... "Don't crash Linus's
desktop!".
If Linus sees a warning, he'll bitch at us. If we crash his box, and he
was to lose any information, he'll strip out our code!
>
> > > +
> > > + /* If we hit the "move s8(fp), sp" instruction before finding
> > > + * where the ra is stored, then this is a leaf function and it
> > > + * does not store the ra on the stack. */
> > > + if ((code & MOV_FP_SP) == MOV_FP_SP)
> > > + return parent_addr;
> > > + } while (((code & S_RA) != S_RA));
> >
> > Hmm, that condition also looks worrisome. Should we just always search
> > for s{d,w} R,X(sp)?
> >
> > Since there should only be stores of registers into the sp above the
> > jump to mcount. The break out loop is a check for move. I think it would
> > be safer to have the break out loop is a check for non storing of a
> > register into SP.
>
>
> Okay, let's look at this with -mlong-calls,
>
> leaf function:
>
> ffffffff80243cd8 <oops_may_print>:
> ffffffff80243cd8: 67bdfff0 daddiu sp,sp,-16
> ffffffff80243cdc: ffbe0008 sd s8,8(sp)
> ffffffff80243ce0: 03a0f02d move s8,sp
> ffffffff80243ce4: 3c038021 lui v1,0x8021
> ffffffff80243ce8: 646316b0 daddiu v1,v1,5808
> ffffffff80243cec: 03e0082d move at,ra
> ffffffff80243cf0: 0060f809 jalr v1
> ffffffff80243cf4: 00020021 nop
>
> non-leaf function:
>
> ffffffff802414c0 <copy_process>:
> ffffffff802414c0: 67bdff40 daddiu sp,sp,-192
> ffffffff802414c4: ffbe00b0 sd s8,176(sp)
> ffffffff802414c8: 03a0f02d move s8,sp
> ffffffff802414cc: ffbf00b8 sd ra,184(sp)
> ffffffff802414d0: ffb700a8 sd s7,168(sp)
> ffffffff802414d4: ffb600a0 sd s6,160(sp)
> ffffffff802414d8: ffb50098 sd s5,152(sp)
> ffffffff802414dc: ffb40090 sd s4,144(sp)
> ffffffff802414e0: ffb30088 sd s3,136(sp)
> ffffffff802414e4: ffb20080 sd s2,128(sp)
> ffffffff802414e8: ffb10078 sd s1,120(sp)
> ffffffff802414ec: ffb00070 sd s0,112(sp)
> ffffffff802414f0: 3c038021 lui v1,0x8021
> ffffffff802414f4: 646316b0 daddiu v1,v1,5808
> ffffffff802414f8: 03e0082d move at,ra
> ffffffff802414fc: 0060f809 jalr v1
> ffffffff80241500: 00020021 nop
> ip -->
>
> At first, we move to "lui, v1, HI_16BIT_OF_MCOUNT", ip = ip - 12(not 8
> when without -mlong-calls, i need to update the source code later).
Yes with -mlong-calls you must jump pass the setting up of the jalr.
>
> and then, we check whether there is a "Store" instruction, if it's not a
> "Store" instruction, the function should be a leaf? otherwise, we
> continue the searching until finding the "s{d,w} ra, offset(sp)"
> instruction, get the offset, calculate the stack address, and finish?
Note, you are commenting different than your code. Your code matches
more what I want than your comments ;-) You keep saying "if the
instruction just after the jump to mcount (and preparation for) is not a
store than it is a leaf", where I'm saying (and the code matches),
search above the jump to mcount and if we don't find the store of ra,
then it is a leaf.
>
> So, we just need to replace this:
>
> if ((code & MOV_FP_SP) == MOV_FP_SP)
> return parent_addr;
>
> by
>
> #define S_INSN (0xafb0 << 16)
>
> if ((code & S_INSN) != S_INSN)
> return parent_addr;
I would be even more paranoid, and make sure each of those stores, store
into sp.
>
> >
> > > +
> > > + sp = fp + (code & STACK_OFFSET_MASK);
> > > + ra = *(unsigned long *)sp;
> >
> > Also might want to make the above into a asm with exception handling.
> >
> > > +
> > > + if (ra == parent)
> > > + return sp;
> > > +
> > > + ftrace_graph_stop();
> > > + WARN_ON(1);
> > > + return parent_addr;
> >
> > Hmm, may need to do more than this. See below.
> >
> > > +}
> > > +
> > > +/*
> > > + * Hook the return address and push it in the stack of return addrs
> > > + * in current thread info.
> > > + */
> > > +void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
> > > + unsigned long fp)
> > > +{
> > > + unsigned long old;
> > > + struct ftrace_graph_ent trace;
> > > + unsigned long return_hooker = (unsigned long)
> > > + &return_to_handler;
> > > +
> > > + if (unlikely(atomic_read(¤t->tracing_graph_pause)))
> > > + return;
> > > +
> > > + /* "parent" is the stack address saved the return address of the caller
> > > + * of _mcount, for a leaf function not save the return address in the
> > > + * stack address, so, we "emulate" one in _mcount's stack space, and
> > > + * hijack it directly, but for a non-leaf function, it will save the
> > > + * return address to the its stack space, so, we can not hijack the
> > > + * "parent" directly, but need to find the real stack address,
> > > + * ftrace_get_parent_addr() does it!
> > > + */
> > > +
> > > + old = *parent;
> > > +
> > > + parent = (unsigned long *)ftrace_get_parent_addr(self_addr, old,
> > > + (unsigned long)parent,
> > > + fp);
> > > +
> > > + *parent = return_hooker;
> >
> > Although you may have turned off fgraph tracer in
> > ftrace_get_parent_addr, nothing stops the below from messing with the
> > stack. The return stack may get off sync and break later. If you fail
> > the above, you should not be calling the push function below.
> >
>
> We need to really stop before ftrace_push_return_trace to avoid messing
> with the stack :-) but if we have stopped the tracer, is it important to
> mess with the stack or not?
The ftrace_push_return_trace does not test if the trace stopped, that is
expected to be done by the caller. If you mess with the stack set up,
you will crash the box. Remember, before the failure, you could have
already replaced return jumps. Those will still be falling back to the
return_to_handler. If you mess with the stack, but don't update the
return, the other returns will be out of sync and call the wrong return
address.
-- Steve
