> On Thu, Oct 26, 2006 at 01:05:52PM +0900, Atsushi Nemoto wrote:
>
> > I think I found the problem at last.
>
> I'm afraid there is more than one problem.
>
> On the 34K core each VPE has its own c0_count and c0_compare registers.
> However the reset values are undefined. Which means the time offset
> calculated by
>
> offset = (clocksource_read(clock) - clock->cycle_last) & clock->mask;
>
> may differ wildly between processors resulting in a time jitter of upto
> almost 215s between both VPEs. Unfortunately there is an unavoidable
> race condition when attempting to synchronize the two counters. But
> the 34K's nature shrinks the time window to somwhere in the single digit
> range of cycles so on a hardcore that would be a handfull of nanoseconds.
I don't see what's different here than in any other SMP case. Is it really
true that the MIPS SMP support *requires* that all CPUs in the system
come out of reset on the same clock, with the same value in Count?
I find that very surprising (and a little disappointing). Is this a general
limitation of Linux? MIPS32/MIPS64 PRAs call out the reset value
of Count as being undefined, and chip specs for pre-MIPS32 CPUs
like the R10000 and the R4400 do not call out any reset value for
Count either.
If there's going to be skew between CPU clocks, all it really means
is that one cannot directly compare timestamps generated by different
CPUs. At a given point in time, "How long will it be until you hit an
absolute Count value X?" will have a slightly different answer on each CPU
if there is skew, but "What will the local Count value be N jiffies from now?"
should be something that can be correctly calculated independently on each
node. Where are we depending on the former, and can that usage be converted
into something more like the later?
Regards,
Kevin K.
