We've noticed that udelay() and related functions are too slow by a factor of 2 on Cavium CPUs. I did some investigation and it seems to be related to the fact that Cavium is the only MIPS CPU family that has ARCH_HAS_READ_CURRENT_TIMER defined to use its hardware cycle counter (CvmCount) to calibrate loops-per-jiffy. Since udelay() uses lpj to calculate the number of loops to wait in __delay() it is affected by the lpj calibration. On a 600MHz Cavium system running 2.6.33-rc8 with HZ=250 the lpj is 2404728. This results in udelay() passing 601 to __delay() for a 1us delay which would work if __delay took 1 cycle per loop however it takes 2 cycles giving a delay of 2us. It seems the easiest way to fix the problem would be to remove the definition of ARCH_HAS_READ_CURRENT_TIMER for Cavium which would use the same lpj calibration delay loop as other MIPS CPUs. With this change on the same 600MHz system lpj is 1196032 which results in udelay() passing 299 to __delay() which would yield close to the desired 1us delay. I'm not sure what all of the implications would be of effectively halving lpj on Cavium CPUs or what the rationale was for defining ARCH_HAS_READ_CURRENT_TIMER for Cavium CPUs in the first place. I'm hoping someone more informed can take a look at it and propose a fix if what I've proposed isn't good enough.
