(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Wed, 28 Jan 2009 11:29:52 -0800 (PST) bugme-daemon@xxxxxxxxxxxxxxxxxxx wrote: > http://bugzilla.kernel.org/show_bug.cgi?id=12564 > > Summary: poor performance while preprocessing source code > Product: IO/Storage Thanks for the report. > Version: 2.5 > KernelVersion: 2.6.28.2 > Platform: All > OS/Version: Linux > Tree: Mainline > Status: NEW > Severity: normal > Priority: P1 > Component: Other > AssignedTo: io_other@xxxxxxxxxxxxxxxxxxxx > ReportedBy: steven@xxxxxxxx > > > Latest working kernel version: 2.6.26-rc3 > Earliest failing kernel version: 2.6.26-rc4 (or 2.6.26-rc3 + patch) (huge performance regression in NFS) > Distribution: gentoo > > Hardware Environment: > Various 32 and 64 bit Intel and AMD machines with various > PATA and SATA disks and various network interfaces. > > Problem Description: > Here's my situation. I've recently upgraded the kernels > on ~30 computers at work (from 2.6.21 to 2.6.27). These > computers are used to build and test software we develop. > We speed up the building process using distcc. However, > after the kernel upgrade, the builds are much much slower. > The preprocessing stage seems to be at least 10 times > slower. > As evidence of this slowdown I am attaching two images created > using distccmon-gnome. Both snapshots were taken shortly > after starting builds in a clean sandbox. The only difference > is the kernel. "fast.png" was generated while running > kernel 2.6.25.20. "slow.png" was generated with 2.6.26. > The light purple sections indicate the preprocessing times > for each file. > This slowdown is observed on both 32 and 64 bit computers > and using either gcc or the intel compiler. (The intel compiler > builds do not use distcc, but that are also slower.) Strangely > enough, it's still faster to use an NFS mounted sandbox on a > machine with an older kernel than the same sandbox on the local > machine with a newer kernel. (This suggests to me that it > is neither a disk or network IO problem.) > I've used git bisect to narrow it down to a single commit: > # bad: [b0b539739fe9b7d75002412a787cfdf4efddbc33] NFS: Ensure that 'noac' > and/or 'actimeo=0' turn off attribute caching And thanks for bisecting it. > This is the first commit after v2.6.26-rc3. I'm not an > experienced git user, so I don't know how to narrow it down > further. Distccmon-gnome snapshots from the bisection > process are similar to the ones noted above. > Naturally, I would like the newer kernels to have similar > performance to the older kernels. > I will be attaching various items. Let me know what other > information might be helpful. > > Steps to reproduce: > distccmon-gnome & > # using a makefile setup to use distcc: > make -j 5 all > # note preprocessing times in distccmon > Something I don't understand from this: If your normal setup is using distcc then what role does NFS have to play? I mean, distcc kind-of replaces NFS in this workload. Perhaps you could briefly describe the topology/data-flow/etc so that we can see how NFS could be a bottleneck here? Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
