This is a full release of all the patches so apologies for the flood. V9 was just a MIPS build fix and did not justify a full release. V10 includes Ingo's scalability patches because even though they increase system CPU usage, they also helped in a number of test cases. It would be worthwhile trying to reduce the system CPU usage by looking closer at how rwsem works and dealing with the contended case a bit better. Otherwise the rate of change in the last few weeks has been tiny as the preliminary objectives had been met and I did not want to invalidate any testing other people had conducted. git tree: git://git.kernel.org/pub/scm/linux/kernel/git/mel/linux-balancenuma.git mm-balancenuma-v10r3 git tag: git://git.kernel.org/pub/scm/linux/kernel/git/mel/linux-balancenuma.git mm-balancenuma-v10 Based on the performance results I have, I still think this tree is what should be merged for 3.8 with numacore or autonuma rebased on top of it for 3.9. The numacore results are based on an oldish tree. I'm travelling this week and tip/master crashes and I haven't had the chance to debug it. Worse, v18 of numacore had a last-minute patch that effectively disabled it. I reported it (https://lkml.org/lkml/2012/12/4/393) but got no feedback. Changelog since V9 o Migration scalability (mingo) Changelog since V8 o Fix build error on MIPS (rientjes) Changelog since V7 o Account for transhuge migrations properly when migrate rate-limiting Changelog since V6 o Transfer last_nid information during transhuge migration (dhillf) o Transfer last_nid information during splits (dhillf) o Drop page reference if target node is full (dhillf) o Account for transhuge allocation failure as migration failure (mel) Changelog since V5 o Fix build errors related to config options, make bisect-safe o Account for transhuge migrations o Count HPAGE_PMD_NR pages when isolating transhuge o Account for local transphuge faults o Fix a memory leak on isolation failure Changelog since V4 o Allow enabling/disable from command line o Delay PTE scanning until tasks are running on a new node o THP migration bits needed for memcg o Adapt the scanning rate depending on whether pages need to migrate o Drop all the scheduler policy stuff on top, it was broken Changelog since V3 o Use change_protection o Architecture-hook twiddling o Port of the THP migration patch. o Additional TLB optimisations o Fixes from Hillf Danton Changelog since V2 o Do not allocate from home node o Mostly remove pmd_numa handling for regular pmds o HOME policy will allocate from and migrate towards local node o Load balancer is more aggressive about moving tasks towards home node o Renames to sync up more with -tip version o Move pte handlers to generic code o Scanning rate starts at 100ms, system CPU usage expected to increase o Handle migration of PMD hinting faults o Rate limit migration on a per-node basis o Alter how the rate of PTE scanning is adapted o Rate limit setting of pte_numa if node is congested o Only flush local TLB is unmapping a pte_numa page o Only consider one CPU in cpu follow algorithm Changelog since V1 o Account for faults on the correct node after migration o Do not account for THP splits as faults. o Account THP faults on the node they occurred o Ensure preferred_node_policy is initialised before use o Mitigate double faults o Add home-node logic o Add some tlb-flush mitigation patches o Add variation of CPU follows memory algorithm o Add last_nid and use it as a two-stage filter before migrating pages o Restart the PTE scanner when it reaches the end of the address space o Lots of stuff I did not note properly There are currently two (three depending on how you look at it) competing approaches to implement support for automatically migrating pages to optimise NUMA locality. Performance results are available but review highlighted different problems in both. They are not compatible with each other even though some fundamental mechanics should have been the same. This series addresses part of the integration and sharing problem by implementing a foundation that either the policy for schednuma or autonuma can be rebased on. The initial policy it implements is a very basic greedy policy called "Migrate On Reference Of pte_numa Node (MORON)". I expect people to build upon this revised policy and rename it to something more sensible that reflects what it means. The ideal *worst-case* behaviour is that it is comparable to current mainline but for some workloads this is an improvement over mainline. This series can be treated as 5 major stages. 1. TLB optimisations that we're likely to want unconditionally. 2. Basic foundation and core mechanics, initial policy that does very little 3. Full PMD fault handling, rate limiting of migration, two-stage migration filter to mitigate poor migration decisions. This will migrate pages on a PTE or PMD level using just the current referencing CPU as a placement hint 4. Scan rate adaption 5. Native THP migration Very broadly speaking the TODOs that spring to mind are 1. Revisit MPOL_NOOP and MPOL_MF_LAZY 2. Other architecture support or at least validation that it could be made work. I'm half-hoping that the PPC64 people are watching because they tend to be interested in this type of thing. Some advantages of the series are; 1. It rate limits migrations to avoid saturating the bus and backs off PTE scanning (in a fairly heavy manner) if the node is rate-limited 2. It keeps major optimisations like THP towards the end to be sure I am not accidentally depending on them 3. It implements a basic policy that acts as a second performance baseline. The three baselines become vanilla kernel, basic placement policy, complex placement policy. This allows like-with-like comparisons with implementations. The comparisons are a bit shorter this time. Kernels are stats-v8r6 TLB flush optimisations and stats from this series numacore-20121130 Tip/master on that date (roughly v17) numacore-20121202 Tip/master on that date (roughly v18) autonuma-v28fastr4 Autonuma v28fast rebased and with THP patch on top balancenuma-v9r2 balancenuma-v9 balancenuma-v10 balancenuma-v10 v9 and v10 only differ by the migration scalability patches. Current tip/master is crashing during boot and has been crashing for the last few days which is why it's not included. As I'm remote I have not had the chance to debug it but it has been reported already. It does mean that the numacore comparison is old and not based on the unified tree but right now there is not much I can do about that. This is less detailed than earlier reports because many of the conclusions are the same as before. AUTONUMA BENCH 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 User NUMA01 65230.85 ( 0.00%) 24835.22 ( 61.93%) 69344.37 ( -6.31%) 30410.22 ( 53.38%) 52436.65 ( 19.61%) 59949.95 ( 8.10%) User NUMA01_THEADLOCAL 60794.67 ( 0.00%) 17856.17 ( 70.63%) 53416.06 ( 12.14%) 17185.34 ( 71.73%) 17829.96 ( 70.67%) 17501.83 ( 71.21%) User NUMA02 7031.50 ( 0.00%) 2084.38 ( 70.36%) 6726.17 ( 4.34%) 2238.73 ( 68.16%) 2079.48 ( 70.43%) 2094.68 ( 70.21%) User NUMA02_SMT 2916.19 ( 0.00%) 1009.28 ( 65.39%) 3207.30 ( -9.98%) 1037.07 ( 64.44%) 997.57 ( 65.79%) 1010.15 ( 65.36%) System NUMA01 39.66 ( 0.00%) 926.55 (-2236.23%) 333.49 (-740.87%) 236.83 (-497.15%) 275.09 (-593.62%) 265.02 (-568.23%) System NUMA01_THEADLOCAL 42.33 ( 0.00%) 513.99 (-1114.25%) 40.59 ( 4.11%) 70.90 (-67.49%) 110.82 (-161.80%) 130.30 (-207.82%) System NUMA02 1.25 ( 0.00%) 18.57 (-1385.60%) 1.04 ( 16.80%) 6.39 (-411.20%) 6.42 (-413.60%) 9.17 (-633.60%) System NUMA02_SMT 16.66 ( 0.00%) 12.32 ( 26.05%) 0.95 ( 94.30%) 3.17 ( 80.97%) 3.58 ( 78.51%) 6.21 ( 62.73%) Elapsed NUMA01 1511.76 ( 0.00%) 575.93 ( 61.90%) 1644.63 ( -8.79%) 701.62 ( 53.59%) 1185.53 ( 21.58%) 1352.74 ( 10.52%) Elapsed NUMA01_THEADLOCAL 1387.17 ( 0.00%) 398.55 ( 71.27%) 1260.92 ( 9.10%) 378.47 ( 72.72%) 397.37 ( 71.35%) 387.93 ( 72.03%) Elapsed NUMA02 176.81 ( 0.00%) 51.14 ( 71.08%) 180.80 ( -2.26%) 53.45 ( 69.77%) 49.51 ( 72.00%) 49.77 ( 71.85%) Elapsed NUMA02_SMT 163.96 ( 0.00%) 48.92 ( 70.16%) 166.96 ( -1.83%) 48.17 ( 70.62%) 47.71 ( 70.90%) 48.63 ( 70.34%) CPU NUMA01 4317.00 ( 0.00%) 4473.00 ( -3.61%) 4236.00 ( 1.88%) 4368.00 ( -1.18%) 4446.00 ( -2.99%) 4451.00 ( -3.10%) CPU NUMA01_THEADLOCAL 4385.00 ( 0.00%) 4609.00 ( -5.11%) 4239.00 ( 3.33%) 4559.00 ( -3.97%) 4514.00 ( -2.94%) 4545.00 ( -3.65%) CPU NUMA02 3977.00 ( 0.00%) 4111.00 ( -3.37%) 3720.00 ( 6.46%) 4200.00 ( -5.61%) 4212.00 ( -5.91%) 4226.00 ( -6.26%) CPU NUMA02_SMT 1788.00 ( 0.00%) 2087.00 (-16.72%) 1921.00 ( -7.44%) 2159.00 (-20.75%) 2098.00 (-17.34%) 2089.00 (-16.83%) numacore-20121130 did reasonably well although its system CPU usage is extremely high. numacore-20121202 is very poor and roughly comparable to mainline. This is likely because numacore is effectively disabled in this release. The reasons it is likely disabled have already been reported and current tip/master looks like it would suffer the same problem if it booted. balancenuma does reasonably well. It's not great at numa01 which is an adverse workload as it does not know how to interleave which is what's needed in this case. It does very well for the other test cases. MMTests Statistics: duration 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6numacore-20121130numacore-20121202autonuma-v28fastr4balancenuma-v9r2balancenuma-v10r3 User 135980.38 45792.55 132701.13 50878.50 73350.91 80563.56 System 100.53 1472.19 376.74 317.89 396.58 411.40 Elapsed 3248.36 1084.63 3262.62 1191.85 1689.70 1847.35 numacore-20121130 has very high system CPU usaage. balancenumas is higher than I'd like but it's acceptable. Specjbb Multiple JVMs, 4 Nodes 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 Mean 1 31311.75 ( 0.00%) 27938.00 (-10.77%) 29681.25 ( -5.21%) 31474.25 ( 0.52%) 31112.00 ( -0.64%) 31281.50 ( -0.10%) Mean 2 62972.75 ( 0.00%) 51899.00 (-17.58%) 60403.00 ( -4.08%) 66654.00 ( 5.85%) 62937.50 ( -0.06%) 62483.50 ( -0.78%) Mean 3 91292.00 ( 0.00%) 80908.00 (-11.37%) 86570.25 ( -5.17%) 97177.50 ( 6.45%) 90665.50 ( -0.69%) 90667.00 ( -0.68%) Mean 4 115768.75 ( 0.00%) 99497.25 (-14.06%) 105982.25 ( -8.45%) 125596.00 ( 8.49%) 116812.50 ( 0.90%) 116193.50 ( 0.37%) Mean 5 137248.50 ( 0.00%) 92837.75 (-32.36%) 115640.50 (-15.74%) 152795.25 ( 11.33%) 139037.75 ( 1.30%) 139055.50 ( 1.32%) Mean 6 155528.50 ( 0.00%) 105554.50 (-32.13%) 124614.75 (-19.88%) 177455.25 ( 14.10%) 155769.25 ( 0.15%) 159129.50 ( 2.32%) Mean 7 156747.50 ( 0.00%) 122582.25 (-21.80%) 133205.00 (-15.02%) 184578.75 ( 17.76%) 157103.25 ( 0.23%) 163234.00 ( 4.14%) Mean 8 152069.50 ( 0.00%) 122439.00 (-19.48%) 132939.25 (-12.58%) 186619.25 ( 22.72%) 157631.00 ( 3.66%) 163077.75 ( 7.24%) Mean 9 146609.75 ( 0.00%) 112410.00 (-23.33%) 123667.25 (-15.65%) 186165.00 ( 26.98%) 152561.00 ( 4.06%) 159656.00 ( 8.90%) Mean 10 142819.00 ( 0.00%) 111456.00 (-21.96%) 117609.00 (-17.65%) 182569.75 ( 27.83%) 145320.00 ( 1.75%) 153414.25 ( 7.42%) Mean 11 128292.25 ( 0.00%) 98027.00 (-23.59%) 112410.25 (-12.38%) 176104.75 ( 37.27%) 138599.50 ( 8.03%) 147194.25 ( 14.73%) Mean 12 128769.75 ( 0.00%) 129469.50 ( 0.54%) 106629.50 (-17.19%) 169003.00 ( 31.24%) 131994.75 ( 2.50%) 140049.75 ( 8.76%) Mean 13 126488.50 ( 0.00%) 110133.75 (-12.93%) 106878.25 (-15.50%) 162725.75 ( 28.65%) 130005.25 ( 2.78%) 139109.75 ( 9.98%) Mean 14 123400.00 ( 0.00%) 117929.75 ( -4.43%) 105558.25 (-14.46%) 163781.25 ( 32.72%) 126340.75 ( 2.38%) 137883.00 ( 11.74%) Mean 15 122139.50 ( 0.00%) 122404.25 ( 0.22%) 102829.25 (-15.81%) 160800.25 ( 31.65%) 128612.75 ( 5.30%) 136624.00 ( 11.86%) Mean 16 116413.50 ( 0.00%) 124573.50 ( 7.01%) 100475.75 (-13.69%) 160882.75 ( 38.20%) 117793.75 ( 1.19%) 134005.75 ( 15.11%) Mean 17 117263.25 ( 0.00%) 121937.25 ( 3.99%) 97237.75 (-17.08%) 159069.75 ( 35.65%) 121991.75 ( 4.03%) 133444.50 ( 13.80%) Mean 18 117277.00 ( 0.00%) 116633.75 ( -0.55%) 96547.00 (-17.68%) 158694.75 ( 35.32%) 119089.75 ( 1.55%) 129650.75 ( 10.55%) Mean 19 113231.00 ( 0.00%) 111035.75 ( -1.94%) 97683.00 (-13.73%) 155563.25 ( 37.39%) 119699.75 ( 5.71%) 123403.25 ( 8.98%) Mean 20 113628.75 ( 0.00%) 113451.25 ( -0.16%) 96311.75 (-15.24%) 154779.75 ( 36.22%) 118400.75 ( 4.20%) 126041.25 ( 10.92%) Mean 21 110982.50 ( 0.00%) 107660.50 ( -2.99%) 93732.50 (-15.54%) 151147.25 ( 36.19%) 115663.25 ( 4.22%) 121906.50 ( 9.84%) Mean 22 107660.25 ( 0.00%) 104771.50 ( -2.68%) 91888.75 (-14.65%) 151180.50 ( 40.42%) 111038.00 ( 3.14%) 125519.00 ( 16.59%) Mean 23 105320.50 ( 0.00%) 88275.25 (-16.18%) 91594.75 (-13.03%) 147032.00 ( 39.60%) 112817.50 ( 7.12%) 124148.25 ( 17.88%) Mean 24 110900.50 ( 0.00%) 85169.00 (-23.20%) 87782.75 (-20.85%) 147407.00 ( 32.92%) 109556.50 ( -1.21%) 122544.00 ( 10.50%) Stddev 1 720.83 ( 0.00%) 982.31 (-36.28%) 1738.11 (-141.13%) 942.80 (-30.79%) 1170.23 (-62.35%) 539.84 ( 25.11%) Stddev 2 466.00 ( 0.00%) 1770.75 (-279.99%) 437.94 ( 6.02%) 1327.32 (-184.83%) 1368.51 (-193.67%) 2103.32 (-351.35%) Stddev 3 509.61 ( 0.00%) 4849.62 (-851.63%) 1892.19 (-271.30%) 1803.72 (-253.94%) 1088.04 (-113.50%) 410.73 ( 19.40%) Stddev 4 1750.10 ( 0.00%) 10708.16 (-511.86%) 5762.55 (-229.27%) 2010.11 (-14.86%) 1456.90 ( 16.75%) 1370.22 ( 21.71%) Stddev 5 700.05 ( 0.00%) 16497.79 (-2256.66%) 4658.04 (-565.39%) 2354.70 (-236.36%) 759.38 ( -8.48%) 1869.54 (-167.06%) Stddev 6 2259.33 ( 0.00%) 24221.98 (-972.09%) 6618.94 (-192.96%) 1516.32 ( 32.89%) 1032.39 ( 54.31%) 1720.87 ( 23.83%) Stddev 7 3390.99 ( 0.00%) 4721.80 (-39.25%) 7337.14 (-116.37%) 2398.34 ( 29.27%) 2487.08 ( 26.66%) 4327.85 (-27.63%) Stddev 8 7533.18 ( 0.00%) 8609.90 (-14.29%) 9431.33 (-25.20%) 2895.55 ( 61.56%) 3902.53 ( 48.20%) 2536.68 ( 66.33%) Stddev 9 9223.98 ( 0.00%) 10731.70 (-16.35%) 10681.30 (-15.80%) 4726.23 ( 48.76%) 5673.20 ( 38.50%) 3377.59 ( 63.38%) Stddev 10 4578.09 ( 0.00%) 11136.27 (-143.25%) 12513.13 (-173.33%) 6705.48 (-46.47%) 5516.47 (-20.50%) 7227.58 (-57.87%) Stddev 11 8201.30 ( 0.00%) 3580.27 ( 56.35%) 18390.50 (-124.24%) 10915.90 (-33.10%) 4757.42 ( 41.99%) 4056.02 ( 50.54%) Stddev 12 5713.70 ( 0.00%) 13923.12 (-143.68%) 15228.05 (-166.52%) 16555.64 (-189.75%) 4573.05 ( 19.96%) 3678.89 ( 35.61%) Stddev 13 5878.95 ( 0.00%) 10471.09 (-78.11%) 14014.88 (-138.39%) 18628.01 (-216.86%) 1680.65 ( 71.41%) 3947.39 ( 32.86%) Stddev 14 4783.95 ( 0.00%) 4051.35 ( 15.31%) 13764.72 (-187.73%) 18324.63 (-283.04%) 2637.82 ( 44.86%) 4806.09 ( -0.46%) Stddev 15 6281.48 ( 0.00%) 3357.07 ( 46.56%) 11925.69 (-89.85%) 17654.58 (-181.06%) 2003.38 ( 68.11%) 3005.22 ( 52.16%) Stddev 16 6948.12 ( 0.00%) 3763.32 ( 45.84%) 13658.66 (-96.58%) 18280.52 (-163.10%) 3526.10 ( 49.25%) 3309.24 ( 52.37%) Stddev 17 5603.77 ( 0.00%) 1452.04 ( 74.09%) 12618.33 (-125.18%) 18230.53 (-225.33%) 1712.95 ( 69.43%) 3516.09 ( 37.25%) Stddev 18 6200.90 ( 0.00%) 1870.12 ( 69.84%) 11261.01 (-81.60%) 18486.73 (-198.13%) 751.36 ( 87.88%) 2412.60 ( 61.09%) Stddev 19 6726.31 ( 0.00%) 1045.21 ( 84.46%) 10748.09 (-59.79%) 18465.25 (-174.52%) 1750.49 ( 73.98%) 4482.82 ( 33.35%) Stddev 20 5713.58 ( 0.00%) 2066.90 ( 63.82%) 12195.08 (-113.44%) 19947.77 (-249.13%) 1892.91 ( 66.87%) 2612.62 ( 54.27%) Stddev 21 4566.92 ( 0.00%) 2460.40 ( 46.13%) 14089.14 (-208.50%) 21189.08 (-363.97%) 3639.75 ( 20.30%) 1963.17 ( 57.01%) Stddev 22 6168.05 ( 0.00%) 2770.81 ( 55.08%) 10037.19 (-62.73%) 20033.82 (-224.80%) 3682.20 ( 40.30%) 1159.17 ( 81.21%) Stddev 23 6295.45 ( 0.00%) 1337.32 ( 78.76%) 13290.13 (-111.11%) 22610.91 (-259.16%) 2013.53 ( 68.02%) 3842.61 ( 38.96%) Stddev 24 3108.17 ( 0.00%) 1381.20 ( 55.56%) 12637.15 (-306.58%) 21243.56 (-583.47%) 4044.16 (-30.11%) 2673.39 ( 13.99%) TPut 1 125247.00 ( 0.00%) 111752.00 (-10.77%) 118725.00 ( -5.21%) 125897.00 ( 0.52%) 124448.00 ( -0.64%) 125126.00 ( -0.10%) TPut 2 251891.00 ( 0.00%) 207596.00 (-17.58%) 241612.00 ( -4.08%) 266616.00 ( 5.85%) 251750.00 ( -0.06%) 249934.00 ( -0.78%) TPut 3 365168.00 ( 0.00%) 323632.00 (-11.37%) 346281.00 ( -5.17%) 388710.00 ( 6.45%) 362662.00 ( -0.69%) 362668.00 ( -0.68%) TPut 4 463075.00 ( 0.00%) 397989.00 (-14.06%) 423929.00 ( -8.45%) 502384.00 ( 8.49%) 467250.00 ( 0.90%) 464774.00 ( 0.37%) TPut 5 548994.00 ( 0.00%) 371351.00 (-32.36%) 462562.00 (-15.74%) 611181.00 ( 11.33%) 556151.00 ( 1.30%) 556222.00 ( 1.32%) TPut 6 622114.00 ( 0.00%) 422218.00 (-32.13%) 498459.00 (-19.88%) 709821.00 ( 14.10%) 623077.00 ( 0.15%) 636518.00 ( 2.32%) TPut 7 626990.00 ( 0.00%) 490329.00 (-21.80%) 532820.00 (-15.02%) 738315.00 ( 17.76%) 628413.00 ( 0.23%) 652936.00 ( 4.14%) TPut 8 608278.00 ( 0.00%) 489756.00 (-19.48%) 531757.00 (-12.58%) 746477.00 ( 22.72%) 630524.00 ( 3.66%) 652311.00 ( 7.24%) TPut 9 586439.00 ( 0.00%) 449640.00 (-23.33%) 494669.00 (-15.65%) 744660.00 ( 26.98%) 610244.00 ( 4.06%) 638624.00 ( 8.90%) TPut 10 571276.00 ( 0.00%) 445824.00 (-21.96%) 470436.00 (-17.65%) 730279.00 ( 27.83%) 581280.00 ( 1.75%) 613657.00 ( 7.42%) TPut 11 513169.00 ( 0.00%) 392108.00 (-23.59%) 449641.00 (-12.38%) 704419.00 ( 37.27%) 554398.00 ( 8.03%) 588777.00 ( 14.73%) TPut 12 515079.00 ( 0.00%) 517878.00 ( 0.54%) 426518.00 (-17.19%) 676012.00 ( 31.24%) 527979.00 ( 2.50%) 560199.00 ( 8.76%) TPut 13 505954.00 ( 0.00%) 440535.00 (-12.93%) 427513.00 (-15.50%) 650903.00 ( 28.65%) 520021.00 ( 2.78%) 556439.00 ( 9.98%) TPut 14 493600.00 ( 0.00%) 471719.00 ( -4.43%) 422233.00 (-14.46%) 655125.00 ( 32.72%) 505363.00 ( 2.38%) 551532.00 ( 11.74%) TPut 15 488558.00 ( 0.00%) 489617.00 ( 0.22%) 411317.00 (-15.81%) 643201.00 ( 31.65%) 514451.00 ( 5.30%) 546496.00 ( 11.86%) TPut 16 465654.00 ( 0.00%) 498294.00 ( 7.01%) 401903.00 (-13.69%) 643531.00 ( 38.20%) 471175.00 ( 1.19%) 536023.00 ( 15.11%) TPut 17 469053.00 ( 0.00%) 487749.00 ( 3.99%) 388951.00 (-17.08%) 636279.00 ( 35.65%) 487967.00 ( 4.03%) 533778.00 ( 13.80%) TPut 18 469108.00 ( 0.00%) 466535.00 ( -0.55%) 386188.00 (-17.68%) 634779.00 ( 35.32%) 476359.00 ( 1.55%) 518603.00 ( 10.55%) TPut 19 452924.00 ( 0.00%) 444143.00 ( -1.94%) 390732.00 (-13.73%) 622253.00 ( 37.39%) 478799.00 ( 5.71%) 493613.00 ( 8.98%) TPut 20 454515.00 ( 0.00%) 453805.00 ( -0.16%) 385247.00 (-15.24%) 619119.00 ( 36.22%) 473603.00 ( 4.20%) 504165.00 ( 10.92%) TPut 21 443930.00 ( 0.00%) 430642.00 ( -2.99%) 374930.00 (-15.54%) 604589.00 ( 36.19%) 462653.00 ( 4.22%) 487626.00 ( 9.84%) TPut 22 430641.00 ( 0.00%) 419086.00 ( -2.68%) 367555.00 (-14.65%) 604722.00 ( 40.42%) 444152.00 ( 3.14%) 502076.00 ( 16.59%) TPut 23 421282.00 ( 0.00%) 353101.00 (-16.18%) 366379.00 (-13.03%) 588128.00 ( 39.60%) 451270.00 ( 7.12%) 496593.00 ( 17.88%) TPut 24 443602.00 ( 0.00%) 340676.00 (-23.20%) 351131.00 (-20.85%) 589628.00 ( 32.92%) 438226.00 ( -1.21%) 490176.00 ( 10.50%) numacore is regressing heavily in this case. It's particularly weird for numacore-20121202 as numacore should be effectively disabled. It's adding overhead somewhere but not doing anything useful with it. balancenuma gets about 1/3 of the performance gain of autonuma and the migration scalabilty patches help quite a lot. SPECJBB PEAKS 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 Expctd Warehouse 12.00 ( 0.00%) 12.00 ( 0.00%) 12.00 ( 0.00%) 12.00 ( 0.00%) 12.00 ( 0.00%) 12.00 ( 0.00%) Expctd Peak Bops 515079.00 ( 0.00%) 517878.00 ( 0.54%) 426518.00 (-17.19%) 676012.00 ( 31.24%) 527979.00 ( 2.50%) 560199.00 ( 8.76%) Actual Warehouse 7.00 ( 0.00%) 12.00 ( 71.43%) 7.00 ( 0.00%) 8.00 ( 14.29%) 8.00 ( 14.29%) 7.00 ( 0.00%) Actual Peak Bops 626990.00 ( 0.00%) 517878.00 (-17.40%) 532820.00 (-15.02%) 746477.00 ( 19.06%) 630524.00 ( 0.56%) 652936.00 ( 4.14%) SpecJBB Bops 465685.00 ( 0.00%) 447214.00 ( -3.97%) 392353.00 (-15.75%) 628328.00 ( 34.93%) 480925.00 ( 3.27%) 521332.00 ( 11.95%) SpecJBB Bops/JVM 116421.00 ( 0.00%) 111804.00 ( -3.97%) 98088.00 (-15.75%) 157082.00 ( 34.93%) 120231.00 ( 3.27%) 130333.00 ( 11.95%) numacore is regressing at the peak and in its overall specjbb score. balancenuma again is getting some solid performance gains -- not as much as autonuma but the objective was to be better than mainline, not necessarily be the best overall. numacore or autonuma can be rebased on top of balancenuma. MMTests Statistics: duration 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6numacore-20121130numacore-20121202autonuma-v28fastr4balancenuma-v9r2balancenuma-v10r3 User 177835.94 171938.81 177810.87 177457.20 177445.71 177513.08 System 166.79 5814.00 168.00 207.74 527.49 503.25 Elapsed 4037.12 4038.74 4030.32 4037.22 4035.76 4037.74 numacores system CPU usage is very high. It's not high in 20121202 because it's mostly disabled. As before, balancenumas is higher than I'd like and the migraiton patches do not hurt. SpecJBB Multiple JVMs, THP disabled 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 Mean 1 26036.50 ( 0.00%) 19595.00 (-24.74%) 24601.50 ( -5.51%) 24738.25 ( -4.99%) 25595.00 ( -1.70%) 25610.50 ( -1.64%) Mean 2 53629.75 ( 0.00%) 38481.50 (-28.25%) 52351.25 ( -2.38%) 55646.75 ( 3.76%) 53045.25 ( -1.09%) 53383.00 ( -0.46%) Mean 3 77385.00 ( 0.00%) 53685.50 (-30.63%) 75993.00 ( -1.80%) 82714.75 ( 6.89%) 76596.00 ( -1.02%) 76502.75 ( -1.14%) Mean 4 100097.75 ( 0.00%) 68253.50 (-31.81%) 92149.50 ( -7.94%) 107883.25 ( 7.78%) 98618.00 ( -1.48%) 99786.50 ( -0.31%) Mean 5 119012.75 ( 0.00%) 74164.50 (-37.68%) 112056.00 ( -5.85%) 130260.25 ( 9.45%) 119354.50 ( 0.29%) 121741.75 ( 2.29%) Mean 6 137419.25 ( 0.00%) 86158.50 (-37.30%) 133604.50 ( -2.78%) 154244.50 ( 12.24%) 136901.75 ( -0.38%) 136990.50 ( -0.31%) Mean 7 138018.25 ( 0.00%) 96059.25 (-30.40%) 136477.50 ( -1.12%) 159501.00 ( 15.57%) 138265.50 ( 0.18%) 139398.75 ( 1.00%) Mean 8 136774.00 ( 0.00%) 97003.50 (-29.08%) 137033.75 ( 0.19%) 162868.00 ( 19.08%) 138554.50 ( 1.30%) 137340.75 ( 0.41%) Mean 9 127966.50 ( 0.00%) 95261.00 (-25.56%) 135496.00 ( 5.88%) 163008.00 ( 27.38%) 137954.00 ( 7.80%) 134200.50 ( 4.87%) Mean 10 124628.75 ( 0.00%) 96202.25 (-22.81%) 128704.25 ( 3.27%) 159696.50 ( 28.14%) 131322.25 ( 5.37%) 126927.50 ( 1.84%) Mean 11 117269.00 ( 0.00%) 95924.25 (-18.20%) 119718.50 ( 2.09%) 154701.50 ( 31.92%) 125032.75 ( 6.62%) 122925.00 ( 4.82%) Mean 12 111962.25 ( 0.00%) 94247.25 (-15.82%) 115400.75 ( 3.07%) 150936.50 ( 34.81%) 118119.50 ( 5.50%) 119931.75 ( 7.12%) Mean 13 111595.50 ( 0.00%) 106538.50 ( -4.53%) 110988.50 ( -0.54%) 147193.25 ( 31.90%) 116398.75 ( 4.30%) 117349.75 ( 5.16%) Mean 14 110881.00 ( 0.00%) 103549.00 ( -6.61%) 111549.00 ( 0.60%) 144584.00 ( 30.40%) 114934.50 ( 3.66%) 115838.25 ( 4.47%) Mean 15 109337.50 ( 0.00%) 101729.00 ( -6.96%) 108927.25 ( -0.38%) 143333.00 ( 31.09%) 115523.75 ( 5.66%) 115151.25 ( 5.32%) Mean 16 107031.75 ( 0.00%) 101983.75 ( -4.72%) 106160.75 ( -0.81%) 141907.75 ( 32.58%) 113666.00 ( 6.20%) 113673.50 ( 6.21%) Mean 17 105491.25 ( 0.00%) 100205.75 ( -5.01%) 104268.75 ( -1.16%) 140691.00 ( 33.37%) 112751.50 ( 6.88%) 113221.25 ( 7.33%) Mean 18 101102.75 ( 0.00%) 96635.50 ( -4.42%) 104045.75 ( 2.91%) 137784.25 ( 36.28%) 112582.50 ( 11.35%) 111533.50 ( 10.32%) Mean 19 103907.25 ( 0.00%) 94578.25 ( -8.98%) 102897.50 ( -0.97%) 135719.25 ( 30.62%) 110152.25 ( 6.01%) 113959.25 ( 9.67%) Mean 20 100496.00 ( 0.00%) 92683.75 ( -7.77%) 98143.50 ( -2.34%) 135264.25 ( 34.60%) 108861.50 ( 8.32%) 113746.00 ( 13.18%) Mean 21 99570.00 ( 0.00%) 92955.75 ( -6.64%) 97375.00 ( -2.20%) 133891.00 ( 34.47%) 110094.00 ( 10.57%) 109462.50 ( 9.94%) Mean 22 98611.75 ( 0.00%) 89781.75 ( -8.95%) 98287.00 ( -0.33%) 132399.75 ( 34.26%) 109322.75 ( 10.86%) 110502.75 ( 12.06%) Mean 23 98173.00 ( 0.00%) 88846.00 ( -9.50%) 98131.00 ( -0.04%) 130726.00 ( 33.16%) 106046.25 ( 8.02%) 107304.25 ( 9.30%) Mean 24 92074.75 ( 0.00%) 88581.00 ( -3.79%) 96459.75 ( 4.76%) 127552.25 ( 38.53%) 102362.00 ( 11.17%) 107119.25 ( 16.34%) Stddev 1 735.13 ( 0.00%) 538.24 ( 26.78%) 973.28 (-32.40%) 121.08 ( 83.53%) 906.62 (-23.33%) 788.06 ( -7.20%) Stddev 2 406.26 ( 0.00%) 3458.87 (-751.39%) 1082.66 (-166.49%) 477.32 (-17.49%) 1322.57 (-225.55%) 468.57 (-15.34%) Stddev 3 644.20 ( 0.00%) 1360.89 (-111.25%) 1334.10 (-107.09%) 922.47 (-43.20%) 609.27 ( 5.42%) 599.26 ( 6.98%) Stddev 4 743.93 ( 0.00%) 2149.34 (-188.92%) 2267.12 (-204.75%) 1385.42 (-86.23%) 1119.02 (-50.42%) 801.13 ( -7.69%) Stddev 5 898.53 ( 0.00%) 2521.01 (-180.57%) 1948.30 (-116.83%) 763.24 ( 15.06%) 942.52 ( -4.90%) 1718.19 (-91.22%) Stddev 6 1126.61 ( 0.00%) 3818.22 (-238.91%) 917.32 ( 18.58%) 1527.03 (-35.54%) 2445.69 (-117.08%) 1754.32 (-55.72%) Stddev 7 2907.61 ( 0.00%) 4419.29 (-51.99%) 2486.28 ( 14.49%) 1536.66 ( 47.15%) 4881.65 (-67.89%) 4863.83 (-67.28%) Stddev 8 3200.64 ( 0.00%) 382.01 ( 88.06%) 5978.31 (-86.78%) 1228.09 ( 61.63%) 5459.06 (-70.56%) 5583.95 (-74.46%) Stddev 9 2907.92 ( 0.00%) 1813.39 ( 37.64%) 4583.53 (-57.62%) 1502.61 ( 48.33%) 2501.16 ( 13.99%) 2525.02 ( 13.17%) Stddev 10 5093.23 ( 0.00%) 1313.58 ( 74.21%) 8194.93 (-60.90%) 2763.19 ( 45.75%) 2973.78 ( 41.61%) 2005.95 ( 60.62%) Stddev 11 4982.41 ( 0.00%) 1163.02 ( 76.66%) 1899.45 ( 61.88%) 4776.28 ( 4.14%) 6068.34 (-21.80%) 4256.77 ( 14.56%) Stddev 12 3051.38 ( 0.00%) 2117.59 ( 30.60%) 2404.89 ( 21.19%) 9252.59 (-203.23%) 3885.96 (-27.35%) 2580.44 ( 15.43%) Stddev 13 2918.03 ( 0.00%) 2252.11 ( 22.82%) 3889.75 (-33.30%) 9384.83 (-221.62%) 1833.07 ( 37.18%) 2523.28 ( 13.53%) Stddev 14 3178.97 ( 0.00%) 2337.49 ( 26.47%) 3612.00 (-13.62%) 9353.03 (-194.22%) 1072.60 ( 66.26%) 1140.55 ( 64.12%) Stddev 15 2438.31 ( 0.00%) 1707.72 ( 29.96%) 2925.87 (-20.00%) 10494.03 (-330.38%) 2295.76 ( 5.85%) 1213.75 ( 50.22%) Stddev 16 2682.25 ( 0.00%) 840.47 ( 68.67%) 3118.36 (-16.26%) 10343.25 (-285.62%) 2416.09 ( 9.92%) 1697.27 ( 36.72%) Stddev 17 2807.66 ( 0.00%) 1546.16 ( 44.93%) 3750.42 (-33.58%) 11446.15 (-307.68%) 2484.08 ( 11.52%) 563.50 ( 79.93%) Stddev 18 3049.27 ( 0.00%) 934.11 ( 69.37%) 3382.16 (-10.92%) 11779.80 (-286.31%) 1472.27 ( 51.72%) 1533.68 ( 49.70%) Stddev 19 2782.65 ( 0.00%) 735.28 ( 73.58%) 2853.22 ( -2.54%) 11416.35 (-310.27%) 514.78 ( 81.50%) 1283.38 ( 53.88%) Stddev 20 2379.12 ( 0.00%) 956.25 ( 59.81%) 2876.85 (-20.92%) 10511.63 (-341.83%) 1641.25 ( 31.01%) 1758.22 ( 26.10%) Stddev 21 2975.22 ( 0.00%) 438.31 ( 85.27%) 2627.61 ( 11.68%) 11292.91 (-279.57%) 1087.60 ( 63.44%) 434.51 ( 85.40%) Stddev 22 2260.61 ( 0.00%) 718.23 ( 68.23%) 2706.69 (-19.73%) 11993.84 (-430.56%) 909.16 ( 59.78%) 322.32 ( 85.74%) Stddev 23 2900.85 ( 0.00%) 275.47 ( 90.50%) 2348.16 ( 19.05%) 12234.80 (-321.77%) 701.39 ( 75.82%) 1444.19 ( 50.21%) Stddev 24 2578.98 ( 0.00%) 481.68 ( 81.32%) 3346.30 (-29.75%) 12769.61 (-395.14%) 732.56 ( 71.60%) 1777.60 ( 31.07%) TPut 1 104146.00 ( 0.00%) 78380.00 (-24.74%) 98406.00 ( -5.51%) 98953.00 ( -4.99%) 102380.00 ( -1.70%) 102442.00 ( -1.64%) TPut 2 214519.00 ( 0.00%) 153926.00 (-28.25%) 209405.00 ( -2.38%) 222587.00 ( 3.76%) 212181.00 ( -1.09%) 213532.00 ( -0.46%) TPut 3 309540.00 ( 0.00%) 214742.00 (-30.63%) 303972.00 ( -1.80%) 330859.00 ( 6.89%) 306384.00 ( -1.02%) 306011.00 ( -1.14%) TPut 4 400391.00 ( 0.00%) 273014.00 (-31.81%) 368598.00 ( -7.94%) 431533.00 ( 7.78%) 394472.00 ( -1.48%) 399146.00 ( -0.31%) TPut 5 476051.00 ( 0.00%) 296658.00 (-37.68%) 448224.00 ( -5.85%) 521041.00 ( 9.45%) 477418.00 ( 0.29%) 486967.00 ( 2.29%) TPut 6 549677.00 ( 0.00%) 344634.00 (-37.30%) 534418.00 ( -2.78%) 616978.00 ( 12.24%) 547607.00 ( -0.38%) 547962.00 ( -0.31%) TPut 7 552073.00 ( 0.00%) 384237.00 (-30.40%) 545910.00 ( -1.12%) 638004.00 ( 15.57%) 553062.00 ( 0.18%) 557595.00 ( 1.00%) TPut 8 547096.00 ( 0.00%) 388014.00 (-29.08%) 548135.00 ( 0.19%) 651472.00 ( 19.08%) 554218.00 ( 1.30%) 549363.00 ( 0.41%) TPut 9 511866.00 ( 0.00%) 381044.00 (-25.56%) 541984.00 ( 5.88%) 652032.00 ( 27.38%) 551816.00 ( 7.80%) 536802.00 ( 4.87%) TPut 10 498515.00 ( 0.00%) 384809.00 (-22.81%) 514817.00 ( 3.27%) 638786.00 ( 28.14%) 525289.00 ( 5.37%) 507710.00 ( 1.84%) TPut 11 469076.00 ( 0.00%) 383697.00 (-18.20%) 478874.00 ( 2.09%) 618806.00 ( 31.92%) 500131.00 ( 6.62%) 491700.00 ( 4.82%) TPut 12 447849.00 ( 0.00%) 376989.00 (-15.82%) 461603.00 ( 3.07%) 603746.00 ( 34.81%) 472478.00 ( 5.50%) 479727.00 ( 7.12%) TPut 13 446382.00 ( 0.00%) 426154.00 ( -4.53%) 443954.00 ( -0.54%) 588773.00 ( 31.90%) 465595.00 ( 4.30%) 469399.00 ( 5.16%) TPut 14 443524.00 ( 0.00%) 414196.00 ( -6.61%) 446196.00 ( 0.60%) 578336.00 ( 30.40%) 459738.00 ( 3.66%) 463353.00 ( 4.47%) TPut 15 437350.00 ( 0.00%) 406916.00 ( -6.96%) 435709.00 ( -0.38%) 573332.00 ( 31.09%) 462095.00 ( 5.66%) 460605.00 ( 5.32%) TPut 16 428127.00 ( 0.00%) 407935.00 ( -4.72%) 424643.00 ( -0.81%) 567631.00 ( 32.58%) 454664.00 ( 6.20%) 454694.00 ( 6.21%) TPut 17 421965.00 ( 0.00%) 400823.00 ( -5.01%) 417075.00 ( -1.16%) 562764.00 ( 33.37%) 451006.00 ( 6.88%) 452885.00 ( 7.33%) TPut 18 404411.00 ( 0.00%) 386542.00 ( -4.42%) 416183.00 ( 2.91%) 551137.00 ( 36.28%) 450330.00 ( 11.35%) 446134.00 ( 10.32%) TPut 19 415629.00 ( 0.00%) 378313.00 ( -8.98%) 411590.00 ( -0.97%) 542877.00 ( 30.62%) 440609.00 ( 6.01%) 455837.00 ( 9.67%) TPut 20 401984.00 ( 0.00%) 370735.00 ( -7.77%) 392574.00 ( -2.34%) 541057.00 ( 34.60%) 435446.00 ( 8.32%) 454984.00 ( 13.18%) TPut 21 398280.00 ( 0.00%) 371823.00 ( -6.64%) 389500.00 ( -2.20%) 535564.00 ( 34.47%) 440376.00 ( 10.57%) 437850.00 ( 9.94%) TPut 22 394447.00 ( 0.00%) 359127.00 ( -8.95%) 393148.00 ( -0.33%) 529599.00 ( 34.26%) 437291.00 ( 10.86%) 442011.00 ( 12.06%) TPut 23 392692.00 ( 0.00%) 355384.00 ( -9.50%) 392524.00 ( -0.04%) 522904.00 ( 33.16%) 424185.00 ( 8.02%) 429217.00 ( 9.30%) TPut 24 368299.00 ( 0.00%) 354324.00 ( -3.79%) 385839.00 ( 4.76%) 510209.00 ( 38.53%) 409448.00 ( 11.17%) 428477.00 ( 16.34%) As before numacore is regressing, autonuma does best and balancenuma does all right with the migration patches helping a little. SPECJBB PEAKS 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 Expctd Warehouse 12.00 ( 0.00%) 12.00 ( 0.00%) 12.00 ( 0.00%) 12.00 ( 0.00%) 12.00 ( 0.00%) 12.00 ( 0.00%) Expctd Peak Bops 447849.00 ( 0.00%) 376989.00 (-15.82%) 461603.00 ( 3.07%) 603746.00 ( 34.81%) 472478.00 ( 5.50%) 479727.00 ( 7.12%) Actual Warehouse 7.00 ( 0.00%) 13.00 ( 85.71%) 8.00 ( 14.29%) 9.00 ( 28.57%) 8.00 ( 14.29%) 7.00 ( 0.00%) Actual Peak Bops 552073.00 ( 0.00%) 426154.00 (-22.81%) 548135.00 ( -0.71%) 652032.00 ( 18.11%) 554218.00 ( 0.39%) 557595.00 ( 1.00%) SpecJBB Bops 415458.00 ( 0.00%) 385328.00 ( -7.25%) 416195.00 ( 0.18%) 554456.00 ( 33.46%) 446405.00 ( 7.45%) 451937.00 ( 8.78%) SpecJBB Bops/JVM 103865.00 ( 0.00%) 96332.00 ( -7.25%) 104049.00 ( 0.18%) 138614.00 ( 33.46%) 111601.00 ( 7.45%) 112984.00 ( 8.78%) Same conclusions. numacore regresses, autonuma is best, balancenuma does all right with the migration scalability patches helping a little. SpecJBB, Single JVM, THP is enabled 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 TPut 1 25550.00 ( 0.00%) 25491.00 ( -0.23%) 26438.00 ( 3.48%) 24233.00 ( -5.15%) 24913.00 ( -2.49%) 26480.00 ( 3.64%) TPut 2 55943.00 ( 0.00%) 51630.00 ( -7.71%) 57004.00 ( 1.90%) 55312.00 ( -1.13%) 55042.00 ( -1.61%) 56920.00 ( 1.75%) TPut 3 87707.00 ( 0.00%) 74497.00 (-15.06%) 88852.00 ( 1.31%) 88569.00 ( 0.98%) 86135.00 ( -1.79%) 88608.00 ( 1.03%) TPut 4 117911.00 ( 0.00%) 98435.00 (-16.52%) 104955.00 (-10.99%) 118561.00 ( 0.55%) 117486.00 ( -0.36%) 117953.00 ( 0.04%) TPut 5 143285.00 ( 0.00%) 133964.00 ( -6.51%) 126238.00 (-11.90%) 145703.00 ( 1.69%) 142821.00 ( -0.32%) 144926.00 ( 1.15%) TPut 6 171208.00 ( 0.00%) 152795.00 (-10.75%) 160028.00 ( -6.53%) 171006.00 ( -0.12%) 170635.00 ( -0.33%) 169394.00 ( -1.06%) TPut 7 195635.00 ( 0.00%) 162517.00 (-16.93%) 172973.00 (-11.58%) 198699.00 ( 1.57%) 196108.00 ( 0.24%) 196491.00 ( 0.44%) TPut 8 222655.00 ( 0.00%) 168679.00 (-24.24%) 179260.00 (-19.49%) 224903.00 ( 1.01%) 223494.00 ( 0.38%) 225978.00 ( 1.49%) TPut 9 244787.00 ( 0.00%) 193394.00 (-20.99%) 238823.00 ( -2.44%) 248313.00 ( 1.44%) 251858.00 ( 2.89%) 251569.00 ( 2.77%) TPut 10 271565.00 ( 0.00%) 237987.00 (-12.36%) 247724.00 ( -8.78%) 272148.00 ( 0.21%) 275869.00 ( 1.58%) 279049.00 ( 2.76%) TPut 11 298270.00 ( 0.00%) 207908.00 (-30.30%) 277513.00 ( -6.96%) 303749.00 ( 1.84%) 301763.00 ( 1.17%) 301399.00 ( 1.05%) TPut 12 320867.00 ( 0.00%) 257937.00 (-19.61%) 281723.00 (-12.20%) 327808.00 ( 2.16%) 329681.00 ( 2.75%) 330506.00 ( 3.00%) TPut 13 343514.00 ( 0.00%) 248474.00 (-27.67%) 301710.00 (-12.17%) 349080.00 ( 1.62%) 340606.00 ( -0.85%) 350817.00 ( 2.13%) TPut 14 365321.00 ( 0.00%) 298876.00 (-18.19%) 314066.00 (-14.03%) 370026.00 ( 1.29%) 379939.00 ( 4.00%) 361752.00 ( -0.98%) TPut 15 377071.00 ( 0.00%) 296562.00 (-21.35%) 334810.00 (-11.21%) 329847.00 (-12.52%) 395421.00 ( 4.87%) 396091.00 ( 5.04%) TPut 16 404979.00 ( 0.00%) 287964.00 (-28.89%) 347142.00 (-14.28%) 411066.00 ( 1.50%) 420551.00 ( 3.85%) 411673.00 ( 1.65%) TPut 17 420593.00 ( 0.00%) 342590.00 (-18.55%) 352738.00 (-16.13%) 428242.00 ( 1.82%) 437461.00 ( 4.01%) 428270.00 ( 1.83%) TPut 18 440178.00 ( 0.00%) 377508.00 (-14.24%) 344421.00 (-21.75%) 440392.00 ( 0.05%) 455014.00 ( 3.37%) 447671.00 ( 1.70%) TPut 19 448876.00 ( 0.00%) 397727.00 (-11.39%) 367002.00 (-18.24%) 462036.00 ( 2.93%) 479223.00 ( 6.76%) 461881.00 ( 2.90%) TPut 20 460513.00 ( 0.00%) 411831.00 (-10.57%) 370870.00 (-19.47%) 476437.00 ( 3.46%) 493176.00 ( 7.09%) 474824.00 ( 3.11%) TPut 21 474161.00 ( 0.00%) 442153.00 ( -6.75%) 374835.00 (-20.95%) 487513.00 ( 2.82%) 505246.00 ( 6.56%) 468938.00 ( -1.10%) TPut 22 474493.00 ( 0.00%) 429921.00 ( -9.39%) 371022.00 (-21.81%) 487920.00 ( 2.83%) 527360.00 ( 11.14%) 475208.00 ( 0.15%) TPut 23 489559.00 ( 0.00%) 460354.00 ( -5.97%) 377444.00 (-22.90%) 508298.00 ( 3.83%) 534820.00 ( 9.25%) 490743.00 ( 0.24%) TPut 24 495378.00 ( 0.00%) 486826.00 ( -1.73%) 376551.00 (-23.99%) 514403.00 ( 3.84%) 545294.00 ( 10.08%) 493974.00 ( -0.28%) TPut 25 491795.00 ( 0.00%) 520474.00 ( 5.83%) 370872.00 (-24.59%) 507373.00 ( 3.17%) 543526.00 ( 10.52%) 489850.00 ( -0.40%) TPut 26 490038.00 ( 0.00%) 465587.00 ( -4.99%) 370093.00 (-24.48%) 376322.00 (-23.21%) 545175.00 ( 11.25%) 491352.00 ( 0.27%) TPut 27 491233.00 ( 0.00%) 469764.00 ( -4.37%) 371915.00 (-24.29%) 366225.00 (-25.45%) 536927.00 ( 9.30%) 489611.00 ( -0.33%) TPut 28 489058.00 ( 0.00%) 489561.00 ( 0.10%) 364465.00 (-25.48%) 414027.00 (-15.34%) 543127.00 ( 11.06%) 473835.00 ( -3.11%) TPut 29 471539.00 ( 0.00%) 492496.00 ( 4.44%) 353470.00 (-25.04%) 400529.00 (-15.06%) 541615.00 ( 14.86%) 486009.00 ( 3.07%) TPut 30 480343.00 ( 0.00%) 488349.00 ( 1.67%) 355023.00 (-26.09%) 405612.00 (-15.56%) 542904.00 ( 13.02%) 478384.00 ( -0.41%) TPut 31 478109.00 ( 0.00%) 460043.00 ( -3.78%) 352440.00 (-26.28%) 401471.00 (-16.03%) 529079.00 ( 10.66%) 466457.00 ( -2.44%) TPut 32 475736.00 ( 0.00%) 472007.00 ( -0.78%) 341509.00 (-28.21%) 401075.00 (-15.69%) 532423.00 ( 11.92%) 467866.00 ( -1.65%) TPut 33 470758.00 ( 0.00%) 474348.00 ( 0.76%) 337127.00 (-28.39%) 399592.00 (-15.12%) 518811.00 ( 10.21%) 464764.00 ( -1.27%) TPut 34 467304.00 ( 0.00%) 475878.00 ( 1.83%) 332477.00 (-28.85%) 394589.00 (-15.56%) 518334.00 ( 10.92%) 446719.00 ( -4.41%) TPut 35 466391.00 ( 0.00%) 487411.00 ( 4.51%) 335639.00 (-28.03%) 382799.00 (-17.92%) 513591.00 ( 10.12%) 447071.00 ( -4.14%) TPut 36 452722.00 ( 0.00%) 478050.00 ( 5.59%) 316889.00 (-30.00%) 381120.00 (-15.82%) 503801.00 ( 11.28%) 452243.00 ( -0.11%) TPut 37 447878.00 ( 0.00%) 478467.00 ( 6.83%) 326939.00 (-27.00%) 382803.00 (-14.53%) 494555.00 ( 10.42%) 442751.00 ( -1.14%) TPut 38 447907.00 ( 0.00%) 455542.00 ( 1.70%) 315719.00 (-29.51%) 341693.00 (-23.71%) 482758.00 ( 7.78%) 444023.00 ( -0.87%) TPut 39 428322.00 ( 0.00%) 367921.00 (-14.10%) 310519.00 (-27.50%) 404210.00 ( -5.63%) 464550.00 ( 8.46%) 440482.00 ( 2.84%) TPut 40 429157.00 ( 0.00%) 394277.00 ( -8.13%) 302742.00 (-29.46%) 378554.00 (-11.79%) 467767.00 ( 9.00%) 411807.00 ( -4.04%) TPut 41 424339.00 ( 0.00%) 415413.00 ( -2.10%) 304680.00 (-28.20%) 399220.00 ( -5.92%) 457669.00 ( 7.85%) 428273.00 ( 0.93%) TPut 42 397440.00 ( 0.00%) 421027.00 ( 5.93%) 298298.00 (-24.95%) 372161.00 ( -6.36%) 458156.00 ( 15.28%) 422535.00 ( 6.31%) TPut 43 405391.00 ( 0.00%) 433900.00 ( 7.03%) 286294.00 (-29.38%) 383936.00 ( -5.29%) 438929.00 ( 8.27%) 410196.00 ( 1.19%) TPut 44 400692.00 ( 0.00%) 427504.00 ( 6.69%) 282819.00 (-29.42%) 374757.00 ( -6.47%) 423538.00 ( 5.70%) 399471.00 ( -0.30%) TPut 45 399623.00 ( 0.00%) 372622.00 ( -6.76%) 273593.00 (-31.54%) 379797.00 ( -4.96%) 407255.00 ( 1.91%) 374068.00 ( -6.39%) TPut 46 391920.00 ( 0.00%) 351205.00 (-10.39%) 277380.00 (-29.23%) 368042.00 ( -6.09%) 411353.00 ( 4.96%) 384363.00 ( -1.93%) TPut 47 378199.00 ( 0.00%) 358150.00 ( -5.30%) 273560.00 (-27.67%) 368744.00 ( -2.50%) 408739.00 ( 8.08%) 385670.00 ( 1.98%) TPut 48 379346.00 ( 0.00%) 387287.00 ( 2.09%) 274168.00 (-27.73%) 373581.00 ( -1.52%) 423791.00 ( 11.72%) 380665.00 ( 0.35%) TPut 49 373614.00 ( 0.00%) 395793.00 ( 5.94%) 270794.00 (-27.52%) 372621.00 ( -0.27%) 423024.00 ( 13.22%) 377985.00 ( 1.17%) TPut 50 372494.00 ( 0.00%) 366488.00 ( -1.61%) 271465.00 (-27.12%) 388778.00 ( 4.37%) 410647.00 ( 10.24%) 378831.00 ( 1.70%) TPut 51 382195.00 ( 0.00%) 381771.00 ( -0.11%) 272796.00 (-28.62%) 387687.00 ( 1.44%) 423249.00 ( 10.74%) 402233.00 ( 5.24%) TPut 52 369118.00 ( 0.00%) 429441.00 ( 16.34%) 272019.00 (-26.31%) 390226.00 ( 5.72%) 410023.00 ( 11.08%) 396558.00 ( 7.43%) TPut 53 366453.00 ( 0.00%) 445744.00 ( 21.64%) 267952.00 (-26.88%) 399257.00 ( 8.95%) 405937.00 ( 10.77%) 383916.00 ( 4.77%) TPut 54 366571.00 ( 0.00%) 375762.00 ( 2.51%) 268229.00 (-26.83%) 395098.00 ( 7.78%) 402220.00 ( 9.72%) 395417.00 ( 7.87%) TPut 55 367580.00 ( 0.00%) 336113.00 ( -8.56%) 267474.00 (-27.23%) 400550.00 ( 8.97%) 420978.00 ( 14.53%) 398098.00 ( 8.30%) TPut 56 367056.00 ( 0.00%) 375635.00 ( 2.34%) 263577.00 (-28.19%) 385743.00 ( 5.09%) 412685.00 ( 12.43%) 384029.00 ( 4.62%) TPut 57 359163.00 ( 0.00%) 354001.00 ( -1.44%) 261130.00 (-27.29%) 389827.00 ( 8.54%) 394688.00 ( 9.89%) 381032.00 ( 6.09%) TPut 58 360552.00 ( 0.00%) 353312.00 ( -2.01%) 261140.00 (-27.57%) 394099.00 ( 9.30%) 388655.00 ( 7.79%) 378132.00 ( 4.88%) TPut 59 354967.00 ( 0.00%) 368534.00 ( 3.82%) 262418.00 (-26.07%) 390746.00 ( 10.08%) 399086.00 ( 12.43%) 387101.00 ( 9.05%) TPut 60 362976.00 ( 0.00%) 388472.00 ( 7.02%) 267468.00 (-26.31%) 383073.00 ( 5.54%) 399713.00 ( 10.12%) 390635.00 ( 7.62%) TPut 61 368072.00 ( 0.00%) 399476.00 ( 8.53%) 265659.00 (-27.82%) 380807.00 ( 3.46%) 372060.00 ( 1.08%) 383187.00 ( 4.11%) TPut 62 356938.00 ( 0.00%) 385648.00 ( 8.04%) 253107.00 (-29.09%) 387736.00 ( 8.63%) 377183.00 ( 5.67%) 378484.00 ( 6.04%) TPut 63 357491.00 ( 0.00%) 404325.00 ( 13.10%) 259404.00 (-27.44%) 396672.00 ( 10.96%) 384221.00 ( 7.48%) 378907.00 ( 5.99%) TPut 64 357322.00 ( 0.00%) 389552.00 ( 9.02%) 260333.00 (-27.14%) 386826.00 ( 8.26%) 378601.00 ( 5.96%) 369852.00 ( 3.51%) TPut 65 341262.00 ( 0.00%) 394964.00 ( 15.74%) 258149.00 (-24.35%) 380271.00 ( 11.43%) 382896.00 ( 12.20%) 382897.00 ( 12.20%) TPut 66 357807.00 ( 0.00%) 384846.00 ( 7.56%) 259279.00 (-27.54%) 362723.00 ( 1.37%) 361530.00 ( 1.04%) 380023.00 ( 6.21%) TPut 67 345092.00 ( 0.00%) 376842.00 ( 9.20%) 259350.00 (-24.85%) 364193.00 ( 5.54%) 374449.00 ( 8.51%) 373877.00 ( 8.34%) TPut 68 350334.00 ( 0.00%) 358330.00 ( 2.28%) 259332.00 (-25.98%) 359368.00 ( 2.58%) 384920.00 ( 9.87%) 381888.00 ( 9.01%) TPut 69 348372.00 ( 0.00%) 356188.00 ( 2.24%) 263076.00 (-24.48%) 364449.00 ( 4.61%) 395611.00 ( 13.56%) 375892.00 ( 7.90%) TPut 70 335077.00 ( 0.00%) 359313.00 ( 7.23%) 259983.00 (-22.41%) 356418.00 ( 6.37%) 375448.00 ( 12.05%) 372358.00 ( 11.13%) TPut 71 341197.00 ( 0.00%) 364168.00 ( 6.73%) 254622.00 (-25.37%) 343847.00 ( 0.78%) 376113.00 ( 10.23%) 384292.00 ( 12.63%) TPut 72 345032.00 ( 0.00%) 356934.00 ( 3.45%) 261060.00 (-24.34%) 345007.00 ( -0.01%) 375313.00 ( 8.78%) 381504.00 ( 10.57%) numacore-20121130 (v17) did well but numacore-20121202 (v18) does not as it's effectively disabled. autonuma does ok here and balancenuma does quite well. As before, the migration scalability patches help in some cases. SPECJBB PEAKS 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 Expctd Warehouse 48.00 ( 0.00%) 48.00 ( 0.00%) 48.00 ( 0.00%) 48.00 ( 0.00%) 48.00 ( 0.00%) 48.00 ( 0.00%) Expctd Peak Bops 379346.00 ( 0.00%) 387287.00 ( 2.09%) 274168.00 (-27.73%) 373581.00 ( -1.52%) 423791.00 ( 11.72%) 380665.00 ( 0.35%) Actual Warehouse 24.00 ( 0.00%) 25.00 ( 4.17%) 23.00 ( -4.17%) 24.00 ( 0.00%) 24.00 ( 0.00%) 24.00 ( 0.00%) Actual Peak Bops 495378.00 ( 0.00%) 520474.00 ( 5.07%) 377444.00 (-23.81%) 514403.00 ( 3.84%) 545294.00 ( 10.08%) 493974.00 ( -0.28%) SpecJBB Bops 183389.00 ( 0.00%) 193652.00 ( 5.60%) 134571.00 (-26.62%) 193461.00 ( 5.49%) 201083.00 ( 9.65%) 195465.00 ( 6.58%) SpecJBB Bops/JVM 183389.00 ( 0.00%) 193652.00 ( 5.60%) 134571.00 (-26.62%) 193461.00 ( 5.49%) 201083.00 ( 9.65%) 195465.00 ( 6.58%) While the migration patches appear to help in some cases note that overall it is not a win in this case. The peak scores are hurt as is the specjbb score. MMTests Statistics: duration 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6numacore-20121130numacore-20121202autonuma-v28fastr4balancenuma-v9r2balancenuma-v10r3 User 316340.52 311420.23 317791.75 314589.64 316061.23 315584.37 System 102.08 3067.27 102.89 352.70 428.76 450.71 Elapsed 7433.22 7436.63 7434.49 7434.74 7432.60 7435.03 Same comments about System CPU time. numacore v17 is very high, v18 is low because it's disabled, balancenuma is higher than I'd like. SpecJBB, Single JVM, THP disabled 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 TPut 1 19861.00 ( 0.00%) 18255.00 ( -8.09%) 21307.00 ( 7.28%) 19636.00 ( -1.13%) 19838.00 ( -0.12%) 20650.00 ( 3.97%) TPut 2 47613.00 ( 0.00%) 37136.00 (-22.00%) 47861.00 ( 0.52%) 47153.00 ( -0.97%) 47481.00 ( -0.28%) 48199.00 ( 1.23%) TPut 3 72438.00 ( 0.00%) 55692.00 (-23.12%) 72271.00 ( -0.23%) 69394.00 ( -4.20%) 72029.00 ( -0.56%) 72932.00 ( 0.68%) TPut 4 98455.00 ( 0.00%) 81301.00 (-17.42%) 91079.00 ( -7.49%) 98577.00 ( 0.12%) 98437.00 ( -0.02%) 99748.00 ( 1.31%) TPut 5 120831.00 ( 0.00%) 89067.00 (-26.29%) 118381.00 ( -2.03%) 120805.00 ( -0.02%) 117218.00 ( -2.99%) 121254.00 ( 0.35%) TPut 6 140013.00 ( 0.00%) 108349.00 (-22.62%) 141994.00 ( 1.41%) 125079.00 (-10.67%) 139878.00 ( -0.10%) 145360.00 ( 3.82%) TPut 7 163553.00 ( 0.00%) 116192.00 (-28.96%) 133084.00 (-18.63%) 164368.00 ( 0.50%) 167133.00 ( 2.19%) 169539.00 ( 3.66%) TPut 8 190148.00 ( 0.00%) 125955.00 (-33.76%) 177239.00 ( -6.79%) 188906.00 ( -0.65%) 183058.00 ( -3.73%) 188936.00 ( -0.64%) TPut 9 211343.00 ( 0.00%) 144068.00 (-31.83%) 180903.00 (-14.40%) 206645.00 ( -2.22%) 205699.00 ( -2.67%) 217322.00 ( 2.83%) TPut 10 233190.00 ( 0.00%) 148098.00 (-36.49%) 215595.00 ( -7.55%) 234533.00 ( 0.58%) 233632.00 ( 0.19%) 227292.00 ( -2.53%) TPut 11 253333.00 ( 0.00%) 146043.00 (-42.35%) 224514.00 (-11.38%) 254167.00 ( 0.33%) 251938.00 ( -0.55%) 259924.00 ( 2.60%) TPut 12 270661.00 ( 0.00%) 131739.00 (-51.33%) 245812.00 ( -9.18%) 271490.00 ( 0.31%) 271393.00 ( 0.27%) 272536.00 ( 0.69%) TPut 13 299807.00 ( 0.00%) 169396.00 (-43.50%) 253075.00 (-15.59%) 299758.00 ( -0.02%) 270594.00 ( -9.74%) 299110.00 ( -0.23%) TPut 14 319243.00 ( 0.00%) 150705.00 (-52.79%) 256078.00 (-19.79%) 318481.00 ( -0.24%) 318566.00 ( -0.21%) 325133.00 ( 1.84%) TPut 15 339054.00 ( 0.00%) 116872.00 (-65.53%) 268646.00 (-20.77%) 331534.00 ( -2.22%) 344672.00 ( 1.66%) 318119.00 ( -6.17%) TPut 16 354315.00 ( 0.00%) 124346.00 (-64.91%) 291148.00 (-17.83%) 352600.00 ( -0.48%) 316761.00 (-10.60%) 364648.00 ( 2.92%) TPut 17 371306.00 ( 0.00%) 118493.00 (-68.09%) 299399.00 (-19.37%) 368260.00 ( -0.82%) 328888.00 (-11.42%) 371088.00 ( -0.06%) TPut 18 386361.00 ( 0.00%) 138571.00 (-64.13%) 303185.00 (-21.53%) 374358.00 ( -3.11%) 356148.00 ( -7.82%) 399913.00 ( 3.51%) TPut 19 401827.00 ( 0.00%) 118855.00 (-70.42%) 320630.00 (-20.21%) 399476.00 ( -0.59%) 393918.00 ( -1.97%) 405771.00 ( 0.98%) TPut 20 411130.00 ( 0.00%) 144024.00 (-64.97%) 315391.00 (-23.29%) 407799.00 ( -0.81%) 377706.00 ( -8.13%) 406038.00 ( -1.24%) TPut 21 425352.00 ( 0.00%) 154264.00 (-63.73%) 326734.00 (-23.19%) 429226.00 ( 0.91%) 431677.00 ( 1.49%) 431583.00 ( 1.46%) TPut 22 438150.00 ( 0.00%) 153892.00 (-64.88%) 329531.00 (-24.79%) 385827.00 (-11.94%) 440379.00 ( 0.51%) 438861.00 ( 0.16%) TPut 23 438425.00 ( 0.00%) 146506.00 (-66.58%) 336454.00 (-23.26%) 433963.00 ( -1.02%) 361427.00 (-17.56%) 445293.00 ( 1.57%) TPut 24 461598.00 ( 0.00%) 138869.00 (-69.92%) 330113.00 (-28.48%) 439691.00 ( -4.75%) 471567.00 ( 2.16%) 488259.00 ( 5.78%) TPut 25 459475.00 ( 0.00%) 141698.00 (-69.16%) 333545.00 (-27.41%) 431373.00 ( -6.12%) 487921.00 ( 6.19%) 447353.00 ( -2.64%) TPut 26 452651.00 ( 0.00%) 142844.00 (-68.44%) 325634.00 (-28.06%) 447517.00 ( -1.13%) 425336.00 ( -6.03%) 469793.00 ( 3.79%) TPut 27 450436.00 ( 0.00%) 140870.00 (-68.73%) 324881.00 (-27.87%) 430805.00 ( -4.36%) 456114.00 ( 1.26%) 461172.00 ( 2.38%) TPut 28 459770.00 ( 0.00%) 143078.00 (-68.88%) 312547.00 (-32.02%) 432260.00 ( -5.98%) 478317.00 ( 4.03%) 452144.00 ( -1.66%) TPut 29 450347.00 ( 0.00%) 142076.00 (-68.45%) 318785.00 (-29.21%) 440423.00 ( -2.20%) 388175.00 (-13.81%) 473273.00 ( 5.09%) TPut 30 449252.00 ( 0.00%) 146900.00 (-67.30%) 310301.00 (-30.93%) 435082.00 ( -3.15%) 440795.00 ( -1.88%) 435189.00 ( -3.13%) TPut 31 446802.00 ( 0.00%) 148008.00 (-66.87%) 304119.00 (-31.93%) 418684.00 ( -6.29%) 417343.00 ( -6.59%) 437562.00 ( -2.07%) TPut 32 439701.00 ( 0.00%) 149591.00 (-65.98%) 297625.00 (-32.31%) 421866.00 ( -4.06%) 438719.00 ( -0.22%) 469763.00 ( 6.84%) TPut 33 434477.00 ( 0.00%) 142801.00 (-67.13%) 293405.00 (-32.47%) 420631.00 ( -3.19%) 454673.00 ( 4.65%) 451224.00 ( 3.85%) TPut 34 423014.00 ( 0.00%) 152308.00 (-63.99%) 288639.00 (-31.77%) 415202.00 ( -1.85%) 415194.00 ( -1.85%) 446735.00 ( 5.61%) TPut 35 429012.00 ( 0.00%) 154116.00 (-64.08%) 283797.00 (-33.85%) 402395.00 ( -6.20%) 425151.00 ( -0.90%) 434230.00 ( 1.22%) TPut 36 421097.00 ( 0.00%) 157571.00 (-62.58%) 276038.00 (-34.45%) 404770.00 ( -3.88%) 430480.00 ( 2.23%) 425324.00 ( 1.00%) TPut 37 414815.00 ( 0.00%) 150771.00 (-63.65%) 272498.00 (-34.31%) 388842.00 ( -6.26%) 393351.00 ( -5.17%) 405824.00 ( -2.17%) TPut 38 412361.00 ( 0.00%) 157070.00 (-61.91%) 270972.00 (-34.29%) 398947.00 ( -3.25%) 401555.00 ( -2.62%) 432074.00 ( 4.78%) TPut 39 402234.00 ( 0.00%) 161487.00 (-59.85%) 258636.00 (-35.70%) 382645.00 ( -4.87%) 423106.00 ( 5.19%) 401091.00 ( -0.28%) TPut 40 380278.00 ( 0.00%) 165947.00 (-56.36%) 256492.00 (-32.55%) 394039.00 ( 3.62%) 405371.00 ( 6.60%) 410739.00 ( 8.01%) TPut 41 393204.00 ( 0.00%) 160540.00 (-59.17%) 254896.00 (-35.17%) 385605.00 ( -1.93%) 403383.00 ( 2.59%) 372466.00 ( -5.27%) TPut 42 380622.00 ( 0.00%) 151946.00 (-60.08%) 248167.00 (-34.80%) 374843.00 ( -1.52%) 380797.00 ( 0.05%) 396227.00 ( 4.10%) TPut 43 371566.00 ( 0.00%) 162369.00 (-56.30%) 238268.00 (-35.87%) 347951.00 ( -6.36%) 386765.00 ( 4.09%) 345633.00 ( -6.98%) TPut 44 365538.00 ( 0.00%) 161127.00 (-55.92%) 239926.00 (-34.36%) 355070.00 ( -2.86%) 344701.00 ( -5.70%) 391276.00 ( 7.04%) TPut 45 359305.00 ( 0.00%) 159062.00 (-55.73%) 237676.00 (-33.85%) 350973.00 ( -2.32%) 370666.00 ( 3.16%) 331191.00 ( -7.82%) TPut 46 343160.00 ( 0.00%) 163889.00 (-52.24%) 231272.00 (-32.61%) 347960.00 ( 1.40%) 380147.00 ( 10.78%) 323176.00 ( -5.82%) TPut 47 346983.00 ( 0.00%) 168666.00 (-51.39%) 228060.00 (-34.27%) 313612.00 ( -9.62%) 362189.00 ( 4.38%) 343154.00 ( -1.10%) TPut 48 338143.00 ( 0.00%) 153448.00 (-54.62%) 224598.00 (-33.58%) 341809.00 ( 1.08%) 365342.00 ( 8.04%) 354348.00 ( 4.79%) TPut 49 333941.00 ( 0.00%) 142784.00 (-57.24%) 224568.00 (-32.75%) 336174.00 ( 0.67%) 371700.00 ( 11.31%) 353148.00 ( 5.75%) TPut 50 334001.00 ( 0.00%) 135713.00 (-59.37%) 221381.00 (-33.72%) 322489.00 ( -3.45%) 367963.00 ( 10.17%) 355823.00 ( 6.53%) TPut 51 338310.00 ( 0.00%) 133402.00 (-60.57%) 219870.00 (-35.01%) 354805.00 ( 4.88%) 372592.00 ( 10.13%) 351194.00 ( 3.81%) TPut 52 322897.00 ( 0.00%) 150293.00 (-53.45%) 217427.00 (-32.66%) 353169.00 ( 9.38%) 363024.00 ( 12.43%) 344846.00 ( 6.80%) TPut 53 329801.00 ( 0.00%) 160792.00 (-51.25%) 224019.00 (-32.07%) 353588.00 ( 7.21%) 365359.00 ( 10.78%) 355499.00 ( 7.79%) TPut 54 336610.00 ( 0.00%) 164696.00 (-51.07%) 214752.00 (-36.20%) 361189.00 ( 7.30%) 377851.00 ( 12.25%) 363987.00 ( 8.13%) TPut 55 325920.00 ( 0.00%) 172380.00 (-47.11%) 219529.00 (-32.64%) 365678.00 ( 12.20%) 375735.00 ( 15.28%) 363697.00 ( 11.59%) TPut 56 318997.00 ( 0.00%) 176071.00 (-44.80%) 218120.00 (-31.62%) 367048.00 ( 15.06%) 380588.00 ( 19.31%) 362614.00 ( 13.67%) TPut 57 321776.00 ( 0.00%) 174531.00 (-45.76%) 214685.00 (-33.28%) 341874.00 ( 6.25%) 378996.00 ( 17.78%) 360366.00 ( 11.99%) TPut 58 308532.00 ( 0.00%) 174202.00 (-43.54%) 208226.00 (-32.51%) 348156.00 ( 12.84%) 361623.00 ( 17.21%) 369693.00 ( 19.82%) TPut 59 318974.00 ( 0.00%) 175343.00 (-45.03%) 214260.00 (-32.83%) 358252.00 ( 12.31%) 360457.00 ( 13.01%) 364556.00 ( 14.29%) TPut 60 325465.00 ( 0.00%) 173694.00 (-46.63%) 213290.00 (-34.47%) 360808.00 ( 10.86%) 362745.00 ( 11.45%) 354232.00 ( 8.84%) TPut 61 319151.00 ( 0.00%) 172320.00 (-46.01%) 206197.00 (-35.39%) 350597.00 ( 9.85%) 371277.00 ( 16.33%) 352478.00 ( 10.44%) TPut 62 320837.00 ( 0.00%) 172312.00 (-46.29%) 211186.00 (-34.18%) 359062.00 ( 11.91%) 361009.00 ( 12.52%) 352930.00 ( 10.00%) TPut 63 318198.00 ( 0.00%) 172297.00 (-45.85%) 215174.00 (-32.38%) 356137.00 ( 11.92%) 347637.00 ( 9.25%) 335322.00 ( 5.38%) TPut 64 321438.00 ( 0.00%) 171894.00 (-46.52%) 212493.00 (-33.89%) 347376.00 ( 8.07%) 346756.00 ( 7.88%) 351410.00 ( 9.32%) TPut 65 314482.00 ( 0.00%) 169147.00 (-46.21%) 204809.00 (-34.87%) 351726.00 ( 11.84%) 357429.00 ( 13.66%) 351236.00 ( 11.69%) TPut 66 316802.00 ( 0.00%) 170234.00 (-46.26%) 199708.00 (-36.96%) 344548.00 ( 8.76%) 362143.00 ( 14.31%) 347058.00 ( 9.55%) TPut 67 312139.00 ( 0.00%) 168180.00 (-46.12%) 208644.00 (-33.16%) 329030.00 ( 5.41%) 353305.00 ( 13.19%) 345903.00 ( 10.82%) TPut 68 323918.00 ( 0.00%) 168392.00 (-48.01%) 206120.00 (-36.37%) 319985.00 ( -1.21%) 344250.00 ( 6.28%) 345703.00 ( 6.73%) TPut 69 307506.00 ( 0.00%) 167082.00 (-45.67%) 204703.00 (-33.43%) 340673.00 ( 10.79%) 339346.00 ( 10.35%) 336071.00 ( 9.29%) TPut 70 306799.00 ( 0.00%) 165764.00 (-45.97%) 201529.00 (-34.31%) 331678.00 ( 8.11%) 349583.00 ( 13.95%) 341944.00 ( 11.46%) TPut 71 304232.00 ( 0.00%) 165289.00 (-45.67%) 203291.00 (-33.18%) 319824.00 ( 5.13%) 335238.00 ( 10.19%) 343396.00 ( 12.87%) TPut 72 301619.00 ( 0.00%) 163909.00 (-45.66%) 203306.00 (-32.60%) 326875.00 ( 8.37%) 345999.00 ( 14.71%) 343949.00 ( 14.03%) numacore does really badly. autonuma is ok and balancenuma is ok. Scalability patches do not help as much. SPECJBB PEAKS 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6 numacore-20121130 numacore-20121202 autonuma-v28fastr4 balancenuma-v9r2 balancenuma-v10r3 Expctd Warehouse 48.00 ( 0.00%) 48.00 ( 0.00%) 48.00 ( 0.00%) 48.00 ( 0.00%) 48.00 ( 0.00%) 48.00 ( 0.00%) Expctd Peak Bops 338143.00 ( 0.00%) 153448.00 (-54.62%) 224598.00 (-33.58%) 341809.00 ( 1.08%) 365342.00 ( 8.04%) 354348.00 ( 4.79%) Actual Warehouse 24.00 ( 0.00%) 56.00 (133.33%) 23.00 ( -4.17%) 26.00 ( 8.33%) 25.00 ( 4.17%) 24.00 ( 0.00%) Actual Peak Bops 461598.00 ( 0.00%) 176071.00 (-61.86%) 336454.00 (-27.11%) 447517.00 ( -3.05%) 487921.00 ( 5.70%) 488259.00 ( 5.78%) SpecJBB Bops 163683.00 ( 0.00%) 83963.00 (-48.70%) 108406.00 (-33.77%) 176379.00 ( 7.76%) 184040.00 ( 12.44%) 179621.00 ( 9.74%) SpecJBB Bops/JVM 163683.00 ( 0.00%) 83963.00 (-48.70%) 108406.00 (-33.77%) 176379.00 ( 7.76%) 184040.00 ( 12.44%) 179621.00 ( 9.74%) balancenuma is doing reasonably well. It's interesting to note that the scalabilty patches make little difference to the peak but there is a difference in the specjbb score. It's veryliekly this is just variance. balancenuma depends on "luck" on what decisions the scheduler makes. MMTests Statistics: duration 3.7.0-rc7 3.7.0-rc6 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 3.7.0-rc7 stats-v8r6numacore-20121130numacore-20121202autonuma-v28fastr4balancenuma-v9r2balancenuma-v10r3 User 316751.91 167098.56 318360.63 307598.67 309109.47 313644.48 System 60.28 122511.08 59.60 4411.81 1820.70 2654.77 Elapsed 7434.08 7451.36 7436.99 7437.52 7438.28 7438.19 numacores CPU usage is off the charts except in v18 where it's more or less disabled. balancenuma has very high CPU usage too unfortunately. Overall, balancenuma is not the known best kernel but it wans't meant to be. It was meant to be better than mainline, establish a performance baseline and be something that either numacore or autonuma can be rebased upon. I think it achieves that and is the best choice for 3.8. Documentation/kernel-parameters.txt | 3 + arch/sh/mm/Kconfig | 1 + arch/x86/Kconfig | 2 + arch/x86/include/asm/pgtable.h | 17 +- arch/x86/include/asm/pgtable_types.h | 20 ++ arch/x86/mm/pgtable.c | 8 +- include/asm-generic/pgtable.h | 110 +++++++++++ include/linux/huge_mm.h | 16 +- include/linux/hugetlb.h | 8 +- include/linux/mempolicy.h | 8 + include/linux/migrate.h | 47 ++++- include/linux/mm.h | 39 ++++ include/linux/mm_types.h | 31 ++++ include/linux/mmzone.h | 13 ++ include/linux/rmap.h | 33 ++-- include/linux/sched.h | 27 +++ include/linux/vm_event_item.h | 12 +- include/linux/vmstat.h | 8 + include/trace/events/migrate.h | 51 +++++ include/uapi/linux/mempolicy.h | 15 +- init/Kconfig | 41 +++++ kernel/fork.c | 3 + kernel/sched/core.c | 71 +++++-- kernel/sched/fair.c | 227 +++++++++++++++++++++++ kernel/sched/features.h | 11 ++ kernel/sched/sched.h | 12 ++ kernel/sysctl.c | 45 ++++- mm/compaction.c | 15 +- mm/huge_memory.c | 105 ++++++++++- mm/hugetlb.c | 10 +- mm/internal.h | 7 +- mm/ksm.c | 6 +- mm/memcontrol.c | 7 +- mm/memory-failure.c | 7 +- mm/memory.c | 196 +++++++++++++++++++- mm/memory_hotplug.c | 3 +- mm/mempolicy.c | 283 +++++++++++++++++++++++++--- mm/migrate.c | 337 +++++++++++++++++++++++++++++++++- mm/mmap.c | 10 +- mm/mprotect.c | 135 +++++++++++--- mm/mremap.c | 2 +- mm/page_alloc.c | 10 +- mm/pgtable-generic.c | 9 +- mm/rmap.c | 66 +++---- mm/vmstat.c | 16 +- 45 files changed, 1932 insertions(+), 171 deletions(-) create mode 100644 include/trace/events/migrate.h -- 1.7.9.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>