On 11/23/2015 04:16 PM, Joonsoo Kim wrote: > Numbers looks fine to me. I guess this performance degradation is > caused by COMPACT_CLUSTER_MAX change (from 32 to 256). THP allocation > is async so should be aborted quickly. But, after isolating 256 > migratable pages, it can't be aborted and will finish 256 pages > migration (at least, current implementation). > > Aaron, please test again with setting COMPACT_CLUSTER_MAX to 32 > (in swap.h)? This is what I found in include/linux/swap.h: #define SWAP_CLUSTER_MAX 32UL #define COMPACT_CLUSTER_MAX SWAP_CLUSTER_MAX Looks like it is already 32, or am I looking at the wrong place? BTW, I'm using v4.3 for all these tests, and I just checked v4.4-rc2, the above definition doesn't change. > > And, please attach always-always's vmstat numbers, too. Sure, attached the vmstat tool output, taken every second. Thanks, Aaron
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- -----timestamp----- r b swpd free buff cache si so bi bo in cs us sy id wa st CST 6 1 0 25647504 580 626540 0 0 0 0 66 19 0 1 99 0 0 2015-11-20 02:19:37 1 0 0 25563796 580 638000 0 0 0 0 769 6085 0 1 99 0 0 2015-11-20 02:19:38 1 0 0 22010336 580 638168 0 0 0 0 1698 930 0 1 99 0 0 2015-11-20 02:19:39 1 0 0 18589868 580 638084 0 0 0 0 1198 793 0 1 99 0 0 2015-11-20 02:19:40 1 0 0 15173252 580 638104 0 0 0 0 1234 738 0 1 99 0 0 2015-11-20 02:19:41 1 0 0 11751756 580 638120 0 0 0 0 1224 679 0 1 99 0 0 2015-11-20 02:19:42 1 0 0 8322416 580 638156 0 0 0 0 1213 726 0 1 99 0 0 2015-11-20 02:19:43 1 0 0 4877336 580 638232 0 0 0 0 1171 726 0 1 99 0 0 2015-11-20 02:19:44 1 0 0 1437496 580 638300 0 0 0 0 1203 641 0 1 99 0 0 2015-11-20 02:19:45 1 0 460904 439088 284 631300 1020 465260 1020 465260 7392 6468 0 2 98 0 0 2015-11-20 02:19:46 3 1 1656704 371028 148 633716 2216 1203792 2216 1203792 253072 5293 0 4 95 1 0 2015-11-20 02:19:47 2 0 2989412 385264 140 631940 1772 1325968 1772 1325968 291189 4987 0 4 95 0 0 2015-11-20 02:19:48 1 0 4271348 396588 140 634024 604 1281156 604 1281156 114622 5095 0 2 97 0 0 2015-11-20 02:19:49 1 0 5590260 391532 140 634208 324 1318916 324 1318916 1550 5516 0 1 99 0 0 2015-11-20 02:19:50 3 0 6735960 373428 140 634744 20 1147804 20 1147804 106941 4821 0 2 98 0 0 2015-11-20 02:19:51 3 0 7933896 374244 140 636020 632 1197576 632 1197572 240440 4690 0 4 96 0 0 2015-11-20 02:19:52 3 0 9262464 366936 140 638332 128 1327512 128 1327516 291280 4277 0 4 96 0 0 2015-11-20 02:19:53 1 0 10465268 400632 140 637240 56 1204884 56 1204884 119208 4982 0 2 97 0 0 2015-11-20 02:19:54 1 0 11487212 401092 140 636896 24 1019904 24 1019904 1579 5249 0 1 99 0 0 2015-11-20 02:19:55 1 0 12398600 400644 140 637240 8 911396 8 911396 1434 4825 0 1 99 0 0 2015-11-20 02:19:56 1 1 13407712 396808 140 636480 108 1010376 108 1010376 1741 9335 0 1 98 0 0 2015-11-20 02:19:57 1 0 14212948 397452 140 637192 120 804160 120 804160 1414 4490 0 1 99 0 0 2015-11-20 02:19:58 1 0 14976844 399148 140 636696 0 763904 0 763904 1473 4379 0 1 99 0 0 2015-11-20 02:19:59 1 0 15765336 401612 140 636656 12 788508 12 788508 1387 4378 0 1 99 0 0 2015-11-20 02:20:00 1 0 16737876 403216 140 636468 80 975368 80 975364 1469 4950 0 1 99 0 0 2015-11-20 02:20:01 1 0 17532708 403472 140 637256 0 792056 0 792060 1375 4558 0 1 99 0 0 2015-11-20 02:20:02 1 0 18263000 402296 140 637784 784 733184 784 733184 15557 4555 0 1 98 0 0 2015-11-20 02:20:03 1 0 19246408 404008 140 639284 0 981040 0 981040 15169 4835 0 1 99 0 0 2015-11-20 02:20:04 1 0 19713820 407392 140 638924 0 467420 0 467420 15464 3788 0 1 99 0 0 2015-11-20 02:20:05 1 0 20326740 401112 140 639860 60 612936 60 612936 15072 4204 0 1 99 0 0 2015-11-20 02:20:06 1 0 21001060 402152 140 640008 0 676376 0 676376 15018 4148 0 1 99 0 0 2015-11-20 02:20:07 1 0 21563284 406060 140 639804 20 560188 20 560188 17919 8419 0 2 98 0 0 2015-11-20 02:20:08 1 0 22077856 403296 140 640604 0 514576 0 514576 15618 3734 0 1 99 0 0 2015-11-20 02:20:09 1 0 22578344 402016 140 640896 32 500516 32 500516 15288 3848 0 1 99 0 0 2015-11-20 02:20:10 1 0 23054368 401156 140 641000 0 476024 0 476024 15534 3896 0 1 99 0 0 2015-11-20 02:20:11 1 0 23678064 403060 140 640980 0 623700 0 623700 15184 4009 0 1 99 0 0 2015-11-20 02:20:12 1 0 24152136 424660 140 646608 7564 483848 7564 483848 3544 4709 0 1 98 0 0 2015-11-20 02:20:13 1 0 24631332 402948 140 646572 124 479232 124 479232 1475 4037 0 1 99 0 0 2015-11-20 02:20:14 1 0 25137188 399836 140 646496 0 505856 0 505856 1546 3745 0 1 99 0 0 2015-11-20 02:20:15 1 0 25809492 399544 140 639732 0 672304 0 672304 1500 4242 0 1 99 0 0 2015-11-20 02:20:16 1 0 26839604 397088 140 639816 100 1030144 100 1030144 1476 5131 0 1 99 0 0 2015-11-20 02:20:17 1 0 27873840 392212 140 640160 0 1034240 0 1034240 1387 5104 0 1 99 0 0 2015-11-20 02:20:18 1 0 28866508 423100 140 634052 40 992692 40 992692 43633 8766 0 2 98 0 0 2015-11-20 02:20:19 1 0 29953544 384884 140 634228 1020 1087856 1020 1087856 244003 2850 0 2 97 0 0 2015-11-20 02:20:20 1 0 30991516 388644 140 634544 928 1038824 928 1038824 104550 4616 0 2 98 0 0 2015-11-20 02:20:21 1 0 32099728 393432 140 634540 40 1108220 40 1108220 36817 5281 0 2 98 0 0 2015-11-20 02:20:22 1 0 33346816 398860 140 634820 864 1248384 864 1248384 203863 4811 0 3 96 0 0 2015-11-20 02:20:23 3 0 34256800 396392 92 636452 376 912708 376 912708 106741 4122 0 2 98 0 0 2015-11-20 02:20:24 3 0 35305064 360224 92 637684 756 1047104 756 1047104 215548 3407 0 4 96 0 0 2015-11-20 02:20:25 1 0 36096908 399740 92 636628 180 791124 180 791124 91507 4034 0 2 98 0 0 2015-11-20 02:20:26 3 0 37168508 388644 92 637652 596 1072448 596 1072444 33876 5317 0 2 98 0 0 2015-11-20 02:20:27 1 0 38356828 383224 92 635984 764 1189040 764 1189044 15618 5839 0 1 98 0 0 2015-11-20 02:20:28 1 0 39697584 383288 92 636848 8 1342204 8 1342204 1466 5839 0 1 99 0 0 2015-11-20 02:20:29 1 0 40936988 393532 92 636784 196 1239952 196 1239952 9621 10147 0 2 98 0 0 2015-11-20 02:20:30 1 0 42314612 393596 92 636908 0 1375824 0 1375824 1513 5957 0 1 99 0 0 2015-11-20 02:20:31 1 0 43648364 388076 92 637308 4 1333860 4 1333860 1403 5806 0 1 99 0 0 2015-11-20 02:20:32 1 0 44909932 395256 92 637168 0 1261472 0 1261472 1407 5562 0 1 99 0 0 2015-11-20 02:20:33 1 0 46161428 387000 92 637192 4 1253376 4 1253376 1341 5563 0 1 99 0 0 2015-11-20 02:20:34 1 0 47420772 389516 92 637100 0 1257472 0 1257472 1490 5671 0 1 99 0 0 2015-11-20 02:20:35 1 0 48688468 389928 92 637712 24 1267712 24 1267712 1406 5615 0 1 99 0 0 2015-11-20 02:20:36 1 1 50005344 385328 92 636720 60 1316908 60 1316908 1472 5744 0 1 99 0 0 2015-11-20 02:20:37 1 0 51232704 383852 92 636824 464 1230888 464 1230888 1564 5727 0 1 98 0 0 2015-11-20 02:20:38 1 0 52472728 383456 92 637680 4 1236964 4 1236964 1419 5537 0 1 99 0 0 2015-11-20 02:20:39 1 0 53671788 381408 92 637304 4 1201744 4 1201744 1368 5411 0 1 99 0 0 2015-11-20 02:20:40 2 0 54444956 401360 92 636396 952 771872 952 771872 55875 9172 0 2 97 0 0 2015-11-20 02:20:41 1 0 55317332 391568 92 637668 852 875836 852 875836 85727 4794 0 2 97 0 0 2015-11-20 02:20:42 1 0 56218404 409888 92 634484 764 900928 764 900928 89499 4867 0 2 97 0 0 2015-11-20 02:20:43 1 0 56989380 392256 92 633052 2016 773700 2016 773700 50196 5161 0 2 98 1 0 2015-11-20 02:20:44 3 0 57633636 378960 92 633920 944 642984 944 642984 32478 4148 0 2 98 0 0 2015-11-20 02:20:45 1 0 58693956 392100 92 633332 792 1061092 792 1061092 92713 4976 0 2 97 0 0 2015-11-20 02:20:46 1 0 59583820 407380 92 633848 912 890916 912 890916 98123 4765 0 2 98 0 0 2015-11-20 02:20:47 1 0 60493136 376532 92 633184 1128 912444 1128 912444 50279 5276 0 2 98 0 0 2015-11-20 02:20:48 1 0 61168796 398636 92 634220 920 674796 920 674796 40032 4843 0 2 98 0 0 2015-11-20 02:20:49 1 0 61952912 387680 92 633732 840 784944 840 784944 10903 4938 0 1 98 0 0 2015-11-20 02:20:50 1 0 63190048 387784 92 634132 564 1237880 564 1237880 1759 5928 0 1 98 0 0 2015-11-20 02:20:51 1 1 64455336 383276 92 633660 76 1265664 76 1265664 1541 5668 0 1 99 0 0 2015-11-20 02:20:52 1 0 65825828 386788 92 633696 84 1370948 84 1370948 2145 9935 0 1 98 0 0 2015-11-20 02:20:53 1 0 66505700 386764 92 634404 876 679936 876 679936 1574 4153 0 1 99 0 0 2015-11-20 02:20:54 1 0 67357744 384320 92 634760 8 852056 8 852056 1470 4443 0 1 99 0 0 2015-11-20 02:20:55 1 0 68500536 386720 92 634268 12 1142800 12 1142800 1367 5366 0 1 99 0 0 2015-11-20 02:20:56 1 0 69670512 385312 92 634572 0 1171456 0 1171456 1403 5325 0 1 99 0 0 2015-11-20 02:20:57 1 0 70771360 378756 92 634888 484 1099776 484 1099776 1459 5365 0 1 99 0 0 2015-11-20 02:20:58 1 0 71882880 384664 92 635176 0 1114192 0 1114192 1454 5193 0 1 99 0 0 2015-11-20 02:20:59 1 0 43315772 379880 92 634952 40 634800 40 634800 1382 3564 0 1 99 0 0 2015-11-20 02:21:00 0 0 33600 25734156 92 634884 4536 0 4536 0 2215 4864 0 1 98 1 0 2015-11-20 02:21:01