[RFC, PATCHv2 0/2] mm: map few pages around fault address if they are in page cache

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I've rewrote it from scratch since v1. Locking multiple pages at once was
not that great idea. lock_page() is mess. It seems we don't have good
locking ordering rules for lock_page() vs. lock_page() and it triggers
deadlocks.

Now we have ->fault_nonblock() to ask filesystem for a page, if it's
reachable without blocking. We request one page a time. It's not terribly
efficient and I will probably re-think the interface once again to expose
iterator or something...

Synthetic tests shows pretty impressive numbers: time to complete is
reduced by 80% on sequential multi-threaded access to a file.

Unfortunately, scripting use-case doesn't show any change. I've tried git
test-suite, kernel build and kernel rebuild: no significant difference.

I will try reduce FAULT_AROUND_PAGES to see if it makes any difference.

One important use-case which is not covered is shmem/tmpfs: we need to
implement separate ->fault_nonblock() for it.

Yet another feature to implement is MAP_POPULATE|MAP_NONBLOCK on top of
->fault_nonblock(). It should be useful for dynamic linker for example.

Benchmark data is below.

Any comments?

=========================================================================

Git test-suite make -j60 test:

Base:
 1,529,278,935,718      cycles                     ( +-  0.23% ) [100.00%]
   799,283,349,594      instructions              #    0.52  insns per cycle
                                                  #    3.14  stalled cycles per insn  ( +-  0.12% ) [100.00%]
 2,509,889,276,415      stalled-cycles-frontend   #  164.12% frontend cycles idle     ( +-  0.18% )
       115,493,976      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 1      major-faults

      59.686645587 seconds time elapsed                                          ( +-  0.30% )
Patched:
 1,613,998,434,527      cycles                     ( +-  0.24% ) [100.00%]
   865,776,308,589      instructions              #    0.54  insns per cycle
                                                  #    3.07  stalled cycles per insn  ( +-  0.18% ) [100.00%]
 2,661,812,494,310      stalled-cycles-frontend   #  164.92% frontend cycles idle     ( +-  0.17% )
        47,428,068      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 1      major-faults                                                  ( +- 50.00% )

      60.241766430 seconds time elapsed                                          ( +-  0.85% )

Run make -j60 on clean allmodconfig kernel tree:

Base:
19,511,808,321,876      cycles                    [100.00%]
17,551,075,994,147      instructions              #    0.90  insns per cycle
                                                  #    1.55  stalled cycles per insn [100.00%]
27,227,785,020,522      stalled-cycles-frontend   #  139.55% frontend cycles idle
       268,039,365      minor-faults                                                 [100.00%]
                 0      major-faults

     132.830612471 seconds time elapsed
Patched:
19,421,069,663,037      cycles                    [100.00%]
17,587,643,648,161      instructions              #    0.91  insns per cycle
                                                  #    1.54  stalled cycles per insn [100.00%]
27,063,152,011,598      stalled-cycles-frontend   #  139.35% frontend cycles idle
       193,550,437      minor-faults                                                 [100.00%]
                 0      major-faults

     132.851823758 seconds time elapsed

Run make -j60 on already built allmodconfig kernel tree:

Base:
   286,539,116,594      cycles                     ( +-  0.05% ) [100.00%]
   386,447,067,412      instructions              #    1.35  insns per cycle
                                                  #    0.96  stalled cycles per insn  ( +-  0.00% ) [100.00%]
   372,549,722,846      stalled-cycles-frontend   #  130.02% frontend cycles idle     ( +-  0.04% )
         4,967,540      minor-faults                                                  ( +-  0.06% ) [100.00%]
                 0      major-faults

      27.215434226 seconds time elapsed                                          ( +-  0.18% )
Patched:
   286,435,472,207      cycles                     ( +-  0.07% ) [100.00%]
   388,217,937,147      instructions              #    1.36  insns per cycle
                                                  #    0.96  stalled cycles per insn  ( +-  0.01% ) [100.00%]
   373,681,951,018      stalled-cycles-frontend   #  130.46% frontend cycles idle     ( +-  0.10% )
         2,285,563      minor-faults                                                  ( +-  0.26% ) [100.00%]
                 0      major-faults

      27.292854546 seconds time elapsed                                          ( +-  0.29% )

Access 16g file sequential:

Threads		Base, seconds	Patched, seconds	Diff, %
1		8.284232583	5.424205564		-34,52
8		18.338648302	6.330400940		-65.48
32		54.004043136	10.774241948		-80.04
60		88.928810991	16.571960471		-81.36
120		144.729413453	28.875873029		-80.04

Access 1g file random:

Threads		Base, seconds	Patched, seconds	Diff, %
1		18.247095738	15.003616508		-17.77
8		15.226716773	14.239478275		-6.48
32		22.080582106	16.797777162		-23.92
60		25.759900660	22.768611526		-11.61
120		44.443953873	38.197095497		-14.05

Below is `perf stat' output for the tests above.

=========================================================================

# usemem -t <threads> -f testfile -o 16g #

## 1 thread ##
### base ###
   28,284,111,426      cycles                     ( +-  1.53% ) [100.00%]
    44,120,803,329      instructions              #    1.56  insns per cycle
                                                  #    1.02  stalled cycles per insn  ( +-  0.00% ) [100.00%]
    45,148,666,958      stalled-cycles-frontend   #  159.63% frontend cycles idle     ( +-  1.89% )
         4,194,709      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 0      major-faults

       8.284232583 seconds time elapsed                                          ( +-  1.67% )
### patched ###
    18,615,784,628      cycles                     ( +-  8.46% ) [100.00%]
    41,590,187,000      instructions              #    2.23  insns per cycle
                                                  #    0.66  stalled cycles per insn  ( +-  0.02% ) [100.00%]
    27,374,671,499      stalled-cycles-frontend   #  147.05% frontend cycles idle     ( +- 11.48% )
           131,260      minor-faults                                                  ( +-  0.03% ) [100.00%]
                 0      major-faults

       5.424205564 seconds time elapsed                                          ( +-  8.54% )

## 8 threads ##
### base ###
   447,114,910,645      cycles                     ( +-  3.22% ) [100.00%]
   350,993,124,470      instructions              #    0.79  insns per cycle
                                                  #    2.29  stalled cycles per insn  ( +-  0.01% ) [100.00%]
   803,040,468,490      stalled-cycles-frontend   #  179.60% frontend cycles idle     ( +-  3.59% )
        33,556,008      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 0      major-faults

      18.338648302 seconds time elapsed                                          ( +-  3.20% )
### patched ###
   136,437,995,805      cycles                     ( +-  0.87% ) [100.00%]
   328,096,731,149      instructions              #    2.40  insns per cycle
                                                  #    0.60  stalled cycles per insn  ( +-  0.02% ) [100.00%]
   197,237,317,698      stalled-cycles-frontend   #  144.56% frontend cycles idle     ( +-  1.19% )
         1,338,884      minor-faults                                                  ( +-  3.11% ) [100.00%]
                 0      major-faults

       6.330400940 seconds time elapsed                                          ( +-  1.50% )

## 32 threads ##
### base ###
 4,844,333,104,028      cycles                     ( +-  2.17% ) [100.00%]
 1,401,233,609,645      instructions              #    0.29  insns per cycle
                                                  #    6.61  stalled cycles per insn  ( +-  0.02% ) [100.00%]
 9,265,429,354,789      stalled-cycles-frontend   #  191.26% frontend cycles idle     ( +-  1.81% )
       134,228,040      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 0      major-faults

      54.004043136 seconds time elapsed                                          ( +-  2.03% )
### patched ###
   512,226,985,443      cycles                     ( +-  2.21% ) [100.00%]
 1,307,350,894,141      instructions              #    2.55  insns per cycle
                                                  #    0.52  stalled cycles per insn  ( +-  0.02% ) [100.00%]
   677,259,628,341      stalled-cycles-frontend   #  132.22% frontend cycles idle     ( +-  1.56% )
         4,820,979      minor-faults                                                  ( +-  3.88% ) [100.00%]
                 0      major-faults

      10.774241948 seconds time elapsed                                          ( +-  2.40% )
## 60 threads ##
### base ###
14,878,749,780,310      cycles                     ( +-  1.24% ) [100.00%]
 2,633,166,794,060      instructions              #    0.18  insns per cycle
                                                  #   10.75  stalled cycles per insn  ( +-  0.01% ) [100.00%]
28,301,405,498,867      stalled-cycles-frontend   #  190.21% frontend cycles idle     ( +-  0.88% )
       251,687,643      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 0      major-faults

      88.928810991 seconds time elapsed                                          ( +-  1.44% )
### patched ###
 1,156,172,977,262      cycles                     ( +-  3.24% ) [100.00%]
 2,450,915,465,588      instructions              #    2.12  insns per cycle
                                                  #    0.63  stalled cycles per insn  ( +-  0.01% ) [100.00%]
 1,534,996,614,819      stalled-cycles-frontend   #  132.77% frontend cycles idle     ( +-  4.37% )
         9,239,706      minor-faults                                                  ( +-  1.66% ) [100.00%]
                 0      major-faults

      16.571960471 seconds time elapsed                                          ( +-  2.64% )
## 120 threads ##
### base ###
47,350,885,336,341      cycles                     ( +-  2.28% ) [100.00%]
 5,282,921,722,672      instructions              #    0.11  insns per cycle
                                                  #    8.96  stalled cycles per insn  ( +-  0.04% ) [100.00%]
47,332,933,971,860      stalled-cycles-frontend   #   99.96% frontend cycles idle     ( +-  2.40% )
       503,357,749      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 0      major-faults

     144.729413453 seconds time elapsed                                          ( +-  1.75% )
### patched ###
 3,791,855,572,557      cycles                     ( +-  2.15% ) [100.00%]
 4,901,681,561,338      instructions              #    1.29  insns per cycle
                                                  #    0.59  stalled cycles per insn  ( +-  0.01% ) [100.00%]
 2,880,236,854,539      stalled-cycles-frontend   #   75.96% frontend cycles idle     ( +-  3.69% )
        18,080,707      minor-faults                                                  ( +-  1.22% ) [100.00%]
                 0      major-faults

      28.875873029 seconds time elapsed                                          ( +-  1.19% )

# usemem -t <threads> -R -f testfile -o 1g #

## 1 thread ##
### base ###
    62,357,670,556      cycles                     ( +-  0.58% ) [100.00%]
    11,612,293,315      instructions              #    0.19  insns per cycle
                                                  #   10.31  stalled cycles per insn  ( +-  0.03% ) [100.00%]
   119,734,125,613      stalled-cycles-frontend   #  192.01% frontend cycles idle     ( +-  0.58% )
           262,723      minor-faults                                                  ( +-  0.01% ) [100.00%]
                 0      major-faults

      18.247095738 seconds time elapsed                                          ( +-  0.58% )
### patched ###
    51,421,735,331      cycles                     ( +- 11.53% ) [100.00%]
    11,446,081,379      instructions              #    0.22  insns per cycle
                                                  #    8.55  stalled cycles per insn  ( +-  0.19% ) [100.00%]
    97,900,718,100      stalled-cycles-frontend   #  190.39% frontend cycles idle     ( +- 12.09% )
             8,629      minor-faults                                                  ( +-  1.20% ) [100.00%]
                 0      major-faults                                                  ( +-100.00% )

      15.003616508 seconds time elapsed                                          ( +- 11.56% )

## 8 threads ##
### base ###
   395,785,250,840      cycles                     ( +-  0.60% ) [100.00%]
    93,850,278,507      instructions              #    0.24  insns per cycle
                                                  #    8.02  stalled cycles per insn  ( +-  0.03% ) [100.00%]
   752,951,585,171      stalled-cycles-frontend   #  190.24% frontend cycles idle     ( +-  0.61% )
         2,097,988      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 0      major-faults

      15.226716773 seconds time elapsed                                          ( +-  0.41% )
### patched ###
 Performance counter stats for 'system wide' (5 runs):

   369,032,514,042      cycles                     ( +-  0.42% ) [100.00%]
    92,357,503,760      instructions              #    0.25  insns per cycle
                                                  #    7.59  stalled cycles per insn  ( +-  0.01% ) [100.00%]
   700,555,940,601      stalled-cycles-frontend   #  189.84% frontend cycles idle     ( +-  0.44% )
            89,186      minor-faults                                                  ( +-  1.88% ) [100.00%]
                 0      major-faults

      14.239478275 seconds time elapsed                                          ( +-  0.48% )

## 32 threads ##
### base ###
 1,791,516,702,606      cycles                     ( +-  1.57% ) [100.00%]
   369,936,392,907      instructions              #    0.21  insns per cycle
                                                  #    9.07  stalled cycles per insn  ( +-  0.07% ) [100.00%]
 3,357,017,162,189      stalled-cycles-frontend   #  187.38% frontend cycles idle     ( +-  1.77% )
         8,390,395      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 0      major-faults

      22.080582106 seconds time elapsed                                          ( +-  9.50% )
### patched ###
 1,517,335,667,819      cycles                     ( +-  0.63% ) [100.00%]
   363,401,817,392      instructions              #    0.24  insns per cycle
                                                  #    7.45  stalled cycles per insn  ( +-  0.01% ) [100.00%]
 2,705,627,050,641      stalled-cycles-frontend   #  178.31% frontend cycles idle     ( +-  1.15% )
           269,287      minor-faults                                                  ( +-  0.16% ) [100.00%]
                 0      major-faults

      16.797777162 seconds time elapsed                                          ( +-  7.69% )

## 60 threads ##
### base ###
 3,983,862,013,747      cycles                     ( +-  1.42% ) [100.00%]
   693,310,458,766      instructions              #    0.17  insns per cycle
                                                  #   10.75  stalled cycles per insn  ( +-  0.01% ) [100.00%]
 7,454,138,580,163      stalled-cycles-frontend   #  187.11% frontend cycles idle     ( +-  1.27% )
        15,735,826      minor-faults                                                  ( +-  0.02% ) [100.00%]
                 1      major-faults                                                  ( +-100.00% )

      25.759900660 seconds time elapsed                                          ( +-  0.64% )
### patched ###
 2,871,988,372,901      cycles                     ( +-  1.17% ) [100.00%]
   681,452,765,745      instructions              #    0.24  insns per cycle
                                                  #    7.61  stalled cycles per insn  ( +-  0.03% ) [100.00%]
 5,186,017,674,808      stalled-cycles-frontend   #  180.57% frontend cycles idle     ( +-  1.24% )
           498,198      minor-faults                                                  ( +-  0.05% ) [100.00%]
                 0      major-faults

      22.768611526 seconds time elapsed                                          ( +-  3.13% )

## 120 threads ##
### base ###
14,891,491,866,813      cycles                     ( +-  0.83% ) [100.00%]
 1,391,880,325,881      instructions              #    0.09  insns per cycle
                                                  #   10.51  stalled cycles per insn  ( +-  0.02% ) [100.00%]
14,629,621,106,104      stalled-cycles-frontend   #   98.24% frontend cycles idle     ( +-  1.04% )
        31,468,057      minor-faults                                                  ( +-  0.00% ) [100.00%]
                 0      major-faults

      44.443953873 seconds time elapsed                                          ( +-  0.91% )
### patched ###
12,450,645,304,818      cycles                     ( +-  2.94% ) [100.00%]
 1,367,989,003,638      instructions              #    0.11  insns per cycle
                                                  #    8.98  stalled cycles per insn  ( +-  0.04% ) [100.00%]
12,282,274,614,247      stalled-cycles-frontend   #   98.65% frontend cycles idle     ( +-  3.34% )
           996,361      minor-faults                                                  ( +-  0.02% ) [100.00%]
                 0      major-faults

      38.197095497 seconds time elapsed                                          ( +-  2.

Kirill A. Shutemov (2):
  mm: introduce vm_ops->fault_nonblock()
  mm: implement ->fault_nonblock() for page cache

 Documentation/filesystems/Locking |  8 ++++++++
 fs/9p/vfs_file.c                  |  2 ++
 fs/btrfs/file.c                   |  1 +
 fs/cifs/file.c                    |  1 +
 fs/ext4/file.c                    |  1 +
 fs/f2fs/file.c                    |  1 +
 fs/fuse/file.c                    |  1 +
 fs/gfs2/file.c                    |  1 +
 fs/nfs/file.c                     |  1 +
 fs/nilfs2/file.c                  |  1 +
 fs/ubifs/file.c                   |  1 +
 fs/xfs/xfs_file.c                 |  1 +
 include/linux/mm.h                |  3 +++
 mm/filemap.c                      | 35 +++++++++++++++++++++++++++++++++++
 mm/memory.c                       | 38 +++++++++++++++++++++++++++++++++++++-
 15 files changed, 95 insertions(+), 1 deletion(-)

-- 
1.9.0.rc3

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [Samba]     [Device Mapper]     [CEPH Development]
  Powered by Linux