Update the discussion of huge pages to note that fio will try to parse /proc/meminfo for the hugepage size if it is not specified. Also note that the default value used when parsing fails is platform specific. Signed-off-by: Vincent Fu <vincent.fu@xxxxxxxxxxx> --- HOWTO.rst | 38 ++++++++++++++++++++++---------------- fio.1 | 29 ++++++++++++++++------------- 2 files changed, 38 insertions(+), 29 deletions(-) diff --git a/HOWTO.rst b/HOWTO.rst index eee386c1..1e9fa353 100644 --- a/HOWTO.rst +++ b/HOWTO.rst @@ -1818,19 +1818,20 @@ Buffers and memory Use GPU memory as the buffers for GPUDirect RDMA benchmark. The :option:`ioengine` must be `rdma`. - The area allocated is a function of the maximum allowed bs size for the job, - multiplied by the I/O depth given. Note that for **shmhuge** and - **mmaphuge** to work, the system must have free huge pages allocated. This - can normally be checked and set by reading/writing - :file:`/proc/sys/vm/nr_hugepages` on a Linux system. Fio assumes a huge page - is 4MiB in size. So to calculate the number of huge pages you need for a - given job file, add up the I/O depth of all jobs (normally one unless - :option:`iodepth` is used) and multiply by the maximum bs set. Then divide - that number by the huge page size. You can see the size of the huge pages in - :file:`/proc/meminfo`. If no huge pages are allocated by having a non-zero - number in `nr_hugepages`, using **mmaphuge** or **shmhuge** will fail. Also - see :option:`hugepage-size`. - + The area allocated is a function of the maximum allowed bs size for the job, + multiplied by the I/O depth given. Note that for **shmhuge** and + **mmaphuge** to work, the system must have free huge pages allocated. This + can normally be checked and set by reading/writing + :file:`/proc/sys/vm/nr_hugepages` on a Linux system. To calculate the + number of huge pages you need for a given job file, add up the I/O + depth of all jobs (normally one unless :option:`iodepth` is used) and + multiply by the maximum bs set. Then divide that number by the huge + page size. You can see the size of huge pages in + :file:`/proc/meminfo`. If no huge pages are allocated by having a + non-zero number in `nr_hugepages`, using **mmaphuge** or **shmhuge** + will fail. See the option :option:`hugepage-size` below to set the + hugepage size for fio to use. + **mmaphuge** also needs to have hugetlbfs mounted and the file location should point there. So if it's mounted in :file:`/huge`, you would use `mem=mmaphuge:/huge/somefile`. @@ -1849,9 +1850,14 @@ Buffers and memory .. option:: hugepage-size=int Defines the size of a huge page. Must at least be equal to the system - setting, see :file:`/proc/meminfo`. Defaults to 4MiB. Should probably - always be a multiple of megabytes, so using ``hugepage-size=Xm`` is the - preferred way to set this to avoid setting a non-pow-2 bad value. + setting, see :file:`/proc/meminfo`. Should probably always be a + multiple of megabytes, so using ``hugepage-size=Xm`` is the preferred + way to set this to avoid setting a non-pow-2 bad value. If this option + is not specified fio will attempt to parse :file:`/proc/meminfo` to + obtain the system huge page size. If this fails fio will use a platform + specific default of 2-4MiB. This value is used by fio to ensure that + the amount of memory requested for I/O buffers will be a multiple of + the huge page size. .. option:: lockmem=int diff --git a/fio.1 b/fio.1 index ded7bbfc..a7b647bf 100644 --- a/fio.1 +++ b/fio.1 @@ -1629,15 +1629,14 @@ The \fBioengine\fR must be \fBrdma\fR. The area allocated is a function of the maximum allowed bs size for the job, multiplied by the I/O depth given. Note that for \fBshmhuge\fR and \fBmmaphuge\fR to work, the system must have free huge pages allocated. This -can normally be checked and set by reading/writing -`/proc/sys/vm/nr_hugepages' on a Linux system. Fio assumes a huge page -is 4MiB in size. So to calculate the number of huge pages you need for a -given job file, add up the I/O depth of all jobs (normally one unless -\fBiodepth\fR is used) and multiply by the maximum bs set. Then divide -that number by the huge page size. You can see the size of the huge pages in -`/proc/meminfo'. If no huge pages are allocated by having a non-zero -number in `nr_hugepages', using \fBmmaphuge\fR or \fBshmhuge\fR will fail. Also -see \fBhugepage\-size\fR. +can normally be checked and set by reading/writing `/proc/sys/vm/nr_hugepages' +on a Linux system. To calculate the number of huge pages you need for a given +job file, add up the I/O depth of all jobs (normally one unless \fBiodepth\fR +is used) and multiply by the maximum bs set. Then divide that number by the +huge page size. You can see the size of the huge pages in `/proc/meminfo'. If +no huge pages are allocated by having a non-zero number in `nr_hugepages', +using \fBmmaphuge\fR or \fBshmhuge\fR will fail. See the \fBhugepage\-size\fR +option below to set the huge page size for fio to use. .P \fBmmaphuge\fR also needs to have hugetlbfs mounted and the file location should point there. So if it's mounted in `/huge', you would use @@ -1655,10 +1654,14 @@ of subsequent I/O memory buffers is the sum of the \fBiomem_align\fR and \fBbs\fR used. .TP .BI hugepage\-size \fR=\fPint -Defines the size of a huge page. Must at least be equal to the system -setting, see `/proc/meminfo'. Defaults to 4MiB. Should probably -always be a multiple of megabytes, so using `hugepage\-size=Xm' is the -preferred way to set this to avoid setting a non-pow-2 bad value. +Defines the size of a huge page. Must at least be equal to the system setting, +see `/proc/meminfo'. Should probably always be a multiple of megabytes, so +using `hugepage\-size=Xm' is the preferred way to set this to avoid setting a +non-pow-2 bad value. If this option is not specified fio will attempt to parse +`/proc/meminfo' to obtain the system huge page size. If this fails, fio will +use a platform-specific default of 2-4MiB. This value is used by fio to ensure +that the amount of memory requested for I/O buffers will be a multiple of the +huge page size. .TP .BI lockmem \fR=\fPint Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to -- 2.25.1
