Re: [KVM AUTOTEST PATCH] KVM test: Add hugepage variant

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



* Luk?? Doktor <ldoktor@xxxxxxxxxx> [2009-07-28 08:22]:
> Yes, this looks more pythonish and actually better than my version. I'm 
> missing only one thing, extra_params += " -mem-path /mnt/hugepage" down 
> in configuration (see below).

Don't we also need to inspect the qemu binary to determine if it's one
of the few releases that used -mempath instead of -mem-path ?  Or are we
ignoring those?


> 
> This cause problem with predefined mount point, because it needs to be 
> the same in extra_params and python script.
> 
> Dne 27.7.2009 23:10, Lucas Meneghel Rodrigues napsal(a):
> >This patch adds a small setup script to set up huge memory
> >pages during the kvm tests execution. Also, added hugepage setup to the
> >fc8_quick sample.
> >
> >Signed-off-by: Luk???? Doktor<ldoktor@xxxxxxxxxx>
> >Signed-off-by: Lucas Meneghel Rodrigues<lmr@xxxxxxxxxx>
> >
> >---
> >  client/tests/kvm/kvm_tests.cfg.sample |    6 ++
> >  client/tests/kvm/kvm_vm.py            |   11 +++
> >  client/tests/kvm/scripts/hugepage.py  |  110 
> >  +++++++++++++++++++++++++++++++++
> >  3 files changed, 127 insertions(+), 0 deletions(-)
> >  create mode 100644 client/tests/kvm/scripts/hugepage.py
> >
> >diff --git a/client/tests/kvm/kvm_tests.cfg.sample 
> >b/client/tests/kvm/kvm_tests.cfg.sample
> >index 2d75a66..4a6a174 100644
> >--- a/client/tests/kvm/kvm_tests.cfg.sample
> >+++ b/client/tests/kvm/kvm_tests.cfg.sample
> >@@ -585,6 +585,11 @@ variants:
> >          only default
> >          image_format = raw
> >
> >+variants:
> >+    - @kvm_smallpages:
> >+    - kvm_hugepages:
> >+        pre_command = "/usr/bin/python scripts/hugepage.py"
> 
> +        extra_params += " -mem-path /mnt/hugepage"
> 
> # ^^Tells qemu to allocate guest memory as hugepage
> 
> I'd rather have this part of cfg look like this:
> variants:
> 	- @kvm_smallpages:
> 	- kvm_hugepages:
> 		pre_command = "/usr/bin/python scripts/hugepage.py 
> 		/mnt/hugepage"
> 		extra_params += " -mem-path /mnt/hugepage"
> 
> because this way it's more clear the relation between the constants. (it 
> doesn't changes the script itself)
> 
> >+
> >
> >  variants:
> >      - @basic:
> >@@ -598,6 +603,7 @@ variants:
> >          only Fedora.8.32
> >          only install setup boot shutdown
> >          only rtl8139
> >+        only kvm_hugepages
> >      - @sample1:
> >          only qcow2
> >          only ide
> >diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> >index d96b359..eba9b84 100644
> >--- a/client/tests/kvm/kvm_vm.py
> >+++ b/client/tests/kvm/kvm_vm.py
> >@@ -397,6 +397,17 @@ class VM:
> >                  self.destroy()
> >                  return False
> >
> >+            # Get the output so far, to see if we have any problems with
> >+            # hugepage setup.
> >+            output = self.process.get_output()
> >+
> >+            if "alloc_mem_area" in output:
> >+                logging.error("Could not allocate hugepage memory; "
> >+                              "qemu command:\n%s" % qemu_command)
> >+                logging.error("Output:" + 
> >kvm_utils.format_str_for_message(
> >+                              self.process.get_output()))
> >+                return False
> >+
> >              logging.debug("VM appears to be alive with PID %d",
> >                            self.process.get_pid())
> >              return True
> >diff --git a/client/tests/kvm/scripts/hugepage.py 
> >b/client/tests/kvm/scripts/hugepage.py
> >new file mode 100644
> >index 0000000..9bc4194
> >--- /dev/null
> >+++ b/client/tests/kvm/scripts/hugepage.py
> >@@ -0,0 +1,110 @@
> >+#!/usr/bin/python
> >+# -*- coding: utf-8 -*-
> >+import os, sys, time
> >+
> >+"""
> >+Simple script to allocate enough hugepages for KVM testing purposes.
> >+"""
> >+
> >+class HugePageError(Exception):
> >+    """
> >+    Simple wrapper for the builtin Exception class.
> >+    """
> >+    pass
> >+
> >+
> >+class HugePage:
> >+    def __init__(self, hugepage_path=None):
> >+        """
> >+        Gets environment variable values and calculates the target number
> >+        of huge memory pages.
> >+
> >+        @param hugepage_path: Path where to mount hugetlbfs path, if not
> >+                yet configured.
> >+        """
> >+        self.vms = len(os.environ['KVM_TEST_vms'].split())
> >+        self.mem = int(os.environ['KVM_TEST_mem'])
> >+        try:
> >+            self.max_vms = int(os.environ['KVM_TEST_max_vms'])
> >+        except KeyError:
> >+            self.max_vms = 0
> >+        if hugepage_path:
> >+            self.hugepage_path = hugepage_path
> >+        else:
> >+            self.hugepage_path = '/mnt/kvm_hugepage'
> >+        self.hugepage_size = self.get_hugepage_size()
> >+        self.target_hugepages = self.get_target_hugepages()
> >+
> >+
> >+    def get_hugepage_size(self):
> >+        """
> >+        Get the current system setting for huge memory page size.
> >+        """
> >+        meminfo = open('/proc/meminfo', 'r').readlines()
> >+        huge_line_list = [h for h in meminfo if 
> >h.startswith("Hugepagesize")]
> >+        try:
> >+            return int(huge_line_list[0].split()[1])
> >+        except ValueError, e:
> >+            raise HugePageError("Could not get huge page size setting 
> >from "
> >+                                "/proc/meminfo: %s" % e)
> >+
> >+
> >+    def get_target_hugepages(self):
> >+        """
> >+        Calculate the target number of hugepages for testing purposes.
> >+        """
> >+        if self.vms<  self.max_vms:
> >+            self.vms = self.max_vms
> >+        vmsm = (self.vms * self.mem) + (self.vms * 64)
> >+        return int(vmsm * 1024 / self.hugepage_size)
> >+
> >+
> >+    def set_hugepages(self):
> >+        """
> >+        Sets the hugepage limit to the target hugepage value calculated.
> >+        """
> >+        hugepage_cfg = open("/proc/sys/vm/nr_hugepages", "r+")
> >+        hp = hugepage_cfg.readline()
> >+        while int(hp)<  self.target_hugepages:
> >+            loop_hp = hp
> >+            hugepage_cfg.write(str(self.target_hugepages))
> >+            hugepage_cfg.flush()
> >+            time.sleep(5)
> >+            hugepage_cfg.seek(0)
> >+            hp = int(hugepage_cfg.readline())
> >+            if loop_hp == hp:
> >+                raise HugePageError("Cannot set the kernel hugepage 
> >setting "
> >+                                    "to the target value of %d 
> >hugepages." %
> >+                                    self.target_hugepages)
> >+        hugepage_cfg.close()
> >+
> >+
> >+    def mount_hugepage_fs(self):
> >+        """
> >+        Verify if there's a hugetlbfs mount set. If there's none, will 
> >set up
> >+        a hugetlbfs mount using the class attribute that defines the mount
> >+        point.
> >+        """
> >+        current_mounts = open("/proc/mounts", "r").readlines()
> >+        huge_mounts = [m for m in current_mounts if m.split()[2] == 
> >"hugetlbfs"]
> >+        if len(huge_mounts) == 0:
> >+            if not os.path.isdir(self.hugepage_path):
> >+                os.makedirs(self.hugepage_path)
> >+            cmd = "mount -t hugetlbfs none %s" % self.hugepage_path
> >+            if os.system(cmd):
> >+                raise HugePageError("Cannot mount hugetlbfs path %s" %
> >+                                    self.hugepage_path)
> >+
> >+
> >+    def setup(self):
> >+        self.set_hugepages()
> >+        self.mount_hugepage_fs()
> >+
> >+
> >+if __name__ == "__main__":
> >+    if len(sys.argv)<  2:
> >+        huge_page = HugePage()
> >+    else:
> >+        huge_page = HugePage(sys.argv[1])
> >+
> >+    huge_page.setup()
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
ryanh@xxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux