Re: [PATCH v1 3/4] kunit: tool: add support for QEMU

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, May 15, 2021 at 12:59 AM David Gow <davidgow@xxxxxxxxxx> wrote:
>
> On Sat, May 8, 2021 at 5:31 AM Brendan Higgins
> <brendanhiggins@xxxxxxxxxx> wrote:
> >
> > Add basic support to run QEMU via kunit_tool. Add support for i386,
> > x86_64, arm, arm64, and a bunch more.
> >
> > Signed-off-by: Brendan Higgins <brendanhiggins@xxxxxxxxxx>
> > Tested-by: David Gow <davidgow@xxxxxxxxxx>
> > ---
> >
> > Changes since last revision:
> >
> > - A number of minor obvious issues pointed out by David and Daniel.
> > - Added facility for merging Kconfigs at Daniel's suggestion.
> > - Broke out qemu_configs each into their own config file which is loaded
> >   dynamically - mostly at David's suggestion.
> >
> > ---
>
> This seems pretty good to me. I only have one real complaint --
> qemu_configs needing to be in a subdirectory of ./tools/testing/kunit
> -- but am able to tolerate that (even if I'd prefer not to have it) if
> it's documented properly.
>
> Otherwise, save for a couple of minor nitpicks, this seems good to go.
>
> Reviewed-by: David Gow <davidgow@xxxxxxxxxx>
>
>
> >  tools/testing/kunit/kunit.py                |  57 ++++++-
> >  tools/testing/kunit/kunit_config.py         |   7 +-
> >  tools/testing/kunit/kunit_kernel.py         | 170 ++++++++++++++++----
> >  tools/testing/kunit/kunit_tool_test.py      |  18 ++-
> >  tools/testing/kunit/qemu_config.py          |  17 ++
> >  tools/testing/kunit/qemu_configs/alpha.py   |  10 ++
> >  tools/testing/kunit/qemu_configs/arm.py     |  13 ++
> >  tools/testing/kunit/qemu_configs/arm64.py   |  12 ++
> >  tools/testing/kunit/qemu_configs/i386.py    |  10 ++
> >  tools/testing/kunit/qemu_configs/powerpc.py |  12 ++
> >  tools/testing/kunit/qemu_configs/riscv.py   |  31 ++++
> >  tools/testing/kunit/qemu_configs/s390.py    |  14 ++
> >  tools/testing/kunit/qemu_configs/sparc.py   |  10 ++
> >  tools/testing/kunit/qemu_configs/x86_64.py  |  10 ++
> >  14 files changed, 350 insertions(+), 41 deletions(-)
> >  create mode 100644 tools/testing/kunit/qemu_config.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/alpha.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/arm.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/arm64.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/i386.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/powerpc.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/riscv.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/s390.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/sparc.py
> >  create mode 100644 tools/testing/kunit/qemu_configs/x86_64.py
> >
> > diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
> > index 5da8fb3762f98..be8d8d4a4e08f 100755
> > --- a/tools/testing/kunit/kunit.py
> > +++ b/tools/testing/kunit/kunit.py
> > @@ -70,10 +70,10 @@ def build_tests(linux: kunit_kernel.LinuxSourceTree,
> >         kunit_parser.print_with_timestamp('Building KUnit Kernel ...')
> >
> >         build_start = time.time()
> > -       success = linux.build_um_kernel(request.alltests,
> > -                                       request.jobs,
> > -                                       request.build_dir,
> > -                                       request.make_options)
> > +       success = linux.build_kernel(request.alltests,
> > +                                    request.jobs,
> > +                                    request.build_dir,
> > +                                    request.make_options)
> >         build_end = time.time()
> >         if not success:
> >                 return KunitResult(KunitStatus.BUILD_FAILURE,
> > @@ -189,6 +189,31 @@ def add_common_opts(parser) -> None:
> >                              'will get  automatically appended.',
> >                              metavar='kunitconfig')
> >
> > +       parser.add_argument('--arch',
> > +                           help=('Specifies the architecture to run tests under. '
> > +                                 'The architecture specified here must match the '
> > +                                 'string passed to the ARCH make param, '
> > +                                 'e.g. i386, x86_64, arm, um, etc. Non-UML '
> > +                                 'architectures run on QEMU.'),
> > +                           type=str, default='um', metavar='arch')
> > +
> > +       parser.add_argument('--cross_compile',
> > +                           help=('Sets make\'s CROSS_COMPILE variable; it should '
> > +                                 'be set to a toolchain path prefix (the prefix '
> > +                                 'of gcc and other tools in your toolchain, for '
> > +                                 'example `sparc64-linux-gnu-` if you have the '
> > +                                 'sparc toolchain installed on your system, or '
> > +                                 '`$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-` '
> Super-minor nit: is there a shorter example we can give that's not
> much longer than all the surrounding lines?

I wanted something which actually uses the paths that matches real gcc
toolchains and they don't get much shorter than this (I guess I could
drop $HOME/toolchains/microblaze, but it would still be very long).
Think maybe I should just drop this secondary example entirely? Anyone
trying to download and use a toolchain probably has better resources
to follow anyway.

> > +                                 'if you have downloaded the microblaze toolchain '
> > +                                 'from the 0-day website to a directory in your '
> > +                                 'home directory called `toolchains`).'),
> > +                           metavar='cross_compile')
> > +
> > +       parser.add_argument('--qemu_config',
> > +                           help=('Takes a path to a path to a file containing '
> > +                                 'a QemuArchParams object.'),
> > +                           type=str, metavar='qemu_config')
> > +
>
> If (as noted below) this file needs to be in a subdirectory of the
> kunit_tool directory, we should document this, or maybe make this path
> relative to the kunit_tool directory.

I'll go into more detail below, but I was able to change this to take
a config from anywhere.

> >  def add_build_opts(parser) -> None:
> >         parser.add_argument('--jobs',
> >                             help='As in the make command, "Specifies  the number of '
> > @@ -270,7 +295,11 @@ def main(argv, linux=None):
> >                         os.mkdir(cli_args.build_dir)
> >
> >                 if not linux:
> > -                       linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
> > +                       linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
> > +                                       kunitconfig_path=cli_args.kunitconfig,
> > +                                       arch=cli_args.arch,
> > +                                       cross_compile=cli_args.cross_compile,
> > +                                       qemu_config_path=cli_args.qemu_config)
> >
> >                 request = KunitRequest(cli_args.raw_output,
> >                                        cli_args.timeout,
> > @@ -289,7 +318,11 @@ def main(argv, linux=None):
> >                         os.mkdir(cli_args.build_dir)
> >
> >                 if not linux:
> > -                       linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
> > +                       linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
> > +                                       kunitconfig_path=cli_args.kunitconfig,
> > +                                       arch=cli_args.arch,
> > +                                       cross_compile=cli_args.cross_compile,
> > +                                       qemu_config_path=cli_args.qemu_config)
> >
> >                 request = KunitConfigRequest(cli_args.build_dir,
> >                                              cli_args.make_options)
> > @@ -301,7 +334,11 @@ def main(argv, linux=None):
> >                         sys.exit(1)
> >         elif cli_args.subcommand == 'build':
> >                 if not linux:
> > -                       linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
> > +                       linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
> > +                                       kunitconfig_path=cli_args.kunitconfig,
> > +                                       arch=cli_args.arch,
> > +                                       cross_compile=cli_args.cross_compile,
> > +                                       qemu_config_path=cli_args.qemu_config)
> >
> >                 request = KunitBuildRequest(cli_args.jobs,
> >                                             cli_args.build_dir,
> > @@ -315,7 +352,11 @@ def main(argv, linux=None):
> >                         sys.exit(1)
> >         elif cli_args.subcommand == 'exec':
> >                 if not linux:
> > -                       linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
> > +                       linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
> > +                                       kunitconfig_path=cli_args.kunitconfig,
> > +                                       arch=cli_args.arch,
> > +                                       cross_compile=cli_args.cross_compile,
> > +                                       qemu_config_path=cli_args.qemu_config)
> >
> >                 exec_request = KunitExecRequest(cli_args.timeout,
> >                                                 cli_args.build_dir,
> > diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py
> > index 1e2683dcc0e7a..c77c7d2ef6220 100644
> > --- a/tools/testing/kunit/kunit_config.py
> > +++ b/tools/testing/kunit/kunit_config.py
> > @@ -52,8 +52,13 @@ class Kconfig(object):
> >                                 return False
> >                 return True
> >
> > +       def merge_in_entries(self, other: 'Kconfig') -> None:
> > +               if other.is_subset_of(self):
> > +                       return
> > +               self._entries = list(self.entries().union(other.entries()))
> > +
> >         def write_to_file(self, path: str) -> None:
> > -               with open(path, 'w') as f:
> > +               with open(path, 'a+') as f:
> >                         for entry in self.entries():
> >                                 f.write(str(entry) + '\n')
> >
> > diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
> > index e22ade9d91ad5..2bd196fd69e5c 100644
> > --- a/tools/testing/kunit/kunit_kernel.py
> > +++ b/tools/testing/kunit/kunit_kernel.py
> > @@ -6,23 +6,31 @@
> >  # Author: Felix Guo <felixguoxiuping@xxxxxxxxx>
> >  # Author: Brendan Higgins <brendanhiggins@xxxxxxxxxx>
> >
> > +from __future__ import annotations
> > +import importlib.util
> >  import logging
> >  import subprocess
> >  import os
> >  import shutil
> >  import signal
> >  from typing import Iterator
> > +from typing import Optional
> >
> >  from contextlib import ExitStack
> >
> > +from collections import namedtuple
> > +
> >  import kunit_config
> >  import kunit_parser
> > +import qemu_config
> >
> >  KCONFIG_PATH = '.config'
> >  KUNITCONFIG_PATH = '.kunitconfig'
> >  DEFAULT_KUNITCONFIG_PATH = 'arch/um/configs/kunit_defconfig'
> >  BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config'
> >  OUTFILE_PATH = 'test.log'
> > +ABS_TOOL_PATH = os.path.abspath(os.path.dirname(__file__))
> > +QEMU_CONFIGS_DIR = os.path.join(ABS_TOOL_PATH, 'qemu_configs')
> >
> >  def get_file_path(build_dir, default):
> >         if build_dir:
> > @@ -40,6 +48,10 @@ class BuildError(Exception):
> >  class LinuxSourceTreeOperations(object):
> >         """An abstraction over command line operations performed on a source tree."""
> >
> > +       def __init__(self, linux_arch: str, cross_compile: Optional[str]):
> > +               self._linux_arch = linux_arch
> > +               self._cross_compile = cross_compile
> > +
> >         def make_mrproper(self) -> None:
> >                 try:
> >                         subprocess.check_output(['make', 'mrproper'], stderr=subprocess.STDOUT)
> > @@ -48,12 +60,21 @@ class LinuxSourceTreeOperations(object):
> >                 except subprocess.CalledProcessError as e:
> >                         raise ConfigError(e.output.decode())
> >
> > +       def make_arch_qemuconfig(self, kconfig: kunit_config.Kconfig) -> None:
> > +               pass
> > +
> > +       def make_allyesconfig(self, build_dir, make_options) -> None:
> > +               raise ConfigError('Only the "um" arch is supported for alltests')
> > +
> >         def make_olddefconfig(self, build_dir, make_options) -> None:
> > -               command = ['make', 'ARCH=um', 'olddefconfig']
> > +               command = ['make', 'ARCH=' + self._linux_arch, 'olddefconfig']
> > +               if self._cross_compile:
> > +                       command += ['CROSS_COMPILE=' + self._cross_compile]
> >                 if make_options:
> >                         command.extend(make_options)
> >                 if build_dir:
> >                         command += ['O=' + build_dir]
> > +               print('Populating config with:\n$', ' '.join(command))
> >                 try:
> >                         subprocess.check_output(command, stderr=subprocess.STDOUT)
> >                 except OSError as e:
> > @@ -61,6 +82,79 @@ class LinuxSourceTreeOperations(object):
> >                 except subprocess.CalledProcessError as e:
> >                         raise ConfigError(e.output.decode())
> >
> > +       def make(self, jobs, build_dir, make_options) -> None:
> > +               command = ['make', 'ARCH=' + self._linux_arch, '--jobs=' + str(jobs)]
> > +               if make_options:
> > +                       command.extend(make_options)
> > +               if self._cross_compile:
> > +                       command += ['CROSS_COMPILE=' + self._cross_compile]
> > +               if build_dir:
> > +                       command += ['O=' + build_dir]
> > +               print('Building with:\n$', ' '.join(command))
> > +               try:
> > +                       proc = subprocess.Popen(command,
> > +                                               stderr=subprocess.PIPE,
> > +                                               stdout=subprocess.DEVNULL)
> > +               except OSError as e:
> > +                       raise BuildError('Could not call execute make: ' + str(e))
> > +               except subprocess.CalledProcessError as e:
> > +                       raise BuildError(e.output)
> > +               _, stderr = proc.communicate()
> > +               if proc.returncode != 0:
> > +                       raise BuildError(stderr.decode())
> > +               if stderr:  # likely only due to build warnings
> > +                       print(stderr.decode())
> > +
> > +       def run(self, params, timeout, build_dir, outfile) -> None:
> > +               pass
> > +
> > +
> > +class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations):
> > +
> > +       def __init__(self, qemu_arch_params: qemu_config.QemuArchParams, cross_compile: Optional[str]):
> > +               super().__init__(linux_arch=qemu_arch_params.linux_arch,
> > +                                cross_compile=cross_compile)
> > +               self._qemuconfig = qemu_arch_params.qemuconfig
> > +               self._qemu_arch = qemu_arch_params.qemu_arch
> > +               self._kernel_path = qemu_arch_params.kernel_path
> > +               self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot'
> > +               self._extra_qemu_params = qemu_arch_params.extra_qemu_params
> > +
> > +       def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None:
> > +               qemuconfig = kunit_config.Kconfig()
> > +               qemuconfig.parse_from_string(self._qemuconfig)
> > +               base_kunitconfig.merge_in_entries(qemuconfig)
> > +
> > +       def run(self, params, timeout, build_dir, outfile):
> > +               kernel_path = os.path.join(build_dir, self._kernel_path)
> > +               qemu_command = ['qemu-system-' + self._qemu_arch,
> > +                               '-nodefaults',
> > +                               '-m', '1024',
> > +                               '-kernel', kernel_path,
> > +                               '-append', '\'' + ' '.join(params + [self._kernel_command_line]) + '\'',
> > +                               '-no-reboot',
> > +                               '-nographic',
> > +                               '-serial stdio'] + self._extra_qemu_params
> > +               print('Running tests with:\n$', ' '.join(qemu_command))
> > +               with open(outfile, 'w') as output:
> > +                       process = subprocess.Popen(' '.join(qemu_command),
> > +                                                  stdin=subprocess.PIPE,
> > +                                                  stdout=output,
> > +                                                  stderr=subprocess.STDOUT,
> > +                                                  text=True, shell=True)
> > +               try:
> > +                       process.wait(timeout=timeout)
> > +               except Exception as e:
> > +                       print(e)
> > +                       process.terminate()
> > +               return process
> > +
> > +class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations):
> > +       """An abstraction over command line operations performed on a source tree."""
> > +
> > +       def __init__(self, cross_compile=None):
> > +               super().__init__(linux_arch='um', cross_compile=cross_compile)
> > +
> >         def make_allyesconfig(self, build_dir, make_options) -> None:
> >                 kunit_parser.print_with_timestamp(
> >                         'Enabling all CONFIGs for UML...')
> > @@ -83,32 +177,16 @@ class LinuxSourceTreeOperations(object):
> >                 kunit_parser.print_with_timestamp(
> >                         'Starting Kernel with all configs takes a few minutes...')
> >
> > -       def make(self, jobs, build_dir, make_options) -> None:
> > -               command = ['make', 'ARCH=um', '--jobs=' + str(jobs)]
> > -               if make_options:
> > -                       command.extend(make_options)
> > -               if build_dir:
> > -                       command += ['O=' + build_dir]
> > -               try:
> > -                       proc = subprocess.Popen(command,
> > -                                               stderr=subprocess.PIPE,
> > -                                               stdout=subprocess.DEVNULL)
> > -               except OSError as e:
> > -                       raise BuildError('Could not call make command: ' + str(e))
> > -               _, stderr = proc.communicate()
> > -               if proc.returncode != 0:
> > -                       raise BuildError(stderr.decode())
> > -               if stderr:  # likely only due to build warnings
> > -                       print(stderr.decode())
> > -
> > -       def linux_bin(self, params, timeout, build_dir) -> None:
> > +       def run(self, params, timeout, build_dir, outfile):
> >                 """Runs the Linux UML binary. Must be named 'linux'."""
> >                 linux_bin = get_file_path(build_dir, 'linux')
> >                 outfile = get_outfile_path(build_dir)
> >                 with open(outfile, 'w') as output:
> >                         process = subprocess.Popen([linux_bin] + params,
> > +                                                  stdin=subprocess.PIPE,
> >                                                    stdout=output,
> > -                                                  stderr=subprocess.STDOUT)
> > +                                                  stderr=subprocess.STDOUT,
> > +                                                  text=True)
> >                         process.wait(timeout)
> >
> >  def get_kconfig_path(build_dir) -> str:
> > @@ -120,13 +198,49 @@ def get_kunitconfig_path(build_dir) -> str:
> >  def get_outfile_path(build_dir) -> str:
> >         return get_file_path(build_dir, OUTFILE_PATH)
> >
> > +def get_source_tree_ops(arch: str, cross_compile: Optional[str]) -> LinuxSourceTreeOperations:
> > +       config_path = os.path.join(QEMU_CONFIGS_DIR, arch + '.py')
> > +       if arch == 'um':
> > +               return LinuxSourceTreeOperationsUml(cross_compile=cross_compile)
> > +       elif os.path.isfile(config_path):
> > +               return get_source_tree_ops_from_qemu_config(config_path, cross_compile)[1]
> > +       else:
> > +               raise ConfigError(arch + ' is not a valid arch')
> > +
> > +def get_source_tree_ops_from_qemu_config(config_path: str,
> > +                                        cross_compile: Optional[str]) -> tuple[
> > +                                                        str, LinuxSourceTreeOperations]:
> > +       abs_config_path = os.path.abspath(config_path)
> > +       if os.path.commonpath([ABS_TOOL_PATH, abs_config_path]) != ABS_TOOL_PATH:
> > +               raise ConfigError('Given QEMU config file is not in a child directory of KUnit tool.')
>
> I really don't like this requirement: it feels very arbitrary and
> undercuts the value of the --qemu_config option a bit: we almost might
> as well keep everything in the QEMU_CONFIG_DIR.
>
> I assume it's here because of the need to import the QemuArchParams
> definition: I take it there's no convenient way to do that?

I thought it was a requirement, but I was wrong. I was able to fix it.
In the next revision, no such requirement will exist.

> At the very least, let's document this restriction properly, as it was
> a bit of a weird one I wasn't expecting. Then people can either put
> their custom qemu configs in the qemu_configs/ directory, or in
> something like a custom_qemu_configs/ one. And if we later can work
> out a more convenient way of removing this restriction entirely, we
> can do so.

No worries, we won't have to have the configs live in any particular
location in future revisions.

> > +       relative_config_path = os.path.relpath(abs_config_path, ABS_TOOL_PATH)
> > +       spec = importlib.util.spec_from_file_location('.' + relative_config_path, config_path)
> > +       config = importlib.util.module_from_spec(spec)
> > +       # TODO(brendanhiggins@xxxxxxxxxx): I looked this up and apparently other
> > +       # Python projects have noted that pytype complains that "No attribute
> > +       # 'exec_module' on _importlib_modulespec._Loader". Disabling for now.
> > +       spec.loader.exec_module(config) # pytype: disable=attribute-error
> > +       return config.QEMU_ARCH.linux_arch, LinuxSourceTreeOperationsQemu(
> > +                       config.QEMU_ARCH, cross_compile=cross_compile)
> > +
> >  class LinuxSourceTree(object):
> >         """Represents a Linux kernel source tree with KUnit tests."""
> >
> > -       def __init__(self, build_dir: str, load_config=True, kunitconfig_path='') -> None:
> > +       def __init__(
> > +             self,
> > +             build_dir: str,
> > +             load_config=True,
> > +             kunitconfig_path='',
> > +             arch=None,
> > +             cross_compile=None,
> > +             qemu_config_path=None) -> None:
> >                 signal.signal(signal.SIGINT, self.signal_handler)
> > -
> > -               self._ops = LinuxSourceTreeOperations()
> > +               if qemu_config_path:
> > +                       self._arch, self._ops = get_source_tree_ops_from_qemu_config(
> > +                                       qemu_config_path, cross_compile)
> > +               else:
> > +                       self._arch = 'um' if arch is None else arch
> > +                       self._ops = get_source_tree_ops(self._arch, cross_compile)
> >
> >                 if not load_config:
> >                         return
> > @@ -170,8 +284,9 @@ class LinuxSourceTree(object):
> >                 kconfig_path = get_kconfig_path(build_dir)
> >                 if build_dir and not os.path.exists(build_dir):
> >                         os.mkdir(build_dir)
> > -               self._kconfig.write_to_file(kconfig_path)
> >                 try:
> > +                       self._ops.make_arch_qemuconfig(self._kconfig)
> > +                       self._kconfig.write_to_file(kconfig_path)
> >                         self._ops.make_olddefconfig(build_dir, make_options)
> >                 except ConfigError as e:
> >                         logging.error(e)
> > @@ -184,6 +299,7 @@ class LinuxSourceTree(object):
> >                 if os.path.exists(kconfig_path):
> >                         existing_kconfig = kunit_config.Kconfig()
> >                         existing_kconfig.read_from_file(kconfig_path)
> > +                       self._ops.make_arch_qemuconfig(self._kconfig)
> >                         if not self._kconfig.is_subset_of(existing_kconfig):
> >                                 print('Regenerating .config ...')
> >                                 os.remove(kconfig_path)
> > @@ -194,7 +310,7 @@ class LinuxSourceTree(object):
> >                         print('Generating .config ...')
> >                         return self.build_config(build_dir, make_options)
> >
> > -       def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
> > +       def build_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
> >                 try:
> >                         if alltests:
> >                                 self._ops.make_allyesconfig(build_dir, make_options)
> > @@ -211,8 +327,8 @@ class LinuxSourceTree(object):
> >                 args.extend(['mem=1G', 'console=tty','kunit_shutdown=halt'])
> >                 if filter_glob:
> >                         args.append('kunit.filter_glob='+filter_glob)
> > -               self._ops.linux_bin(args, timeout, build_dir)
> >                 outfile = get_outfile_path(build_dir)
> > +               self._ops.run(args, timeout, build_dir, outfile)
> >                 subprocess.call(['stty', 'sane'])
> >                 with open(outfile, 'r') as file:
> >                         for line in file:
> > diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
> > index 2e809dd956a77..a3b7f68e1cf9f 100755
> > --- a/tools/testing/kunit/kunit_tool_test.py
> > +++ b/tools/testing/kunit/kunit_tool_test.py
> > @@ -303,7 +303,7 @@ class KUnitMainTest(unittest.TestCase):
> >
> >                 self.linux_source_mock = mock.Mock()
> >                 self.linux_source_mock.build_reconfig = mock.Mock(return_value=True)
> > -               self.linux_source_mock.build_um_kernel = mock.Mock(return_value=True)
> > +               self.linux_source_mock.build_kernel = mock.Mock(return_value=True)
> >                 self.linux_source_mock.run_kernel = mock.Mock(return_value=all_passed_log)
> >
> >         def test_config_passes_args_pass(self):
> > @@ -314,7 +314,7 @@ class KUnitMainTest(unittest.TestCase):
> >         def test_build_passes_args_pass(self):
> >                 kunit.main(['build'], self.linux_source_mock)
> >                 self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 0)
> > -               self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '.kunit', None)
> > +               self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, '.kunit', None)
> >                 self.assertEqual(self.linux_source_mock.run_kernel.call_count, 0)
> >
> >         def test_exec_passes_args_pass(self):
> > @@ -396,7 +396,7 @@ class KUnitMainTest(unittest.TestCase):
> >         def test_build_builddir(self):
> >                 build_dir = '.kunit'
> >                 kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock)
> > -               self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None)
> > +               self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, build_dir, None)
> >
> >         def test_exec_builddir(self):
> >                 build_dir = '.kunit'
> > @@ -410,14 +410,22 @@ class KUnitMainTest(unittest.TestCase):
> >                 mock_linux_init.return_value = self.linux_source_mock
> >                 kunit.main(['run', '--kunitconfig=mykunitconfig'])
> >                 # Just verify that we parsed and initialized it correctly here.
> > -               mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
> > +               mock_linux_init.assert_called_once_with('.kunit',
> > +                                                       kunitconfig_path='mykunitconfig',
> > +                                                       arch='um',
> > +                                                       cross_compile=None,
> > +                                                       qemu_config_path=None)
> >
> >         @mock.patch.object(kunit_kernel, 'LinuxSourceTree')
> >         def test_config_kunitconfig(self, mock_linux_init):
> >                 mock_linux_init.return_value = self.linux_source_mock
> >                 kunit.main(['config', '--kunitconfig=mykunitconfig'])
> >                 # Just verify that we parsed and initialized it correctly here.
> > -               mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
> > +               mock_linux_init.assert_called_once_with('.kunit',
> > +                                                       kunitconfig_path='mykunitconfig',
> > +                                                       arch='um',
> > +                                                       cross_compile=None,
> > +                                                       qemu_config_path=None)
> >
> >  if __name__ == '__main__':
> >         unittest.main()
> > diff --git a/tools/testing/kunit/qemu_config.py b/tools/testing/kunit/qemu_config.py
> > new file mode 100644
> > index 0000000000000..aff1fe0442dbc
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_config.py
> > @@ -0,0 +1,17 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# Collection of configs for building non-UML kernels and running them on QEMU.
> > +#
> > +# Copyright (C) 2021, Google LLC.
> > +# Author: Brendan Higgins <brendanhiggins@xxxxxxxxxx>
> > +
> > +from collections import namedtuple
> > +
> > +
> > +QemuArchParams = namedtuple('QemuArchParams', ['linux_arch',
> > +                                              'qemuconfig',
> > +                                              'qemu_arch',
> > +                                              'kernel_path',
> > +                                              'kernel_command_line',
> > +                                              'extra_qemu_params'])
> > +
>
> Nit: newline at end of file.

Thanks, will fix.

> > diff --git a/tools/testing/kunit/qemu_configs/alpha.py b/tools/testing/kunit/qemu_configs/alpha.py
> > new file mode 100644
> > index 0000000000000..2cc64f848ca2c
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/alpha.py
> > @@ -0,0 +1,10 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='alpha',
> > +                          qemuconfig='''
> > +CONFIG_SERIAL_8250=y
> > +CONFIG_SERIAL_8250_CONSOLE=y''',
> > +                          qemu_arch='alpha',
> > +                          kernel_path='arch/alpha/boot/vmlinux',
> > +                          kernel_command_line='console=ttyS0',
> > +                          extra_qemu_params=[''])
> > diff --git a/tools/testing/kunit/qemu_configs/arm.py b/tools/testing/kunit/qemu_configs/arm.py
> > new file mode 100644
> > index 0000000000000..29a043b0531a0
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/arm.py
> > @@ -0,0 +1,13 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='arm',
> > +                          qemuconfig='''
> > +CONFIG_ARCH_VIRT=y
> > +CONFIG_SERIAL_AMBA_PL010=y
> > +CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
> > +CONFIG_SERIAL_AMBA_PL011=y
> > +CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
> > +                          qemu_arch='arm',
> > +                          kernel_path='arch/arm/boot/zImage',
> > +                          kernel_command_line='console=ttyAMA0',
> > +                          extra_qemu_params=['-machine virt'])
> > diff --git a/tools/testing/kunit/qemu_configs/arm64.py b/tools/testing/kunit/qemu_configs/arm64.py
> > new file mode 100644
> > index 0000000000000..1ba200bc99f0f
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/arm64.py
> > @@ -0,0 +1,12 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='arm64',
> > +                          qemuconfig='''
> > +CONFIG_SERIAL_AMBA_PL010=y
> > +CONFIG_SERIAL_AMBA_PL010_CONSOLE=y
> > +CONFIG_SERIAL_AMBA_PL011=y
> > +CONFIG_SERIAL_AMBA_PL011_CONSOLE=y''',
> > +                          qemu_arch='aarch64',
> > +                          kernel_path='arch/arm64/boot/Image.gz',
> > +                          kernel_command_line='console=ttyAMA0',
> > +                          extra_qemu_params=['-machine virt', '-cpu cortex-a57'])
> > diff --git a/tools/testing/kunit/qemu_configs/i386.py b/tools/testing/kunit/qemu_configs/i386.py
> > new file mode 100644
> > index 0000000000000..3998af306468e
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/i386.py
> > @@ -0,0 +1,10 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='i386',
> > +                          qemuconfig='''
> > +CONFIG_SERIAL_8250=y
> > +CONFIG_SERIAL_8250_CONSOLE=y''',
> > +                          qemu_arch='x86_64',
> > +                          kernel_path='arch/x86/boot/bzImage',
> > +                          kernel_command_line='console=ttyS0',
> > +                          extra_qemu_params=[''])
> > diff --git a/tools/testing/kunit/qemu_configs/powerpc.py b/tools/testing/kunit/qemu_configs/powerpc.py
> > new file mode 100644
> > index 0000000000000..46292ce9e368e
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/powerpc.py
> > @@ -0,0 +1,12 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='powerpc',
> > +                          qemuconfig='''
> > +CONFIG_PPC64=y
> > +CONFIG_SERIAL_8250=y
> > +CONFIG_SERIAL_8250_CONSOLE=y
> > +CONFIG_HVC_CONSOLE=y''',
> > +                          qemu_arch='ppc64',
> > +                          kernel_path='vmlinux',
> > +                          kernel_command_line='console=ttyS0',
> > +                          extra_qemu_params=['-M pseries', '-cpu power8'])
> > diff --git a/tools/testing/kunit/qemu_configs/riscv.py b/tools/testing/kunit/qemu_configs/riscv.py
> > new file mode 100644
> > index 0000000000000..de8c62d465723
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/riscv.py
> > @@ -0,0 +1,31 @@
> > +from ..qemu_config import QemuArchParams
> > +import os
> > +import os.path
> > +import sys
> > +
> > +GITHUB_OPENSBI_URL = 'https://github.com/qemu/qemu/raw/master/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin'
> > +OPENSBI_FILE = os.path.basename(GITHUB_OPENSBI_URL)
> > +
> > +if not os.path.isfile(OPENSBI_FILE):
> > +       print('\n\nOpenSBI file is not in the current working directory.\n'
> > +             'Would you like me to download it for you from:\n' + GITHUB_OPENSBI_URL + ' ?\n')
> > +       response = input('yes/[no]: ')
> > +       if response.strip() == 'yes':
> > +               os.system('wget ' + GITHUB_OPENSBI_URL)
> > +       else:
> > +               sys.exit()
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='riscv',
> > +                          qemuconfig='''
> > +CONFIG_SOC_VIRT=y
> > +CONFIG_SERIAL_8250=y
> > +CONFIG_SERIAL_8250_CONSOLE=y
> > +CONFIG_SERIAL_OF_PLATFORM=y
> > +CONFIG_SERIAL_EARLYCON_RISCV_SBI=y''',
> > +                          qemu_arch='riscv64',
> > +                          kernel_path='arch/riscv/boot/Image',
> > +                          kernel_command_line='console=ttyS0',
> > +                          extra_qemu_params=[
> > +                                          '-machine virt',
> > +                                          '-cpu rv64',
> > +                                          '-bios opensbi-riscv64-generic-fw_dynamic.bin'])
> > diff --git a/tools/testing/kunit/qemu_configs/s390.py b/tools/testing/kunit/qemu_configs/s390.py
> > new file mode 100644
> > index 0000000000000..04c90332f1098
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/s390.py
> > @@ -0,0 +1,14 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='s390',
> > +                          qemuconfig='''
> > +CONFIG_EXPERT=y
> > +CONFIG_TUNE_ZEC12=y
> > +CONFIG_NUMA=y
> > +CONFIG_MODULES=y''',
> > +                          qemu_arch='s390x',
> > +                          kernel_path='arch/s390/boot/bzImage',
> > +                          kernel_command_line='console=ttyS0',
> > +                          extra_qemu_params=[
> > +                                          '-machine s390-ccw-virtio',
> > +                                          '-cpu qemu',])
> > diff --git a/tools/testing/kunit/qemu_configs/sparc.py b/tools/testing/kunit/qemu_configs/sparc.py
> > new file mode 100644
> > index 0000000000000..f26b5f27cc5a1
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/sparc.py
> > @@ -0,0 +1,10 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='sparc',
> > +                          qemuconfig='''
> > +CONFIG_SERIAL_8250=y
> > +CONFIG_SERIAL_8250_CONSOLE=y''',
> > +                          qemu_arch='sparc',
> > +                          kernel_path='arch/sparc/boot/zImage',
> > +                          kernel_command_line='console=ttyS0 mem=256M',
> > +                          extra_qemu_params=['-m 256'])
> > diff --git a/tools/testing/kunit/qemu_configs/x86_64.py b/tools/testing/kunit/qemu_configs/x86_64.py
> > new file mode 100644
> > index 0000000000000..bd5ab733b92ac
> > --- /dev/null
> > +++ b/tools/testing/kunit/qemu_configs/x86_64.py
> > @@ -0,0 +1,10 @@
> > +from ..qemu_config import QemuArchParams
> > +
> > +QEMU_ARCH = QemuArchParams(linux_arch='x86_64',
> > +                          qemuconfig='''
> > +CONFIG_SERIAL_8250=y
> > +CONFIG_SERIAL_8250_CONSOLE=y''',
> > +                          qemu_arch='x86_64',
> > +                          kernel_path='arch/x86/boot/bzImage',
> > +                          kernel_command_line='console=ttyS0',
> > +                          extra_qemu_params=[''])
> > --
> > 2.31.1.607.g51e8a6a459-goog
> >



[Index of Archives]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]     [Linux Resources]

  Powered by Linux