On Wed, Jan 11, 2017 at 12:00:23PM +0100, Andrew Jones wrote:
> On Wed, Jan 11, 2017 at 01:29:35PM +0800, Peter Xu wrote:
> > run_task.sh is getting slow. This patch is trying to make it faster by
> > running the tests concurrently.
> >
> > We provide a new parameter "-j" for the run_tests.sh, which can be used
> > to specify how many run queues we want for the tests. Default queue
> > length is 1, which is the old behavior.
> >
> > Quick test on my laptop (4 cores, 2 threads each) shows 3x speed boost:
> >
> > |-----------------+-----------|
> > | command | time used |
> > |-----------------+-----------|
> > | run_test.sh | 75s |
> > | run_test.sh -j8 | 27s |
> > |-----------------+-----------|
> >
> > Signed-off-by: Peter Xu <peterx@xxxxxxxxxx>
> > ---
> > run_tests.sh | 12 ++++++++++--
> > scripts/common.bash | 16 +++++++++++++++-
> > 2 files changed, 25 insertions(+), 3 deletions(-)
> >
> > diff --git a/run_tests.sh b/run_tests.sh
> > index b6a1059..477d4fb 100755
> > --- a/run_tests.sh
> > +++ b/run_tests.sh
> > @@ -13,10 +13,11 @@ function usage()
> > {
> > cat <<EOF
> >
> > -Usage: $0 [-g group] [-h] [-v]
> > +Usage: $0 [-g group] [-h] [-v] [-j num_run_queues]
> >
> > -g: Only execute tests in the given group
> > -h: Output this help text
> > + -j: Execute tests in parallel
> > -v: Enables verbose mode
> >
> > Set the environment variable QEMU=/path/to/qemu-system-ARCH to
> > @@ -28,7 +29,7 @@ EOF
> > RUNTIME_arch_run="./$TEST_DIR/run"
> > source scripts/runtime.bash
> >
> > -while getopts "g:hv" opt; do
> > +while getopts "g:hj:v" opt; do
> > case $opt in
> > g)
> > only_group=$OPTARG
> > @@ -37,6 +38,13 @@ while getopts "g:hv" opt; do
> > usage
> > exit
> > ;;
> > + j)
> > + unittest_run_queues=$OPTARG
> > + if (( $unittest_run_queues <= 0 )); then
> > + echo "Invalid -j option: $unittest_run_queues"
> > + exit 1
>
> We should probably use 'exit 2' here, and below in the *) case.
>
> > + fi
> > + ;;
> > v)
> > verbose="yes"
> > ;;
> > diff --git a/scripts/common.bash b/scripts/common.bash
> > index 2dd7360..ef103ee 100644
> > --- a/scripts/common.bash
> > +++ b/scripts/common.bash
> > @@ -1,11 +1,19 @@
> > : ${unittest_log_dir:=logs}
> > +: ${unittest_run_queues:=1}
> >
> > function run_task()
> > {
> > local testname="$2"
> >
> > + while (( $(jobs | wc -l) == $unittest_run_queues )); do
> > + # wait for any background test to finish
> > + wait -n
> > + done
> > +
> > RUNTIME_log_file="${unittest_log_dir}/${testname}.log"
> > - "$@"
> > +
> > + # start the testcase in the background
> > + "$@" &
>
> If you check the logs before and after applying this patch series you'll
> see a bunch of "stty: 'standard input': Inappropriate ioctl for device"
> are now present. These messages come from the stty calls in run_qemu,
> which we need to avoid the loss of terminal echo when QEMU aborts. We
> can get rid of these new "inappropriate ioctl" messages by changing the
> above line to
>
> "$@" <$(tty) &
Actually I just came up with a better solution for this. Since we don't
need input to our unit tests, i.e. we never attempt to read from the
serial port within them, then we can just use /dev/null for stdin. That
allows us to leave your code above alone, and also remove the stty stuff
in run_qemu. I'll send a patch.
drew
>
>
> > }
> >
> > function for_each_unittest()
> > @@ -22,6 +30,8 @@ function for_each_unittest()
> > local accel
> > local timeout
> >
> > + trap "wait; exit 130" SIGINT
> > +
> > exec {fd}<"$unittests"
> >
> > while read -u $fd line; do
> > @@ -55,5 +65,9 @@ function for_each_unittest()
> > fi
> > done
> > run_task "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout"
> > +
> > + # wait until all task finish
>
> Still not quite right :-) s/task/tasks/
>
> > + wait
> > +
> > exec {fd}<&-
> > }
> > --
> > 2.7.4
> >
> >
>
> Thanks,
> drew
>
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
