From: Martin Wilck <mwilck@xxxxxxxx> Until now, our CI only tested builds on Ubuntu, the default github runner. This commit adds containerized build/test workflows, which can cover a much larger range of distributions and environments. I've distinguished "native" runs, where the architecture is compatible with the architecture of the runners (assumed to be x86_64), and "foreign" runs with different architectures. The former can run "make test" just like in any ordinary environment. To run tests in "foreign arch" environments, I use qemu-user for runtime emulation. That's done with the help of the "multiarch/qemu-user-static" container. Runtime containers for testing need to have the qemu-user-static binary (compiled for x86_64) for their target architecture built in. In theory, we could do the same thing for build containers too, but it would be grossly inefficient. Instead, I've focused on Debian environments for the foreign architectures, relying on Debian's nice multiarch / cross-compilation features. Compilation is run in a dedicated cross-build container, and only the test runs are carried out in the emulation. The set of tests currently includes: native: - Debian Jessie, x86_64/i386 (gcc 4.9, clang 3.5, glibc 2.19) - Debian Buster, x86_64/i386 (gcc 8.3, clang 7.0, glibc 2.28) - Debian Sid, x86_64/i386 (gcc 10.2, clang 11.0, glibc 2.31) - Alpine, x86_64/i386 (gcc 10.2, clang 10.0, musl libc 1.2) - Fedora 34, x86_64 (gcc 11.0, clang 12.0, glibc 2.33) foreign: - Debian Buster, ppc64le/aarch64/s390x This covers a rather broad range of compiler and C library versions and should be fine for some time to come. Note: In theory, it would be possible to just fetch base containers via github actions, and install the dependencies as part of the build procedure. But that would be quite resource-intensive and slow. Therefore I've decided to use pre-built containers. The current container setup fetches containers from my docker hub repository. The containers there (multipath-build-$os-$arch and multipath-run-$os-$arch) come from my "build-multipath" repository (https://github.com/mwilck/build-multipath), and are created via github actions, too. The upload of the built container images to docker hub requires the use of tokens and secrets. I'd considered adding these container definitions and workflows to the multipath-tools repository. We'd just need to create reasonable rules for running the respective workflows. I expect these container images to remain relatively stable; it makes no sense to rebuild the images for every multipath-tools commit. Tell me if you want this in the multipath-tools repo, and if you're ok with hosting the images in my docker hub repo. Signed-off-by: Martin Wilck <mwilck@xxxxxxxx> --- .github/workflows/build-and-unittest.yaml | 4 +- .github/workflows/foreign.yaml | 65 +++++++++++++++++++++++ .github/workflows/native.yaml | 31 +++++++++++ 3 files changed, 99 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/foreign.yaml create mode 100644 .github/workflows/native.yaml diff --git a/.github/workflows/build-and-unittest.yaml b/.github/workflows/build-and-unittest.yaml index 4173576..bf37b13 100644 --- a/.github/workflows/build-and-unittest.yaml +++ b/.github/workflows/build-and-unittest.yaml @@ -1,5 +1,7 @@ name: basic-build-and-ci -on: [push] +on: + push: + branches: [master queue tip] jobs: bionic: runs-on: ubuntu-18.04 diff --git a/.github/workflows/foreign.yaml b/.github/workflows/foreign.yaml new file mode 100644 index 0000000..505a777 --- /dev/null +++ b/.github/workflows/foreign.yaml @@ -0,0 +1,65 @@ +name: compile and unit test on foreign arch +on: + push: + branches: + - master + - queue + - tip + +jobs: + + build: + runs-on: ubuntu-20.04 + strategy: + matrix: + os: [buster] + arch: ['ppc64le', 'aarch64', 's390x'] + container: mwilck/multipath-build-${{ matrix.os }}-${{ matrix.arch }} + steps: + - name: checkout + uses: actions/checkout@v1 + - name: build and test + if: ${{ matrix.arch == '' || matrix.arch == '-i386' }} + run: make test + - name: build + if: ${{ matrix.arch != '' && matrix.arch != '-i386' }} + run: make test-progs + - name: archive + if: ${{ matrix.arch != '' && matrix.arch != '-i386' }} + run: > + tar cfv binaries.tar + Makefile* + libmpathcmd/*.so* libmultipath/*.so* + tests/lib tests/*-test tests/Makefile tests/*.so* + - uses: actions/upload-artifact@v1 + if: ${{ matrix.arch != '' && matrix.arch != '-i386' }} + with: + name: multipath-${{ matrix.os }}-${{ matrix.arch }} + path: binaries.tar + + test: + runs-on: ubuntu-20.04 + needs: build + strategy: + matrix: + os: [buster] + arch: ['ppc64le', 'aarch64', 's390x'] + steps: + - name: get binaries + uses: actions/download-artifact@v1 + with: + name: multipath-${{ matrix.os }}-${{ matrix.arch }} + - name: unpack + run: tar xfv multipath-${{ matrix.os }}-${{ matrix.arch }}/binaries.tar + - name: enable foreign arch + run: sudo docker run --rm --privileged multiarch/qemu-user-static:register --reset + - name: run tests + # Github actions doesn't support referencing docker images with + # context variables. Workaround: use mosteo-actions/docker-run action + # See https://github.community/t/expressions-in-docker-uri/16271 + uses: mosteo-actions/docker-run@v1 + with: + image: mwilck/multipath-run-${{ matrix.os }}-${{ matrix.arch }} + # The runner is an image that has "make" as entrypoint + # So run "make -C tests" here + command: -C tests diff --git a/.github/workflows/native.yaml b/.github/workflows/native.yaml new file mode 100644 index 0000000..abd39a0 --- /dev/null +++ b/.github/workflows/native.yaml @@ -0,0 +1,31 @@ +name: compile and unit test on native arch +on: + push: + branches: + - master + - queue + - tip + +jobs: + build-and-test: + runs-on: ubuntu-20.04 + strategy: + matrix: + os: [buster, jessie, sid, alpine, fedora-34] + arch: ['', '-i386'] + exclude: + - os: fedora-34 + arch: '-i386' + container: mwilck/multipath-build-${{ matrix.os }}${{ matrix.arch }} + steps: + - name: checkout + uses: actions/checkout@v1 + - name: build and test + run: make test + - name: clean + run: make clean + - name: clang + env: + CC: clang + run: make test + -- 2.30.1 -- dm-devel mailing list dm-devel@xxxxxxxxxx https://listman.redhat.com/mailman/listinfo/dm-devel
