Re: [PATCH] tests: fix mocking of stat() / lstat() functions

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 4/2/19 10:58 AM, Daniel P. Berrangé wrote:
> On Mon, Apr 01, 2019 at 06:03:29PM +0100, Daniel P. Berrangé wrote:
>> Quite a few of the tests have a need to mock the stat() / lstat()
>> functions and they are taking somewhat different & inconsistent
>> approaches none of which are actually fully correct. This is shown
>> by fact that 'make check' fails on 32-bit hosts. Investigation
>> revealed that the code was calling into the native C library impl,
>> not getting intercepted by our mocks.
>>
>> The POSIX stat() function might resolve to any number of different
>> symbols in the C library.
>>
>> The may be an additional stat64() function exposed by the headers
>> too.
>>
>> On 64-bit hosts the stat & stat64 functions are identical, always
>> refering to the 64-bit ABI.
>>
>> On 32-bit hosts they refer to the 32-bit & 64-bit ABIs respectively.
>>
>> Libvirt uses _FILE_OFFSET_BITS=64 on 32-bit hosts, which causes the
>> C library to transparently rewrite stat() calls to be stat64() calls.
>> Libvirt will never see the 32-bit ABI from the traditional stat()
>> call. We cannot assume this rewriting is done using a macro. It might
>> be, but on GLibC it is done with a magic __asm__ statement to apply
>> the rewrite at link time instead of at preprocessing.
>>
>> In GLibC there may be two additional functions exposed by the headers,
>> __xstat() and __xstat64(). When these exist, stat() and stat64() are
>> transparently rewritten to call __xstat() and __xstat64() respectively.
>> The former symbols will not actally exist in the library at all, only
>> the header. The leading "__" indicates the symbols are a private impl
>> detail of the C library that applications should not care about.
>> Unfortunately, because we are trying to mock replace the C library,
>> we need to know about this internal impl detail.
>>
>> With all this in mind the list of functions we have to mock will depend
>> on several factors
>>
>>   - If _FILE_OFFSET_BITS is set, then we are on a 32-bit host, and we
>>     only need to mock stat64 and __xstat64. The other stat / __xstat
>>     functions exist, but we'll never call them so they can be ignored
>>     for mocking.
>>
>>   - If _FILE_OFFSET_BITS is not set, then we are on a 64-bit host and
>>     we should mock stat, stat64, __xstat & __xstat64. Either may be
>>     called by app code.
>>
>>   - If __xstat & __xstat64 exist, then stat & stat64 will not exist
>>     as symbols in the library, so the latter should not be mocked.
>>
>> The same all applies to lstat()
>>
>> These rules are complex enough that we don't want to duplicate them
>> across every mock file, so this centralizes all the logic in a helper
>> file virmockstathelper.c that should be #included when needed. The
>> code merely need to provide a filename rewriting callback called
>> virMockStatRedirect(). Optionally VIR_MOCK_STAT_HOOK can be defined
>> as a macro if further processing is needed inline.
>>
>> Signed-off-by: Daniel P. Berrangé <berrange@xxxxxxxxxx>
>> ---
>>   build-aux/mock-noinline.pl |   3 +
>>   cfg.mk                     |   4 +-
>>   tests/qemusecuritymock.c   | 131 +++++-----------
>>   tests/vircgroupmock.c      | 146 +++--------------
>>   tests/virfilewrapper.c     |  85 ++--------
>>   tests/virmock.h            |  11 --
>>   tests/virmockstathelpers.c | 310 +++++++++++++++++++++++++++++++++++++
>>   tests/virpcimock.c         |  93 +----------
>>   tests/virtestmock.c        | 140 +----------------
>>   9 files changed, 396 insertions(+), 527 deletions(-)
>>   create mode 100644 tests/virmockstathelpers.c
>>
>> NB I have tested on Fedora 32-bit and 64-bit but have not yet
>> tested on other platforms, so they may break. This is to be
>> investigated before pushing so we don't break other platforms
>> more than they are already broken wrt mocking :-)
> 
> It passes on FreeBSD 11, but fails on RHEL-7, so need a v2 of this.
> 

I've just finished testing on all 32bit systems I have (fedora i686 and gentoo on rpi). It passes on all of them.


On rhel7 I see qemufirmwaretest and qemuxml2argvtest failing and I've tracked it down to qemuFirmwareBuildFileList(). When it's iterating over given directory, it can see all those json files but ent->d_type is DT_UNKNOWN:

qemuFirmwareBuildFileList (files=0x777de0, dir=0x50df88 "/usr/share/qemu/firmware") at qemu/qemu_firmware.c:912

928             if (ent->d_type != DT_REG && ent->d_type != DT_LNK)
qemuFirmwareBuildFileList 1 $ p *ent
$1 = {d_ino = 961973, d_off = 9, d_reclen = 32, d_type = 0 '\000', d_name = "40-bios.json\000\212\002\017\000\000\000\000\000\r\000\000\000\000\000\000\000(\000\000\065\060-ovmf-sb-keys.json\000\262\034\017\000\000\000\000\000\021\000\000\000\000\000\000\000(\000\000\066

So the actual bug might be in the code we're testing because it's not handling DT_UKNOWN as manpage says. This looks like a fix:



diff --git a/src/qemu/qemu_firmware.c b/src/qemu/qemu_firmware.c
index 4ce010c..7d58122 100644
--- a/src/qemu/qemu_firmware.c
+++ b/src/qemu/qemu_firmware.c
@@ -924,8 +924,9 @@ qemuFirmwareBuildFileList(virHashTablePtr files, const char *dir)
     while ((rc = virDirRead(dirp, &ent, dir)) > 0) {
         VIR_AUTOFREE(char *) filename = NULL;
         VIR_AUTOFREE(char *) path = NULL;
+        struct stat sb;
 
-        if (ent->d_type != DT_REG && ent->d_type != DT_LNK)
+        if (ent->d_type != DT_REG && ent->d_type != DT_LNK && ent->d_type != DT_UNKNOWN)
             continue;
 
         if (STRPREFIX(ent->d_name, "."))
@@ -937,6 +938,11 @@ qemuFirmwareBuildFileList(virHashTablePtr files, const char *dir)
         if (virAsprintf(&path, "%s/%s", dir, filename) < 0)
             goto cleanup;
 
+        if (ent->d_type == DT_UNKNOWN &&
+            stat(path, &sb) >= 0 &&
+            ((sb.st_mode & S_IFMT) != S_IFREG && (sb.st_mode & S_IFMT) != S_IFLNK))
+            continue;
+
         if (virHashUpdateEntry(files, filename, path) < 0)
             goto cleanup;


So how about I send this as a separate patch and ACK yours? There are more places where DT_UNKNOWN is not handled though.


BTW: squash this into your patch:

diff --git i/tests/Makefile.am w/tests/Makefile.am
index d3cdbff8bb..1319c3b12c 100644
--- i/tests/Makefile.am
+++ w/tests/Makefile.am
@@ -146,6 +146,7 @@ EXTRA_DIST = \
        virjsondata \
        virmacmaptestdata \
        virmock.h \
+       virmockstathelpers.c \
        virnetdaemondata \
        virnetdevtestdata \
        virnwfilterbindingxml2xmldata \

Thanks for looking into this.

Michal

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux