Running conformance tests on Windows

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




The attached script will let you compile and run the conformance tests
on Windows using Visual C++. Here's how:

 * first save the script as program/winetest/make_win32
 * then from Wine's top directory run
   ./program/winetest/make_win32 --zip
 * this will generate Visual C++ makefiles, and then a zip file
containing all you need.
 * uncompress this zip file on a Windows machine
 * retrieve the latest MS Windows SDK. The Visual C++ 6 headers are not
sufficient to compile the conformance tests.
 * run the SDK's SetEnv.bat script
 * in the top directory type:
   nmake -f wine_tests.mak

This will recursively build and run the tests. You can also go to a test
directory and just build or run the tests in that directory:

    cd dlls\ntdll\tests
    rem Just build the tests in that directory
    nmake -f wine_tests.mak build
    rem now run them too
    nmake -f wine_tests.mak test
    rem delete the result files
    nmake -f wine_tests.mak testclean
    rem re-run the tests, equivalent to the 'test' target
    nmake -f wine_tests.mak
    rem clean everything
    nmake -f wine_tests.mak clean

The zip file will also contain a batch file that can be used to run the
tests once compiled. So what you can do is compile the tests, and then
make a new zip file that contains just that batch file and the test
executables. This will make it possible for someone with no compiler to
run the tests.

   Oh, and make-win32 --help is your friend.


Early results:

Well, no very good.
 * First compiling the tests is not a given. The requirement of using a
recent Microsoft Windows SDK is quite annoying but there appears to be
no way around that.
 * Second, I would really like to be able to compile the tests directly
from Wine's sources just by mounting them via Samba. But then I get into
big mess between Wine's headers and the Windows headers. I'll keep
poking at this.
 * Many tests don't compile. But that's ok, I have patches pending that
solve most issues so that should no be an problem for long anymore.
 * And once you get them compiled quite a few of them fail! :-(
   That's why it is ultra important to run the conformance tests on Windows.


Changelog:

 * programs/winetest/make_win32
   programs/winetest/runtest.bat

   Script to generate Visual C++ makefiles and this make it possible to
compile and run the tests on Windows.


-- 
Francois Gouget         fgouget@free.fr        http://fgouget.free.fr/
     The software said it requires Win95 or better, so I installed Linux.
#!/usr/bin/perl -w
use strict;

# TODO:
# * generate makefiles for Cygwin/Mingw/etc.
# * make it possible to rezip just the executables and runtests.bat
# * generate dsp/dsw files?
# * make it possible to compile the tests directly from Wine's sources


my $name0=$0;
$name0 =~ s+^.*/++;

my $basename="wine_tests";

my $topdir;
my $do_zip;
my $do_tgz;
my $do_msvc=1;
my $to_archive;


###
#
# Helper functions
#
###

sub path_to_dotdot($)
{
    my $totop=$_[0];
    if (-f $totop)
    {
        $totop=~ s%/[^/]*$%%;
    }
    else
    {
        $totop=~ s%/$%%;
    }
    if ($totop !~ /^\//)
    {
        $totop=~ s%^./%%;
        $totop=~ s%/./%%;
        $totop=~ s%[^/]+%..%g;
    }
    return $totop;
}

###
#
# Generate Visual C++ Makefiles
#
###

sub write_list($)
{
    my $list=$_[0];

    foreach my $item (@$list)
    {
        print MAKEFILE " \\\r\n\t$item";
    }
    print MAKEFILE "\r\n" if (@$list);
}

sub write_msvc_makefile($$$$$)
{
    my $filename=$_[0];
    my $module=$_[1];
    my $imports=$_[2];
    my $perl_tests=$_[3];
    my $c_tests=$_[4];

    open(MAKEFILE,">$filename") || die "Could not write to $filename\n";

    my $totop=path_to_dotdot($filename);
    $totop=~ s+/+\\+g;
    print MAKEFILE "TOTOP = $totop\r\n";
    print MAKEFILE "\r\n";
    my $msvc_module=@$module[0];
    $msvc_module=~s/\.dll$//;
    print MAKEFILE "MODULE=$msvc_module\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE "IMPORTS = ";
    my @msvc_imports=();
    foreach my $dll (@$imports)
    {
        push @msvc_imports, "$dll.lib" if (defined $dll and $dll ne "ntdll");
    }
    print "import=[",join("|",@msvc_imports),"]\n";
    push @msvc_imports, "$msvc_module.lib" if ($msvc_module ne "ntdll");
    write_list(\@msvc_imports);
    print MAKEFILE "\r\n";
    # FIXME: We don't handle the Perl tests yet
    print MAKEFILE "PLTESTS = ";
    write_list($perl_tests);
    print MAKEFILE "\r\n";
    print MAKEFILE "CTESTS = ";
    write_list($c_tests);
    print MAKEFILE "\r\n";
    print MAKEFILE "CC = cl -nologo -c\r\n";
    print MAKEFILE "CFLAGS = -DWIN32 -D_X86_ -D__i386__\r\n";
    print MAKEFILE "INCLUDE=\$(TOTOP)\\include;\$(INCLUDE)\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE "RUNTEST      = \$(TOTOP)\\programs\\winetest\\runtest.bat\r\n";
    print MAKEFILE "RUNTESTFLAGS = \r\n";
    print MAKEFILE "TESTRESULTS  = \$(CTESTS:.c=.msvcok)\r\n";
    print MAKEFILE "TESTPROGRAM  = \$(MODULE)_test\r\n";
    print MAKEFILE "TESTLIST     = testlist.c\r\n";
    print MAKEFILE "TESTOBJS     = \$(TESTMAIN) \$(TESTLIST:.c=.obj) \$(CTESTS:.c=.obj)\r\n";
    print MAKEFILE "TESTMAIN     = \$(TOTOP)\\programs\\winetest\\wtmain.obj\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE "all: build test\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE "build: \$(TESTPROGRAM).exe\r\n";
    print MAKEFILE "test: build \$(CTESTS:.c=.msvcok)\r\n";
    print MAKEFILE "clean: testclean\r\n";
    print MAKEFILE "\tdel \$(TESTPROGRAM).exe \$(CTESTS:.c=.obj) 2>nul\r\n";
    print MAKEFILE "testclean:\r\n";
    print MAKEFILE "\tdel \$(CTESTS:.c=.msvcok) 2>nul\r\n";
    print MAKEFILE "\r\n";

    print MAKEFILE "\$(CTESTS:.c=.msvcok): \$(TESTPROGRAM).exe\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE ".c.msvcok:\r\n";
    print MAKEFILE "\t\$(RUNTEST) \$(TESTPROGRAM).exe \$(<B) \$\@ \$(RUNTESTFLAGS)\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE ".c.obj:\r\n";
    print MAKEFILE "\t\$(CC) \$(CFLAGS) -Fo\$*.obj \$<\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE "\$(TESTPROGRAM).exe: \$(TESTOBJS)\r\n";
    print MAKEFILE "\tlink -nologo -out:\$\@ \$(LDFLAGS) \$(TESTOBJS) \$(IMPORTS:%=%.lib)\r\n";

    close(MAKEFILE);

    # List the files that need to be archived
    print "building archive list\n";
    my $dirname=$filename;
    $dirname =~ s+/[^/]*$++;
    $to_archive->{$filename}=1;
    foreach my $file (@$perl_tests,@$c_tests)
    {
        print "   $dirname/$file\n";
        $to_archive->{"$dirname/$file"}=1;
    }
    if (@$c_tests)
    {
        $to_archive->{"$dirname/testlist.c"}=1;
    }
}

sub write_msvc_top_makefile($$)
{
    my $filename=$_[0];
    my $subdirs=$_[1];

    open(MAKEFILE,">$filename") || die "Could not write to $filename\n";

    print MAKEFILE "SUBDIRS = ";
    # FIXME: is this modifying the content of $subdirs"
    my @dirs=map { s+^./++; s+/+\\+g; "$_.dir"; } @$subdirs;
    write_list(\@dirs);
    print MAKEFILE "\r\n";
    print MAKEFILE "all: test\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE "build: \$(SUBDIRS:.dir=.build)\r\n";
    print MAKEFILE "\$(SUBDIRS:.dir=.build):\r\n";
    print MAKEFILE "\tcd \$*\r\n";
    print MAKEFILE "\tnmake -nologo -f $basename.mak build\r\n";
    print MAKEFILE "\tcd \$(MAKEDIR)\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE "test: \$(SUBDIRS:.dir=.test)\r\n";
    print MAKEFILE "\$(SUBDIRS:.dir=.test):\r\n";
    print MAKEFILE "\tcd \$*\r\n";
    print MAKEFILE "\tnmake -nologo -f $basename.mak test\r\n";
    print MAKEFILE "\tcd \$(MAKEDIR)\r\n";
    print MAKEFILE "\r\n";
    print MAKEFILE "clean: \$(SUBDIRS:.dir=.clean)\r\n";
    print MAKEFILE "\$(SUBDIRS:.dir=.clean):\r\n";
    print MAKEFILE "\tcd \$*\r\n";
    print MAKEFILE "\tnmake -nologo -f $basename.mak clean\r\n";
    print MAKEFILE "\tcd \$(MAKEDIR)\r\n";

    close(MAKEFILE);

    # List the files that need to be archived
    my $dirname=$filename;
    $dirname =~ s+/[^/]*$++;
    $to_archive->{$filename}=1;
    $to_archive->{"$dirname/include/wine/test.h"}=1;
    $to_archive->{"$dirname/include/winternl.h"}=1;
    $to_archive->{"$dirname/programs/winetest/wtmain.c"}=1;
    $to_archive->{"$dirname/programs/winetest/runtest.bat"}=1;
}


###
#
# Generate Mingw/Cygwin Makefiles
#
###

# To change the default location of the Cygwin / Mingw / Wine headers,
# modify the following variables.
my $cygwin_windows_includes ="/usr/include/w32api";
my $mingw_windows_includes ="/mingw/include";
my $wine_windows_includes ="./include";

sub write_cyg_makefile($$$$$)
{
    my $filename=$_[0];
    my $module=$_[1];
    my $imports=$_[2];
    my $perl_tests=$_[3];
    my $c_tests=$_[4];

    # FIXME: TODO
}


###
#
# Generate a batch file to run all the tests
# (for those who don't have a C compiler on Windows)
#
###

sub write_runtests_bat($$$$$)
{
    my $filename=$_[0];
    my $module=$_[1];
    my $imports=$_[2];
    my $perl_tests=$_[3];
    my $c_tests=$_[4];

    open(FILE,">$filename") || die "Could not write to $filename\n";

    my $totop=path_to_dotdot($filename);
    $totop=~ s+/+\\+g;
    my $module=@$module[0];
    $module=~s/\.dll$//;

    print FILE "\r\n";
    print FILE "nmake /f wine_tests.mak test\r\n";
    print FILE "goto end\r\n";
    print FILE "\@set RUNTESTFLAGS=\r\n";
    print FILE "\r\n";

    foreach my $test (@$c_tests)
    {
        $test =~ s/\.c$//;
        print FILE "call $totop\\programs\\winetest\\runtest.bat ${module}_test.exe $test $test.msvcok \%RUNTESTFLAGS\%\r\n";
        print FILE "if errorlevel 1 goto end\r\n";
    }
    print FILE "\r\n";
    print FILE ":end\r\n";

    close(FILE);

    # List the files that need to be archived
    $to_archive->{$filename}=1;
}


###
#
# Read Wine's makefiles
#
###

sub read_makefile($)
{
    my $filename=$_[0];
    my $module=[];
    my $imports=[];
    my $perl_tests=[];
    my $c_tests=[];
    my $subdirs=[];

    if (open(MAKEFILE,$filename))
    {
        my $current_list;
        my $dirname=$filename;
        if (!($dirname =~ s+/[^/]*$++))
        {
            $dirname=".";
        }

        while (<MAKEFILE>)
        {
            if (s/^\s*TESTDLL\s*=//)
            {
                $current_list=$module;
            }
            elsif (s/^\s*IMPORTS\s*=//)
            {
                $current_list=$imports;
            }
            elsif (s/^\s*(LIBPROG)?SUBDIRS\s*=\s*//)
            {
                $current_list=$subdirs;
            }
            elsif (s/^\s*BASEDIRS\s*=\s*//)
            {
                $current_list=$subdirs;
            }
            elsif (s/^\s*PLTESTS\s*=//)
            {
                $current_list=$perl_tests;
            }
            elsif (s/^\s*CTESTS\s*=\s*//)
            {
                $current_list=$c_tests;
            }
            if (defined $current_list)
            {
                my $continue=(s/\\$//?1:0);
                s/^\s*//;
                push @$current_list, split /\s+/;
                undef $current_list if (!$continue);
            }
        }
        close(MAKEFILE);
    }
    return ($module,$imports,$perl_tests,$c_tests,$subdirs);
}

sub process_makefile($$);
sub process_makefile($$)
{
    my $test_dirs=$_[0];
    my $filename=$_[1];
    my $dirname=$filename;
    $dirname =~ s+/[^/]*$++;

    #print "reading $filename (in $dirname)\n";
    my ($module,$imports,$perl_tests,$c_tests,$subdirs)=read_makefile($filename);
    if (@$module>0 and (@$perl_tests>0 or @$c_tests>0))
    {
        print "$dirname has tests\n";
        push @$test_dirs,$dirname;
        if (@$c_tests)
        {
            # create the testslist.c file
            my $totop=path_to_dotdot($dirname);
            print("$topdir/programs/winetest/make_ctests @$c_tests >$dirname/testlist.c");
            system("$topdir/programs/winetest/make_ctests @$c_tests >$dirname/testlist.c");
        }
        if ($do_msvc)
        {
            write_msvc_makefile("$dirname/$basename.mak",$module,$imports,$perl_tests,$c_tests);
            write_runtests_bat("$dirname/runtests.bat",$module,$imports,$perl_tests,$c_tests);
        }
    }
    foreach my $subdir (@$subdirs)
    {
        print "$dirname/$subdir/Makefile.in\n";
        if (-e "$dirname/$subdir/Makefile.in")
        {
            process_makefile($test_dirs,"$dirname/$subdir/Makefile.in");
        }
    }
}


###
#
# Main: Parse command-line options
#
###

my $usage;
while (@ARGV > 0)
{
    my $arg = shift @ARGV;
    if ($arg eq "--top")
    {
        if (defined $topdir)
        {
            print STDERR "$name0:error: --topdir can only be specified once\n";
            $usage = 1;
            last;
        }
        if (!@ARGV)
        {
            print STDERR "$name0:error: missing argument for --topdir\n";
            $usage = 1;
            last;
        }
        $topdir = shift @ARGV;
    }
    elsif ($arg eq "--zip")
    {
      $do_zip = 1;
    }
    elsif ($arg eq "--tgz")
    {
      $do_tgz = 1;
    }
    elsif ($arg eq "--msvc")
    {
      $do_msvc = 1;
    }
    elsif ($arg eq "--no-msvc")
    {
      undef $do_msvc;
    }
    elsif ($arg eq "-?" or $arg eq "-h" or $arg eq "--help")
    {
        $usage = 2;
    }
    else
    {
        print STDERR "$name0:error: unknown option\n";
        $usage = 1;
    }
}

# Check or detect $topdir
if (defined $topdir)
{
    if (! -f "$topdir/configure")
    {
        print STDERR "$name0:error: wrong --topdir argument, $topdir/configure does not exist\n";
        $usage = 1;
    }
}
else
{
    # try to detect it automatically
    if (-f "./configure") { $topdir = "."; }
    elsif (-f "../configure") { $topdir = ".."; }
    elsif (-f "../../configure") { $topdir = "../.."; }
    elsif (-f "../../../configure") { $topdir = "../../.."; }
    else {
        print STDERR "$name0:error: could not locate 'configure'. Use the --topdir option.\n";
        $usage = 1;
    }
}
if ($usage)
{
    print STDERR "Usage: $name0 [--help] [--topdir directory]\n";
    print STDERR "              [--zip] [--tgz] [--msvc|--no-msvc]\n";
    print STDERR "Options:\n";
    print STDERR "\n";
    print STDERR "   --topdir  specifies the Wine source directory (autodetected if not specified)\n";
    print STDERR "   --zip     zip all needed files for test (default)\n";
    print STDERR "   --tgz     archive (tar.gz) all needed files for test (default)\n";
    print STDERR "   --msvc    create makefiles for MS Visual C++ (default)\n";
    print STDERR "   --no-msvc do not create makefiles for MS Visual C++\n";
    print STDERR "   --help,-h shows this help message\n";

    exit $usage;
}


###
#
# Start generating files
#
###
my $test_dirs=[];
process_makefile($test_dirs,"$topdir/Makefile.in");
if ($do_msvc)
{
    write_msvc_top_makefile("$topdir/$basename.mak",$test_dirs);
}
if ($do_zip)
{
    system("zip","$basename.zip",keys %$to_archive);
}
if ($do_tgz)
{
    system("tar","cfz","$basename.tar.gz",keys %$to_archive);
}

Attachment: runtest.bat
Description: runtest.bat


[Index of Archives]     [Gimp for Windows]     [Red Hat]     [Samba]     [Yosemite Camping]     [Graphics Cards]     [Wine Home]

  Powered by Linux