Re: Are the xfstests exclusion files on wiki.samba.org up to date?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Sep 12, 2019 at 6:50 PM Murphy Zhou <jencce.kernel@xxxxxxxxx> wrote:
>
> On Mon, Sep 09, 2019 at 06:23:10PM -0500, Steve French wrote:
> > We have done a lot of work (with Ronnie, Aurelien, Pavel and others)
> > on xfstest automation.
>
> Kudos!
>
> >
> > As you might guess it is frustrating for two reasons:
> > 1) updated xfstests can be flaky (as the tests themselves are updated,
> > they add subtle required features, or regress from time to time)
> > 2) test automation can run into resource constraints (VMs trying to
> > run tests with less memory than might be optimal - especially those
> > run against Samba with "VMs inside VMs").
> >
> > But the good news is that we have VERY good data on which tests pass
> > to various servers (just as noted, need to update the external pages)
> > and we should have even more data as we go through two weeks of SMB3
> > testing events in late September.
> >
> > In general we want to test against all typical servers and have test
> > targets setup for
> >  - Samba (reasonably current stable) on ext4/xfs and Samba with btrfs
> > (which has various optional extensions enabled in the server)
> >  - Samba with POSIX Extensions
> >  - Windows (and against both NTFS and REFS server file system)
> > - Azure (Cloud0
>
> Thanks for sharing this!
>
> Besides server type, server configuration options and client mount
> options also make this matrix even bigger..
>
> How that is handled in the buildbot ?

We have several different targets, which maps to the local.config for
xfstests so for every single test we run
we also specify the different targets to run that for.

Here is an example:
all_tests = [
        [ "cifs/100", "smb3azureseal"],
        [ "cifs/101", "smb3multiuser"],
        [ "generic/001", "smb3sign", "smb3"],
        [ "generic/002", "smb3", "smb3sign", "smb21", "smb3samba"],
        [ "generic/005", "smb3", "smb21", "smb3samba", "smb3sambabtrfs"],
        [ "generic/006", "smb3"],
        [ "generic/007", "smb3"],
        [ "generic/010", "smb3"],
        [ "generic/011", "smb3"],
        [ "generic/013", "smb3samba"],
        [ "generic/014", "smb3"],
        [ "generic/020", "smb3samba"],
        [ "generic/023", "smb3samba"],
        [ "generic/024", "smb3", "smb3samba"],
        [ "generic/028", "smb3", "smb3samba"],
...


The first argument is which xfstest to run and the remainder of the
arguments are a list of which local.config.* files to use.
That way we can run a test against different servers and also
different configurations.
I try to keep the target names meaningful, so

"smb3" just means a basic smb3   windows 2016 server running in a vm
"smb3samba" is just a generic smb3 server with samba.
etc etc.


We started doing this upstream about a year ago which is when I
started setting it up
and now I am honest but since we do this for every pull request we
send to linus,
we have SIGNIFICANTLY increased the quality of cifs.ko.

We run quite a lot of tests nowadays and I always try to remind Steve
to "please put a link to the buildbot result in the pull request"
to bring awareness to our buildbot.

For upstream, here is the most recent test:
http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/249

cifs-testing is the most generic test target. We also have specific
targets that are just for Azure, and other targets, but the aim is
that the canonical cifs-target tests will all pass before we pass a
pull request onto linus.


Please contact me directly if you want more info about the buildbot. I
have one setup internally at rh too that runs many, but not all, the
tests we run upstream.
Any help to maintain and expand the tests are super welcome.


Rant-off
ronnie sahlberg

>
> >
> > But would really like to add test targets for other common servers and
> > in a perfect world would like to be able to have external test
> > automation pull our trees periodically to run these (similar to what
> > is done for the six targets above):
> >   - Macs
> >   - NetApp
> > (and any others of interest to the community)
> >
> > Obviously xfstest has a few hundred tests which only make sense for
> > local file systems with block devices, but there are hundreds of
> > xfstests of value, and most should run on cifs.ko and we are working
> > through them but already have a very good set running.
>
> Agree!  Thanks!
>
> Murphy
>
> >
> > On Mon, Sep 9, 2019 at 5:50 PM Aurélien Aptel <aaptel@xxxxxxxx> wrote:
> > >
> > > "Murphy Zhou" <jencce.kernel@xxxxxxxxx> writes:
> > > > As $subject. Is this wiki being maintained ?
> > > >
> > > > Looks like the last update was in January 2019.
> > > >
> > > > https://wiki.samba.org/index.php/Xfstesting-cifs#Exclusion_files
> > >
> > > We have a buildbot running xfstests relatively often here
> > >
> > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/
> > >
> > > Each group has slightly different tests run, you can see which one gets
> > > run by clicking on a build:
> > >
> > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/249
> > >
> > > Overall xfstests+cifs is very finicky and frustrating to get working
> > > reliably. Not to mention long. So good luck :)
> > >
> > > Cheers,
> > > --
> > > Aurélien Aptel / SUSE Labs Samba Team
> > > GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
> > > SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
> > > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)
> >
> >
> >
> > --
> > Thanks,
> >
> > Steve




[Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux