On Thu, Sep 12, 2019 at 07:06:52PM +1000, ronnie sahlberg wrote: > On Thu, Sep 12, 2019 at 6:50 PM Murphy Zhou <jencce.kernel@xxxxxxxxx> wrote: > > > > On Mon, Sep 09, 2019 at 06:23:10PM -0500, Steve French wrote: > > > We have done a lot of work (with Ronnie, Aurelien, Pavel and others) > > > on xfstest automation. > > > > Kudos! > > > > > > > > As you might guess it is frustrating for two reasons: > > > 1) updated xfstests can be flaky (as the tests themselves are updated, > > > they add subtle required features, or regress from time to time) > > > 2) test automation can run into resource constraints (VMs trying to > > > run tests with less memory than might be optimal - especially those > > > run against Samba with "VMs inside VMs"). > > > > > > But the good news is that we have VERY good data on which tests pass > > > to various servers (just as noted, need to update the external pages) > > > and we should have even more data as we go through two weeks of SMB3 > > > testing events in late September. > > > > > > In general we want to test against all typical servers and have test > > > targets setup for > > > - Samba (reasonably current stable) on ext4/xfs and Samba with btrfs > > > (which has various optional extensions enabled in the server) > > > - Samba with POSIX Extensions > > > - Windows (and against both NTFS and REFS server file system) > > > - Azure (Cloud0 > > > > Thanks for sharing this! > > > > Besides server type, server configuration options and client mount > > options also make this matrix even bigger.. > > > > How that is handled in the buildbot ? > > We have several different targets, which maps to the local.config for > xfstests so for every single test we run > we also specify the different targets to run that for. > > Here is an example: > all_tests = [ > [ "cifs/100", "smb3azureseal"], > [ "cifs/101", "smb3multiuser"], > [ "generic/001", "smb3sign", "smb3"], > [ "generic/002", "smb3", "smb3sign", "smb21", "smb3samba"], > [ "generic/005", "smb3", "smb21", "smb3samba", "smb3sambabtrfs"], > [ "generic/006", "smb3"], > [ "generic/007", "smb3"], > [ "generic/010", "smb3"], > [ "generic/011", "smb3"], > [ "generic/013", "smb3samba"], > [ "generic/014", "smb3"], > [ "generic/020", "smb3samba"], > [ "generic/023", "smb3samba"], > [ "generic/024", "smb3", "smb3samba"], > [ "generic/028", "smb3", "smb3samba"], > ... > > > The first argument is which xfstest to run and the remainder of the > arguments are a list of which local.config.* files to use. > That way we can run a test against different servers and also > different configurations. This is smart. :) > I try to keep the target names meaningful, so > > "smb3" just means a basic smb3 windows 2016 server running in a vm > "smb3samba" is just a generic smb3 server with samba. > etc etc. > > > We started doing this upstream about a year ago which is when I > started setting it up > and now I am honest but since we do this for every pull request we > send to linus, > we have SIGNIFICANTLY increased the quality of cifs.ko. /Thumbs up! > > We run quite a lot of tests nowadays and I always try to remind Steve > to "please put a link to the buildbot result in the pull request" > to bring awareness to our buildbot. > > For upstream, here is the most recent test: > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/249 > > cifs-testing is the most generic test target. We also have specific > targets that are just for Azure, and other targets, but the aim is > that the canonical cifs-target tests will all pass before we pass a > pull request onto linus. > > > Please contact me directly if you want more info about the buildbot. I > have one setup internally at rh too that runs many, but not all, the > tests we run upstream. Sure! Have replied to you. Thank you very much for sharing! If anybody resting cifs or I know that the expected setup and result of every tests for you developers, it will save much time of investigation and report false alarms. After several months working on cifs, I've got a feeling that cifs is much different from other Linux filesystems. Maybe because of Windows or the SMB protocols. I'm wondering that xfstests which is a typical testsuite for Unix/Linux filesystems, maybe is not that suitable for cifs. I agree with Steve that there are hundreds of tests suitable. If, only if, :) if we could know how MS testing NTFS or SMB protocols, would that help or more suitable? Thanks! Murphy > Any help to maintain and expand the tests are super welcome. > > > Rant-off > ronnie sahlberg > > > > > > > > > But would really like to add test targets for other common servers and > > > in a perfect world would like to be able to have external test > > > automation pull our trees periodically to run these (similar to what > > > is done for the six targets above): > > > - Macs > > > - NetApp > > > (and any others of interest to the community) > > > > > > Obviously xfstest has a few hundred tests which only make sense for > > > local file systems with block devices, but there are hundreds of > > > xfstests of value, and most should run on cifs.ko and we are working > > > through them but already have a very good set running. > > > > Agree! Thanks! > > > > Murphy > > > > > > > > On Mon, Sep 9, 2019 at 5:50 PM Aurélien Aptel <aaptel@xxxxxxxx> wrote: > > > > > > > > "Murphy Zhou" <jencce.kernel@xxxxxxxxx> writes: > > > > > As $subject. Is this wiki being maintained ? > > > > > > > > > > Looks like the last update was in January 2019. > > > > > > > > > > https://wiki.samba.org/index.php/Xfstesting-cifs#Exclusion_files > > > > > > > > We have a buildbot running xfstests relatively often here > > > > > > > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/ > > > > > > > > Each group has slightly different tests run, you can see which one gets > > > > run by clicking on a build: > > > > > > > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/249 > > > > > > > > Overall xfstests+cifs is very finicky and frustrating to get working > > > > reliably. Not to mention long. So good luck :) > > > > > > > > Cheers, > > > > -- > > > > Aurélien Aptel / SUSE Labs Samba Team > > > > GPG: 1839 CB5F 9F5B FB9B AA97 8C99 03C8 A49B 521B D5D3 > > > > SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE > > > > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München) > > > > > > > > > > > > -- > > > Thanks, > > > > > > Steve
