On Fri, Sep 13, 2019 at 05:03:57PM -0700, Steve French wrote: > Actually the xfstests we pass compared favorably with other network and > cluster fs but just different :) Agreed :) M > > On Fri, Sep 13, 2019, 17:02 Murphy Zhou <jencce.kernel@xxxxxxxxx> wrote: > > > On Thu, Sep 12, 2019 at 07:06:52PM +1000, ronnie sahlberg wrote: > > > On Thu, Sep 12, 2019 at 6:50 PM Murphy Zhou <jencce.kernel@xxxxxxxxx> > > wrote: > > > > > > > > On Mon, Sep 09, 2019 at 06:23:10PM -0500, Steve French wrote: > > > > > We have done a lot of work (with Ronnie, Aurelien, Pavel and others) > > > > > on xfstest automation. > > > > > > > > Kudos! > > > > > > > > > > > > > > As you might guess it is frustrating for two reasons: > > > > > 1) updated xfstests can be flaky (as the tests themselves are > > updated, > > > > > they add subtle required features, or regress from time to time) > > > > > 2) test automation can run into resource constraints (VMs trying to > > > > > run tests with less memory than might be optimal - especially those > > > > > run against Samba with "VMs inside VMs"). > > > > > > > > > > But the good news is that we have VERY good data on which tests pass > > > > > to various servers (just as noted, need to update the external pages) > > > > > and we should have even more data as we go through two weeks of SMB3 > > > > > testing events in late September. > > > > > > > > > > In general we want to test against all typical servers and have test > > > > > targets setup for > > > > > - Samba (reasonably current stable) on ext4/xfs and Samba with btrfs > > > > > (which has various optional extensions enabled in the server) > > > > > - Samba with POSIX Extensions > > > > > - Windows (and against both NTFS and REFS server file system) > > > > > - Azure (Cloud0 > > > > > > > > Thanks for sharing this! > > > > > > > > Besides server type, server configuration options and client mount > > > > options also make this matrix even bigger.. > > > > > > > > How that is handled in the buildbot ? > > > > > > We have several different targets, which maps to the local.config for > > > xfstests so for every single test we run > > > we also specify the different targets to run that for. > > > > > > Here is an example: > > > all_tests = [ > > > [ "cifs/100", "smb3azureseal"], > > > [ "cifs/101", "smb3multiuser"], > > > [ "generic/001", "smb3sign", "smb3"], > > > [ "generic/002", "smb3", "smb3sign", "smb21", "smb3samba"], > > > [ "generic/005", "smb3", "smb21", "smb3samba", "smb3sambabtrfs"], > > > [ "generic/006", "smb3"], > > > [ "generic/007", "smb3"], > > > [ "generic/010", "smb3"], > > > [ "generic/011", "smb3"], > > > [ "generic/013", "smb3samba"], > > > [ "generic/014", "smb3"], > > > [ "generic/020", "smb3samba"], > > > [ "generic/023", "smb3samba"], > > > [ "generic/024", "smb3", "smb3samba"], > > > [ "generic/028", "smb3", "smb3samba"], > > > ... > > > > > > > > > The first argument is which xfstest to run and the remainder of the > > > arguments are a list of which local.config.* files to use. > > > That way we can run a test against different servers and also > > > different configurations. > > > > This is smart. :) > > > > > I try to keep the target names meaningful, so > > > > > > "smb3" just means a basic smb3 windows 2016 server running in a vm > > > "smb3samba" is just a generic smb3 server with samba. > > > etc etc. > > > > > > > > > We started doing this upstream about a year ago which is when I > > > started setting it up > > > and now I am honest but since we do this for every pull request we > > > send to linus, > > > we have SIGNIFICANTLY increased the quality of cifs.ko. > > > > /Thumbs up! > > > > > > > > We run quite a lot of tests nowadays and I always try to remind Steve > > > to "please put a link to the buildbot result in the pull request" > > > to bring awareness to our buildbot. > > > > > > For upstream, here is the most recent test: > > > > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/249 > > > > > > cifs-testing is the most generic test target. We also have specific > > > targets that are just for Azure, and other targets, but the aim is > > > that the canonical cifs-target tests will all pass before we pass a > > > pull request onto linus. > > > > > > > > > Please contact me directly if you want more info about the buildbot. I > > > have one setup internally at rh too that runs many, but not all, the > > > tests we run upstream. > > > > Sure! Have replied to you. Thank you very much for sharing! > > > > If anybody resting cifs or I know that the expected setup and result of > > every tests for you developers, it will save much time of investigation > > and report false alarms. > > > > After several months working on cifs, I've got a feeling that cifs is > > much different from other Linux filesystems. Maybe because of Windows > > or the SMB protocols. I'm wondering that xfstests which is a typical > > testsuite for Unix/Linux filesystems, maybe is not that suitable for > > cifs. I agree with Steve that there are hundreds of tests suitable. > > If, only if, :) if we could know how MS testing NTFS or SMB protocols, > > would that help or more suitable? > > > > > > Thanks! > > Murphy > > > > > > > Any help to maintain and expand the tests are super welcome. > > > > > > > > > Rant-off > > > ronnie sahlberg > > > > > > > > > > > > > > > > > But would really like to add test targets for other common servers > > and > > > > > in a perfect world would like to be able to have external test > > > > > automation pull our trees periodically to run these (similar to what > > > > > is done for the six targets above): > > > > > - Macs > > > > > - NetApp > > > > > (and any others of interest to the community) > > > > > > > > > > Obviously xfstest has a few hundred tests which only make sense for > > > > > local file systems with block devices, but there are hundreds of > > > > > xfstests of value, and most should run on cifs.ko and we are working > > > > > through them but already have a very good set running. > > > > > > > > Agree! Thanks! > > > > > > > > Murphy > > > > > > > > > > > > > > On Mon, Sep 9, 2019 at 5:50 PM Aurélien Aptel <aaptel@xxxxxxxx> > > wrote: > > > > > > > > > > > > "Murphy Zhou" <jencce.kernel@xxxxxxxxx> writes: > > > > > > > As $subject. Is this wiki being maintained ? > > > > > > > > > > > > > > Looks like the last update was in January 2019. > > > > > > > > > > > > > > https://wiki.samba.org/index.php/Xfstesting-cifs#Exclusion_files > > > > > > > > > > > > We have a buildbot running xfstests relatively often here > > > > > > > > > > > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/ > > > > > > > > > > > > Each group has slightly different tests run, you can see which one > > gets > > > > > > run by clicking on a build: > > > > > > > > > > > > > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/249 > > > > > > > > > > > > Overall xfstests+cifs is very finicky and frustrating to get > > working > > > > > > reliably. Not to mention long. So good luck :) > > > > > > > > > > > > Cheers, > > > > > > -- > > > > > > Aurélien Aptel / SUSE Labs Samba Team > > > > > > GPG: 1839 CB5F 9F5B FB9B AA97 8C99 03C8 A49B 521B D5D3 > > > > > > SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 > > Nürnberg, DE > > > > > > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG > > München) > > > > > > > > > > > > > > > > > > > > -- > > > > > Thanks, > > > > > > > > > > Steve > >
