On Fri, Aug 16, 2019 at 1:15 AM Tom Talpey <ttalpey@xxxxxxxxxxxxx> wrote:
>
> > -----Original Message-----
> > From: linux-cifs-owner@xxxxxxxxxxxxxxx <linux-cifs-owner@xxxxxxxxxxxxxxx> On
> > Behalf Of ronnie sahlberg
> > Sent: Wednesday, August 14, 2019 11:31 PM
> > To: linux-cifs <linux-cifs@xxxxxxxxxxxxxxx>
> > Subject: FSCTL_QUERY_ALLOCATED_RANGES issue with Windows2016
> >
> > I am seeing issues with how FSCTL_QUERY_ALLOCATED_RANGES behaves
> > under windows,
> > in particular that it is inconsistent so we can't run certain xfstests
> > against windows servers.
> >
> >
> > The behavior can be triggered using the following command from xfstests :
> > ./src/seek_sanity_test -s 19 -e 19 /mnt/file
> > (where /mnt is an smb share mounted from a windows 2016 server.)
> >
> > In cifs.ko we use this FSCTL to implement both SEEK_HOLE/SEEK_DATA as well
> > as
> > fiemap(). But since the behavior of this FSCTL is not deteministic it often
> > cause the tests to fail.
> >
> >
> > This is a test tool for SEEK_DATA and SEEK_HOLE. As part of this it performs
> > a check to try to discover if the filesystem supports sparse files or not.
> > For non-sparse files SEEK_DATA is basically a no-op and SEEK_HOLE just returns
> > the file size.
> > Later during the test, the result of whether the file is "sparse" or not
> > will affect what the expected results are. If this check gets the
> > sparse-supported wrong the test cases later will fail.
> >
> >
> > How the check works:
> > ====================
> > The actual check for whether the file supports being sparse or not is the
> > following snippet :
> > ftruncate(fd, 0);
> > bufsz = alloc_size * 2;
> > filsz = bufsz * 2;
> >
> > buf = do_malloc(bufsz);
> > if (!buf)
> > goto out;
> > memset(buf, 'a', bufsz);
> >
> > /* File with 2 allocated blocks.... */
> > ret = do_pwrite(fd, buf, bufsz, 0);
> > if (ret)
> > goto out;
> >
> > /* followed by a hole... */
> > ret = do_truncate(fd, filsz);
> > if (ret)
> > goto out;
> >
> > /* Is SEEK_DATA and SEEK_HOLE supported in the kernel? */
> > pos = lseek(fd, 0, SEEK_HOLE);
> > if (pos == -1) {
> > fprintf(stderr, "Kernel does not support llseek(2) extension "
> > "SEEK_HOLE. Aborting.\n");
> > ret = -1;
> > goto out;
> > }
> >
> > if (pos == filsz) {
> > default_behavior = 1;
> > fprintf(stderr, "File system supports the default behavior.\n")\
> > ;
> > }
> >
> > I.e.
> > 1, ftruncate to 0
> > 2, write 2M to the start of the file
> > 3, ftruncate the file to be 4Mb
> > 4, SEEK_HOLE and check if it returns 4Mb or no.
> > If it returns 4Mb then we switch back to default_behavior and we allow
> > the semantics as if the file is not sparse.
> > I.e. allow SEEK_DATA to behave as if the file is either sparse or not-sparse.
> >
> > Also note that if it looks like the sparse-file is not supported then
> > it prints ""File system supports the default behavior." which may help when
> > running the test tool.
> >
> > Strace for this check (when the check failed.)
> > =============================================
> > 18:22:14.949612 ftruncate(3, 0) = 0 <0.011513>
> > 18:22:14.963725 mmap(NULL, 2101248, PROT_READ|PROT_WRITE,
> > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe8d9f14000 <0.0
> > 00192>
> > 18:22:14.970334 pwrite64(3,
> > "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> > aaaaaaaaaaaaaaaaaa
> > aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> > aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> > aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"...,
> > 2097152, 0) = 2097152 <0.002127>
> > 18:22:14.972620 ftruncate(3, 4194304) = 0 <0.582491>
> > 18:22:15.557308 lseek(3, 0, SEEK_HOLE) = 4194304 <0.012791>
> > 18:22:15.572457 write(2, "File system supports the default
> > behavior.\n", 43) = 43 <0.000318>
> >
> >
> > Example of when the check goes "Wrong". Capture seek_good.cap
> > =============================================================
> > The relevant packets here are
> > 1820, set end of file == 0
> > (1822/1823 update timestamps)
> > 2930, write 2Mb to offset 0
> > 3008, set the file sparse
> > 3010, set end of file to 4M
> > 3019, query-allocated-ranges
> >
> > In 3019/3020 the server responds that the full 0-4M range is allocated.
> > This is wrong since file should only be allocated in the first 2Mb at
> > this stage.
> > And this makes the test tool think that we don't support sparse files,
> > and sets default_behavior to 1.
> >
> > A different run when the check is successful (seek_bad.cap)
> > ============================================================
> > In the seek_bad capture you can see the same sequence in packets
> > 1869, set end of file == 0
> > 2990, write the first 2M of the file
> > 3067, set file sparse
> > 3069, set end of file to 4M
> > 3078, query-allocated-ranges
> > But this time, query-allocated-ranges report that only 0-2M is mapped,
> > which is the correct range,
> > and thus the test tool assumes that we can handle holes properly.
> >
> > The captures are ~5MByte each unfiltered so too big for the list.
> > Email me directly and I will send them to you.
> >
> >
> > So the question here is what is the actual semantics for sparse files
> > and query-allocated-ranges on windows?
>
> I'll try to get you an answer, but in the meantime just a question...
> Does the behavior change if the file is opened with FILE_FLAG_WRITE_THROUGH?
I will try that shortly.
There is a SMB2_FLUSH call on the handle immediately before the call
to QUERY-ALLOCATED_RANGES.
Would that have the same effect as the writethrough flag?
I am happy to email the wireshark traces to you to look at exactly
what goes on on the wire.
I have two example traces, taken a minute apart.
One instance QAR reports a range of 0-2M, the other the reported range is 0-4M
>
> Tom.
