> -----Original Message-----
> From: linux-cifs-owner@xxxxxxxxxxxxxxx <linux-cifs-owner@xxxxxxxxxxxxxxx> On
> Behalf Of ronnie sahlberg
> Sent: Wednesday, August 14, 2019 11:31 PM
> To: linux-cifs <linux-cifs@xxxxxxxxxxxxxxx>
> Subject: FSCTL_QUERY_ALLOCATED_RANGES issue with Windows2016
>
> I am seeing issues with how FSCTL_QUERY_ALLOCATED_RANGES behaves
> under windows,
> in particular that it is inconsistent so we can't run certain xfstests
> against windows servers.
>
>
> The behavior can be triggered using the following command from xfstests :
> ./src/seek_sanity_test -s 19 -e 19 /mnt/file
> (where /mnt is an smb share mounted from a windows 2016 server.)
>
> In cifs.ko we use this FSCTL to implement both SEEK_HOLE/SEEK_DATA as well
> as
> fiemap(). But since the behavior of this FSCTL is not deteministic it often
> cause the tests to fail.
>
>
> This is a test tool for SEEK_DATA and SEEK_HOLE. As part of this it performs
> a check to try to discover if the filesystem supports sparse files or not.
> For non-sparse files SEEK_DATA is basically a no-op and SEEK_HOLE just returns
> the file size.
> Later during the test, the result of whether the file is "sparse" or not
> will affect what the expected results are. If this check gets the
> sparse-supported wrong the test cases later will fail.
>
>
> How the check works:
> ====================
> The actual check for whether the file supports being sparse or not is the
> following snippet :
> ftruncate(fd, 0);
> bufsz = alloc_size * 2;
> filsz = bufsz * 2;
>
> buf = do_malloc(bufsz);
> if (!buf)
> goto out;
> memset(buf, 'a', bufsz);
>
> /* File with 2 allocated blocks.... */
> ret = do_pwrite(fd, buf, bufsz, 0);
> if (ret)
> goto out;
>
> /* followed by a hole... */
> ret = do_truncate(fd, filsz);
> if (ret)
> goto out;
>
> /* Is SEEK_DATA and SEEK_HOLE supported in the kernel? */
> pos = lseek(fd, 0, SEEK_HOLE);
> if (pos == -1) {
> fprintf(stderr, "Kernel does not support llseek(2) extension "
> "SEEK_HOLE. Aborting.\n");
> ret = -1;
> goto out;
> }
>
> if (pos == filsz) {
> default_behavior = 1;
> fprintf(stderr, "File system supports the default behavior.\n")\
> ;
> }
>
> I.e.
> 1, ftruncate to 0
> 2, write 2M to the start of the file
> 3, ftruncate the file to be 4Mb
> 4, SEEK_HOLE and check if it returns 4Mb or no.
> If it returns 4Mb then we switch back to default_behavior and we allow
> the semantics as if the file is not sparse.
> I.e. allow SEEK_DATA to behave as if the file is either sparse or not-sparse.
>
> Also note that if it looks like the sparse-file is not supported then
> it prints ""File system supports the default behavior." which may help when
> running the test tool.
>
> Strace for this check (when the check failed.)
> =============================================
> 18:22:14.949612 ftruncate(3, 0) = 0 <0.011513>
> 18:22:14.963725 mmap(NULL, 2101248, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe8d9f14000 <0.0
> 00192>
> 18:22:14.970334 pwrite64(3,
> "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> aaaaaaaaaaaaaaaaaa
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"...,
> 2097152, 0) = 2097152 <0.002127>
> 18:22:14.972620 ftruncate(3, 4194304) = 0 <0.582491>
> 18:22:15.557308 lseek(3, 0, SEEK_HOLE) = 4194304 <0.012791>
> 18:22:15.572457 write(2, "File system supports the default
> behavior.\n", 43) = 43 <0.000318>
>
>
> Example of when the check goes "Wrong". Capture seek_good.cap
> =============================================================
> The relevant packets here are
> 1820, set end of file == 0
> (1822/1823 update timestamps)
> 2930, write 2Mb to offset 0
> 3008, set the file sparse
> 3010, set end of file to 4M
> 3019, query-allocated-ranges
>
> In 3019/3020 the server responds that the full 0-4M range is allocated.
> This is wrong since file should only be allocated in the first 2Mb at
> this stage.
> And this makes the test tool think that we don't support sparse files,
> and sets default_behavior to 1.
>
> A different run when the check is successful (seek_bad.cap)
> ============================================================
> In the seek_bad capture you can see the same sequence in packets
> 1869, set end of file == 0
> 2990, write the first 2M of the file
> 3067, set file sparse
> 3069, set end of file to 4M
> 3078, query-allocated-ranges
> But this time, query-allocated-ranges report that only 0-2M is mapped,
> which is the correct range,
> and thus the test tool assumes that we can handle holes properly.
>
> The captures are ~5MByte each unfiltered so too big for the list.
> Email me directly and I will send them to you.
>
>
> So the question here is what is the actual semantics for sparse files
> and query-allocated-ranges on windows?
I'll try to get you an answer, but in the meantime just a question...
Does the behavior change if the file is opened with FILE_FLAG_WRITE_THROUGH?
Tom.
