On 11/19/2021 12:03 PM, Junio C Hamano wrote:
> Torsten Bögershausen <tboegi@xxxxxx> writes:
>
>> Should we conclude that the underlying os/zfs is not stable ?
>> Things don't seem to be reproducable
>>
>> What Git needs here in t0050 is that stat("ä") behaves the same as stat("a¨"),
>> when either "ä" or "a¨" exist on disk.
>> The same for open() and all other file system functions.
>
> We either need to see these two are treated as the same thing, or
> these two are treated as two distict filesystem entities, just like
> stat("a") and stat("b") are. What we absolutely need is the
> unification either always happens or never happens consistently.
>
> I wonder what readdir() is returning. After creat("ä") in an empty
> directory, does readdir() in there return "ä" or "a¨? And vice
> versa? Is this also inconsistent?
Following this suggestion, I added a test helper with this code:
int cmd__create_and_read(int argc, const char **argv)
{
DIR *dir;
struct dirent *de;
if (strcmp(argv[0], "--nfc"))
creat("\303\244", 0766);
else if (strcmp(argv[0], "--nfd"))
creat("\141\314\210", 0766);
else
die("select --nfc or --nfd");
dir = opendir(".");
readdir(dir);
while ((de = readdir(dir)) != NULL)
printf("%s\n", de->d_name);
return 0;
}
And then added this test:
test_expect_success 'unicode stuff' '
mkdir nfc &&
(
cd nfc &&
test-tool create-and-read --nfc >../nfc.txt
) &&
mkdir nfd &&
(
cd nfd &&
test-tool create-and-read --nfd >../nfd.txt
) &&
test_cmp nfc.txt nfd.txt
'
This test always passes for me, and is essentially doing
a similar check that the prereq is doing, except that it
actually writes both names to files instead of writing
one and doing a read with the other.
After changing the "$test_unicode" instances to instances of
"test_expect_success", I ran t0050 under --stress and quickly
got a failure on the 'merge (silent unicode normalization)'
test:
expecting success of 0050.11 'merge (silent unicode normalization)':
git reset --hard initial &&
git merge topic
+ git reset --hard initial
error: unable to unlink old 'ä': No such file or directory
fatal: Could not reset index file to revision 'initial'.
error: last command exited with $?=128
not ok 11 - merge (silent unicode normalization)
Deleting that test gave mostly-consistent results, although I once
got a failure on the "setup unicode normalization tests" tests with
a similar error message:
+ git checkout -f main
error: unable to unlink old 'ä': No such file or directory
Switched to branch 'main'
error: last command exited with $?=1
not ok 8 - setup unicode normalization tests
>> ("ä" is the precomposed form "a¨" is the decomposed form,
>> typically both render to the same glyph on the screen,
>> and a hex dump or xxd will show what we had.
>> I just use this notation here for illustration)
>>
>> Should we contact the zfs developers ?
Hopefully someone has a good way to contact them, and I
can start a thread at the appropriate place. To optimize
for their time, what is our minimal reproduction steps?
1. Build Git at the v2.34.0 tag.
2. cd to t/
3. ./t0050-filesystem.sh --stress
Those instructions (given enough time) should get the
repro on test 8, 'setup unicode normalization tests'.
To get the faster stress, the same steps work except
use the 'zfs-minimal' branch at my fork [1] because it
changes the tests to expect success, and demonstrates
the unpredictable tests more quickly.
[1] https://github.com/derrickstolee/git/tree/zfs-minimal
Thanks,
-Stolee
