Here is part 3 of my builtin FSMonitor series.
Part 3 builds upon part 2 (jh/builtin-fsmonitor-part2) which is currently in
"seen", so this series should be considered a preview until part 2 moves to
"next". (And part 2 should not graduate to "master" without this part.)
Part 2 established the client code (used by commands like git status) and an
MVP implementation of the FSMonitor daemon. This was sufficient to test the
concepts and basic functionality.
Part 3 finishes the daemon and adds additional tests. This includes:
* On Windows, handle short- and long-name aliasing.
* On Mac, handle Unicode aliasing.
* Mark bare, virtual, and remote working directories incompatible with
FSMonitor.
* On Mac, ignore xattr change FSEvents.
* On Windows, cd out of the working directory root.
* Create a thread to monitor the health and shutdown the daemon if
necessary.
* Speed up handling of directory notification events.
* Test directory move and rename events.
* Add performance test.
Here is performance data from t/perf/p7527-builtin-fsmonitor.sh on a
synthetic repo containing 1M files on a Macbook Pro. It shows the effects of
the untracked cache (uc) and FSMonitor (fsm) on git status.
$ ./p7527-builtin-fsmonitor.sh
# passed all 67 test(s)
1..67
Test this tree
---------------------------------------------------------------------------------------
7527.4: [uc false][fsm false] status after checkout 29.99(3.14+80.12)
7527.6: [uc false][fsm false] status after big change 73.32(5.11+97.24)
7527.8: [uc false][fsm false] status after add all 47.80(5.12+90.47)
7527.10: [uc false][fsm false] status after add dot 49.22(5.16+92.05)
7527.12: [uc false][fsm false] status after commit 51.53(3.35+100.74)
7527.14: [uc false][fsm false] status after reset hard 33.74(3.03+85.31)
7527.16: [uc false][fsm false] status after create untracked files 41.71(3.24+89.75)
7527.18: [uc false][fsm false] status after clean 34.33(3.07+89.36)
7527.20: [uc false][fsm true] status after checkout 29.23(1.94+10.84)
7527.22: [uc false][fsm true] status after big change 64.23(4.66+24.86)
7527.24: [uc false][fsm true] status after add all 45.45(4.37+18.70)
7527.26: [uc false][fsm true] status after add dot 44.42(4.02+17.10)
7527.28: [uc false][fsm true] status after commit 30.52(1.95+10.91)
7527.30: [uc false][fsm true] status after reset hard 28.70(2.70+13.89)
7527.32: [uc false][fsm true] status after create untracked files 28.63(2.59+10.71)
7527.34: [uc false][fsm true] status after clean 28.97(2.59+10.78)
7527.36: [uc true][fsm false] status after checkout 35.06(3.17+86.11)
7527.38: [uc true][fsm false] status after big change 74.65(5.14+101.50)
7527.40: [uc true][fsm false] status after add all 49.96(5.22+90.96)
7527.42: [uc true][fsm false] status after add dot 49.77(5.24+91.72)
7527.44: [uc true][fsm false] status after commit 36.95(3.27+92.25)
7527.46: [uc true][fsm false] status after reset hard 33.89(3.18+85.68)
7527.48: [uc true][fsm false] status after create untracked files 41.44(3.40+92.99)
7527.50: [uc true][fsm false] status after clean 34.60(3.26+90.19)
7527.52: [uc true][fsm true] status after checkout 0.58(0.45+0.10)
7527.54: [uc true][fsm true] status after big change 65.16(4.91+25.64)
7527.56: [uc true][fsm true] status after add all 45.43(4.45+18.92)
7527.58: [uc true][fsm true] status after add dot 15.56(2.57+6.32)
7527.60: [uc true][fsm true] status after commit 0.98(0.46+0.11)
7527.62: [uc true][fsm true] status after reset hard 30.30(2.96+14.49)
7527.64: [uc true][fsm true] status after create untracked files 2.15(1.73+0.40)
7527.66: [uc true][fsm true] status after clean 1.68(1.56+0.32)
Here is performance data from t/perf/p7519-fsmonitor.sh on the same
synthetic repo containing 1M files on a Macbook Pro. This test performs the
same series of commands on all three FSMonitor options: (1) using the hook
interface to talk to Watchman, (2) no FSMonitor, and (3) the builtin
FSMonitor daemon.
$ ./p7519-fsmonitor.sh
# passed all 42 test(s)
1..42
Test this tree
-------------------------------------------------------------------------------
7519.4: status (fsmonitor=fsmonitor-watchman) 0.89(0.63+0.17)
7519.5: status -uno (fsmonitor=fsmonitor-watchman) 0.56(0.43+0.13)
7519.6: status -uall (fsmonitor=fsmonitor-watchman) 24.52(1.82+9.90)
7519.7: status (dirty) (fsmonitor=fsmonitor-watchman) 32.40(6.54+9.94)
7519.8: diff (fsmonitor=fsmonitor-watchman) 0.66(0.38+0.42)
7519.9: diff HEAD (fsmonitor=fsmonitor-watchman) 1.60(0.69+0.80)
7519.10: diff -- 0_files (fsmonitor=fsmonitor-watchman) 0.37(0.29+0.08)
7519.11: diff -- 10_files (fsmonitor=fsmonitor-watchman) 0.42(0.32+0.09)
7519.12: diff -- 100_files (fsmonitor=fsmonitor-watchman) 0.42(0.32+0.10)
7519.13: diff -- 1000_files (fsmonitor=fsmonitor-watchman) 0.41(0.31+0.09)
7519.14: diff -- 10000_files (fsmonitor=fsmonitor-watchman) 0.44(0.32+0.12)
7519.15: add (fsmonitor=fsmonitor-watchman) 24.81(2.44+10.06)
7519.18: status (fsmonitor=disabled) 11.18(1.80+74.68)
7519.19: status -uno (fsmonitor=disabled) 7.11(1.50+73.95)
7519.20: status -uall (fsmonitor=disabled) 30.09(3.01+91.11)
7519.21: status (dirty) (fsmonitor=disabled) 31.46(5.03+77.75)
7519.22: diff (fsmonitor=disabled) 5.79(1.27+62.97)
7519.23: diff HEAD (fsmonitor=disabled) 7.29(1.57+76.03)
7519.24: diff -- 0_files (fsmonitor=disabled) 0.68(0.30+0.09)
7519.25: diff -- 10_files (fsmonitor=disabled) 0.32(0.29+0.07)
7519.26: diff -- 100_files (fsmonitor=disabled) 0.32(0.29+0.07)
7519.27: diff -- 1000_files (fsmonitor=disabled) 0.35(0.29+0.09)
7519.28: diff -- 10000_files (fsmonitor=disabled) 0.56(0.29+0.21)
7519.29: add (fsmonitor=disabled) 29.49(3.52+85.88)
7519.31: status (builtin fsmonitor--daemon) 0.67(0.54+0.09)
7519.32: status -uno (builtin fsmonitor--daemon) 0.42(0.35+0.08)
7519.33: status -uall (builtin fsmonitor--daemon) 22.77(1.71+10.35)
7519.34: status (dirty) (builtin fsmonitor--daemon) 21.79(4.18+9.97)
7519.35: diff (builtin fsmonitor--daemon) 0.52(0.34+0.36)
7519.36: diff HEAD (builtin fsmonitor--daemon) 1.55(0.66+0.80)
7519.37: diff -- 0_files (builtin fsmonitor--daemon) 0.37(0.31+0.08)
7519.38: diff -- 10_files (builtin fsmonitor--daemon) 0.35(0.29+0.07)
7519.39: diff -- 100_files (builtin fsmonitor--daemon) 0.35(0.29+0.07)
7519.40: diff -- 1000_files (builtin fsmonitor--daemon) 0.35(0.29+0.07)
7519.41: diff -- 10000_files (builtin fsmonitor--daemon) 0.37(0.29+0.09)
7519.42: add (builtin fsmonitor--daemon) 24.64(2.40+10.04)
Jeff Hostetler (23):
fsm-listen-win32: handle shortnames
t7527: test FS event reporing on macOS WRT case and Unicode
t7527: test builtin FSMonitor watching repos with unicode paths
t/helper/fsmonitor-client: create stress test
fsmonitor-settings: bare repos are incompatible with FSMonitor
fsmonitor-settings: stub in platform-specific incompatibility checking
fsmonitor-settings: virtual repos are incompatible with FSMonitor
fsmonitor-settings: stub in macOS-specific incompatibility checking
fsmonitor-settings: remote repos on macOS are incompatible with
FSMonitor
fsmonitor-settings: remote repos on Windows are incompatible with
FSMonitor
unpack-trees: initialize fsmonitor_has_run_once in o->result
fsm-listen-darwin: ignore FSEvents caused by xattr changes on macOS
fsmonitor--daemon: print start message only if
fsmonitor.announceStartup
fsmonitor--daemon: cd out of worktree root
fsmonitor--daemon: prepare for adding health thread
fsmonitor--daemon: rename listener thread related variables
fsmonitor--daemon: stub in health thread
fsm-health-win32: add framework to monitor daemon health
fsm-health-win32: force shutdown daemon if worktree root moves
fsm-listen-darwin: shutdown daemon if worktree root is moved/renamed
fsmonitor: optimize processing of directory events
t7527: FSMonitor tests for directory moves
t/perf/p7527: add perf test for builtin FSMonitor
Makefile | 19 +-
builtin/fsmonitor--daemon.c | 138 ++++++++-
builtin/update-index.c | 9 +
compat/fsmonitor/fsm-health-darwin.c | 24 ++
compat/fsmonitor/fsm-health-win32.c | 257 ++++++++++++++++
compat/fsmonitor/fsm-health.h | 47 +++
compat/fsmonitor/fsm-listen-darwin.c | 89 +++++-
compat/fsmonitor/fsm-listen-win32.c | 409 ++++++++++++++++++++-----
compat/fsmonitor/fsm-listen.h | 2 +-
compat/fsmonitor/fsm-settings-darwin.c | 75 +++++
compat/fsmonitor/fsm-settings-win32.c | 137 +++++++++
config.mak.uname | 5 +
contrib/buildsystems/CMakeLists.txt | 8 +
fsmonitor--daemon.h | 11 +-
fsmonitor-settings.c | 84 +++++
fsmonitor-settings.h | 26 ++
fsmonitor.c | 71 ++++-
t/helper/test-fsmonitor-client.c | 105 +++++++
t/perf/p7527-builtin-fsmonitor.sh | 257 ++++++++++++++++
t/t7519-status-fsmonitor.sh | 35 +++
t/t7527-builtin-fsmonitor.sh | 157 ++++++++++
unpack-trees.c | 1 +
22 files changed, 1838 insertions(+), 128 deletions(-)
create mode 100644 compat/fsmonitor/fsm-health-darwin.c
create mode 100644 compat/fsmonitor/fsm-health-win32.c
create mode 100644 compat/fsmonitor/fsm-health.h
create mode 100644 compat/fsmonitor/fsm-settings-darwin.c
create mode 100644 compat/fsmonitor/fsm-settings-win32.c
create mode 100755 t/perf/p7527-builtin-fsmonitor.sh
base-commit: ce8aa175364c7a02115c4b4f91d5674f4d6d8ae8
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1143%2Fjeffhostetler%2Fbuiltin-fsmonitor-part3-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1143/jeffhostetler/builtin-fsmonitor-part3-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1143
--
gitgitgadget
