This is based on ds/maintenance-part-3. After sitting with the background maintenance as it has been cooking, I wanted to come back around and implement the background maintenance for Windows. However, I noticed that there were some things bothering me with background maintenance on my macOS machine. These are detailed in PATCH 3, but the tl;dr is that 'cron' is not recommended by Apple and instead 'launchd' satisfies our needs. This series implements the background scheduling so git maintenance (start|stop) works on those platforms. I've been operating with these schedules for a while now without the problems described in the patches. There is a particularly annoying case about console windows popping up on Windows, but PATCH 4 describes a plan to get around that. Updates in V2 ============= * This is a faster turnaround for a v2 than I would normally like, but Eric inspired extra documentation about how to customize background schedules. * New extensions to git-maintenance.txt include guidelines for inspecting what git maintenance start does and how to customize beyond that. This includes a new PATCH 2 that includes documentation for 'cron' on non-macOS non-Windows systems. * Several improvements, especially in the tests, are included. * While testing manually, I noticed that somehow I had incorrectly had an opening <dict> tag instead of a closing </dict> tag in the hourly format on macOS. I found that the xmllint tool can verify the XML format of a file, which catches the bug. This seems like a good approach since the test is macOS-only. Does anyone have concerns about adding this dependency? Thanks, -Stolee Derrick Stolee (4): maintenance: extract platform-specific scheduling maintenance: include 'cron' details in docs maintenance: use launchctl on macOS maintenance: use Windows scheduled tasks Documentation/git-maintenance.txt | 119 +++++++++ builtin/gc.c | 428 ++++++++++++++++++++++++++++-- t/t7900-maintenance.sh | 83 +++++- t/test-lib.sh | 4 + 4 files changed, 614 insertions(+), 20 deletions(-) base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/776 Range-diff vs v1: 1: d35f1aa162 = 1: d35f1aa162 maintenance: extract platform-specific scheduling -: ---------- > 2: 709a173720 maintenance: include 'cron' details in docs 2: 832fdf1687 ! 3: 0fafd75d10 maintenance: use launchctl on macOS @@ Commit message and 'git maintenance stop' by injecting a script that writes the command-line arguments into GIT_TEST_CRONTAB. + An earlier version of this patch accidentally had an opening + "<dict>" tag when it should have had a closing "</dict>" tag. This + was caught during manual testing with actual 'launchctl' commands, + but we do not want to update developers' tasks when running tests. + It appears that macOS includes the "xmllint" tool which can verify + the XML format, so call it from the macOS-specific tests to ensure + the .plist files are well-formatted. + + Helped-by: Eric Sunshine <sunshine@xxxxxxxxxxxxxx> Signed-off-by: Derrick Stolee <dstolee@xxxxxxxxxxxxx> + ## Documentation/git-maintenance.txt ## +@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and + executing the correct binaries in your schedule. + + ++BACKGROUND MAINTENANCE ON MACOS SYSTEMS ++--------------------------------------- ++ ++While macOS technically supports `cron`, using `crontab -e` requires ++elevated privileges and the executed process do not have a full user ++context. Without a full user context, Git and its credential helpers ++cannot access stored credentials, so some maintenance tasks are not ++functional. ++ ++Instead, `git maintenance start` interacts with the `launchctl` tool, ++which is the recommended way to ++https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS]. ++ ++Scheduling maintenance through `git maintenance (start|stop)` requires ++some `launchctl` features available only in macOS 10.11 or later. ++ ++Your user-specific scheduled tasks are stored as XML-formatted `.plist` ++files in `~/Library/LaunchAgents/`. You can see the currently-registered ++tasks using the following command: ++ ++----------------------------------------------------------------------- ++$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git ++org.git-scm.git.daily.plist ++org.git-scm.git.hourly.plist ++org.git-scm.git.weekly.plist ++----------------------------------------------------------------------- ++ ++One task is registered for each `--schedule=<frequency>` option. To ++inspect how the XML format describes each schedule, open one of these ++`.plist` files in an editor and inspect the `<array>` element following ++the `<key>StartCalendarInterval</key>` element. ++ ++`git maintenance start` will overwrite these files and register the ++tasks again with `launchctl`, so any customizations should be done by ++creating your own `.plist` files with distinct names. Similarly, the ++`git maintenance stop` command will unregister the tasks with `launchctl` ++and delete the `.plist` files. ++ ++To create more advanced customizations to your background tasks, see ++https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation] ++for more information. ++ ++ + GIT + --- + Part of the linkgit:git[1] suite + ## builtin/gc.c ## @@ builtin/gc.c: static int maintenance_unregister(void) return run_command(&config_unset); @@ builtin/gc.c: static int maintenance_unregister(void) + repeat = "<dict>\n" + "<key>Hour</key><integer>%d</integer>\n" + "<key>Minute</key><integer>0</integer>\n" -+ "<dict>\n"; ++ "</dict>\n"; + for (i = 1; i <= 23; i++) + fprintf(plist, repeat, i); + break; @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' ' + for frequency in hourly daily weekly + do + PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" && ++ xmllint "$PLIST" >/dev/null && + grep schedule=$frequency "$PLIST" && + echo "bootout gui/$UID $PLIST" >>expect && + echo "bootstrap gui/$UID $PLIST" >>expect || return 1 3: a9221cc4aa ! 4: 84eb44de31 maintenance: use Windows scheduled tasks @@ Commit message short term. In the long term, we can consider creating this GUI shim application within core Git, perhaps in contrib/. + Helped-by: Eric Sunshine <sunshine@xxxxxxxxxxxxxx> Signed-off-by: Derrick Stolee <dstolee@xxxxxxxxxxxxx> + ## Documentation/git-maintenance.txt ## +@@ Documentation/git-maintenance.txt: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSy + for more information. + + ++BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS ++----------------------------------------- ++ ++Windows does not support `cron` and instead has its own system for ++scheduling background tasks. The `git maintenance start` command uses ++the `schtasks` command to submit tasks to this system. You can inspect ++all background tasks using the Task Scheduler application. The tasks ++added by Git have names of the form `Git Maintenance (<frequency>)`. ++The Task Scheduler GUI has ways to inspect these tasks, but you can also ++export the tasks to XML files and view the details there. ++ ++Note that since Git is a console application, these background tasks ++create a console window visible to the current user. This can be changed ++manually by selecting the "Run whether user is logged in or not" option ++in Task Scheduler. This change requires a password input, which is why ++`git maintenance start` does not select it by default. ++ ++If you want to customize the background tasks, please rename the tasks ++so future calls to `git maintenance (start|stop)` do not overwrite your ++custom tasks. ++ ++ + GIT + --- + Part of the linkgit:git[1] suite + ## builtin/gc.c ## @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd) else @@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop ma ' +test_expect_success MINGW 'start and stop Windows maintenance' ' -+ echo "echo \$@ >>args" >print-args && -+ chmod a+x print-args && ++ write_script print-args <<-\EOF && ++ echo $* >>args ++ EOF + + rm -f args && + GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start && -+ cat args && + + # start registers the repo + git config --get --global maintenance.repo "$(pwd)" && + -+ rm expect && + for frequency in hourly daily weekly + do -+ echo "/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml" >>expect \ -+ || return 1 -+ done && ++ printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \ ++ $frequency $frequency ++ done >expect && + test_cmp expect args && + + rm -f args && -+ GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop && ++ GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop && + + # stop does not unregister the repo + git config --get --global maintenance.repo "$(pwd)" && + + rm expect && -+ for frequency in hourly daily weekly -+ do -+ echo "/delete /tn Git Maintenance ($frequency) /f" >>expect \ -+ || return 1 -+ done && ++ printf "/delete /tn Git Maintenance (%s) /f\n" \ ++ hourly daily weekly >expect && + test_cmp expect args +' + -- gitgitgadget
