> -----Original Message----- > From: Eric DeCosta > Sent: Tuesday, September 6, 2022 3:34 PM > To: 'Jeff Hostetler' <git@xxxxxxxxxxxxxxxxx>; 'Eric DeCosta via GitGitGadget' > <gitgitgadget@xxxxxxxxx>; 'git@xxxxxxxxxxxxxxx' <git@xxxxxxxxxxxxxxx> > Subject: RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the > real path > > > > > -----Original Message----- > > From: Eric DeCosta > > Sent: Tuesday, September 6, 2022 3:02 PM > > To: Jeff Hostetler <git@xxxxxxxxxxxxxxxxx>; Eric DeCosta via > > GitGitGadget <gitgitgadget@xxxxxxxxx>; git@xxxxxxxxxxxxxxx > > Subject: RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths > > to the real path > > > > > > > > > -----Original Message----- > > > From: Jeff Hostetler <git@xxxxxxxxxxxxxxxxx> > > > Sent: Tuesday, September 6, 2022 1:14 PM > > > To: Eric DeCosta <edecosta@xxxxxxxxxxxxx>; Eric DeCosta via > > > GitGitGadget <gitgitgadget@xxxxxxxxx>; git@xxxxxxxxxxxxxxx > > > Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event > > > paths to the real path > > > > > > > > > > > > On 9/2/22 12:35 PM, Eric DeCosta wrote: > > > > > > > > > > > >> -----Original Message----- > > > >> From: Jeff Hostetler <git@xxxxxxxxxxxxxxxxx> > > > >> Sent: Thursday, September 1, 2022 4:06 PM > > > >> To: Eric DeCosta via GitGitGadget <gitgitgadget@xxxxxxxxx>; > > > >> git@xxxxxxxxxxxxxxx > > > >> Cc: Eric DeCosta <edecosta@xxxxxxxxxxxxx> > > > >> Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event > > > >> paths to the real path > > > >> > > > >> > > > >> > > > >> On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote: > > > >>> From: Eric DeCosta <edecosta@xxxxxxxxxxxxx> > > > >>> > > > >>> Consider the following network working directory that is mounted > > > >>> under > > > >>> /System/Volumes/Data: > > > >>> > > > >>> /network/working/directory > > > >>> > > > >>> The git working directory path is: > > > >>> > > > >>> /System/Volumes/Data/network/working/directory > > > >>> > > > >>> The paths reported by FSEvents always start with /network. > > > >>> fsmonitor expects paths to be under the working directory; > > > >>> therefore it fails to match /network/... and ignores the change. > > > >> > > > >> I'm not sure I understand what's going on here. > > > >> Are you saying that FSEvents reports network mounted directories > > > >> with a path relative to the mount-point rather than an absolute path? > > > >> > > > > > > > > Yes > > > > > > > >> In your example, is "/network/working/directory" a valid path on > > > >> your system (that also happens to be the same directory as > > > >> "/System/Volumes/...") > > > >> > > > >> That is, do you have some aliasing going on where both paths are > > > >> valid -- like a pair of hard-linked directories? > > > >> Or, is there something special about a network mount point? > > > >> > > > >> > > > >> Did you have to "cd /System/Volumes/..." to get Git to have the > > > >> absolute path be this? Or were you doing a "cd /network/..."? > > > >> (Sorry to ask so many questions but I don't have a pair of > > > >> systems to test any of this on right now.) > > > >> > > > > > > > > "/network/working/directory" is mounted under > > > > "/System/Volumes/Data/network/working/directory" > > > > > > > > There is also a symlink: > > > > > > > > "/network" -> "/System/Volumes/Data/network" > > > > > > > > No matter if I "cd /System/Volumes/Data/network/working/directory" > > > > or "cd /network/working/directory" the paths reported by FSEvents > > > > always start with "/network/working/directory" > > > > > > > > If I do a similar symlink with local directories, I do not get > > > > this unexpected behavior. > > > > > > > > I need to remove the symlink and try it that way, but I need to > > > > coordinate with the machine's owner. > > > > > > > > > I think you have stumbled upon a recent MacOS feature called > "firmlinks". > > > I'm just reading up on it myself, so I shouldn't speculate here > > > (yet), but maybe you too could do some reading on the subject. > > > > > > This makes me wonder if the symlink is historical. The real magic is > > > in the firmlinks. For example, if I do: > > > > > > $ (cd / ; ls -l | grep Users) > > > drwxr-xr-x 6 root admin 192 Apr 6 2020 Users > > > > > > $ (cd /Users ; df .) > > > Filesystem 512-blocks Used Available Capacity iused ifree %iused > > > Mounted on > > > /dev/disk1s1 976490576 608954344 338850488 65% 4181323 4878271557 > 0% > > > /System/Volumes/Data > > > > > > we can see that /Users is on /System/Volumes/Data (and there is a > > > /System/Volumes/Data/Users directory with the same metadata), but it > > > is not a symlink. > > > > > > > > > See [1] for more info. > > > > > > [1] > > > http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink- > > > magic.html <https://protect- > > > > > > us.mimecast.com/s/QN4qCYEZB2h1p7jWcG1Cbw?domain=swiftforensics.co > > > m> > > > > > > > > > > Hmm, I don't see anything that suggests a firmlink is involved but > > this is new to me so maybe I just don't see it. There is an automount > > in addition to the symlink though. > > > > So, dispensing with the abstract "/network" path, and getting to the > > details of my environment: > > > > % mount | grep /mathworks > > map auto.sfs.sol2.mathworks on /System/Volumes/Data/mathworks > (autofs, > > automounted, nobrowse) triggered map auto.sfs.sol2.devel on > > /System/Volumes/Data/mathworks/devel (autofs, automounted, > > nobrowse) triggered map sfs.worldwide.US-Natick.sol2.devel.sbs on > > /System/Volumes/Data/mathworks/devel/sbs (autofs, automounted, > > nobrowse) triggered map sfs.worldwide.US-Natick.devel.sbs.29 on > > /System/Volumes/Data/mathworks/devel/sbs/29 (autofs, automounted, > > nobrowse) > > batfs-sb29-nfs:/vmgr/sbs29/edecosta.Bbashprod1.j1928560.2 on > > > /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19 > > 28560.2 (nfs, nodev, automounted, noatime, nobrowse) ... > > > > % ls -l / | grep mathworks > > lrwxr-xr-x 1 root wheel 35 Aug 29 10:25 home@ -> > > /System/Volumes/Data/mathworks/home > > lrwxr-xr-x 1 root wheel 30 Aug 29 10:25 mathworks@ -> > > /System/Volumes/Data/mathworks > > > > My worktree is: > > > /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19 > > 28560.2 > > or, equivalently via the symlink just: > > /mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2 > > > > I have sudo'ers access now; so I can try mounting things manually, > > mess about with symlinks and see what I get. > > > > > [...] > > > >>> @@ -209,7 +209,12 @@ static void > > > >> fsevent_callback(ConstFSEventStreamRef streamRef, > > > >>> /* > > > >>> * On Mac, we receive an array of absolute paths. > > > >>> */ > > > >>> - path_k = paths[k]; > > > >>> + if (fsm_settings__get_allow_remote(the_repository) > 0) { > > > >>> + strbuf_reset(&tmp); strbuf_realpath_forgiving(&tmp, paths[k], > > > >>> + 0); path_k = tmp.buf; } else path_k = paths[k]; > > > >> > > > >> This feels wrong. > > > >> > > > >> It is not that fsmonitor.allowRemote is true, but whether or not > > > >> this particular file system *IS* remote, right? > > > > > > > > True. I suppose each path could be checked, or some bit could be > > > > set if the working directory is remote, perhaps along the lines of > > > > fsmonitor_ipc__get_path. Then the determination of the IPC path > > > > and whether it is remote would be in one place. fsm-settings-* > > > > would then just get that bit and check it against allowRemote. > > > > > > > > Thoughts? > > > > > > I'll do some digging here. There ought to be a way to tell if a > > > component directory in a pathname has a firmlink peer. (But I'm > > > traveling for GitMerge, so I might not have time to look at this > > > until > > > afterwards.) > > > > > > This would indicate that a "bi-directional wormhole" (their words) > > > alias is available (and hopefully, a way to computer the other > > > peer....) > > > > > > I'm thinking that the "/network/..." path in the FSEvents is because > > > FSEventD is using a particular peer spelling (that we weren't > > > expecting). > > > > > > If we can compute the spellings of the peers once when the daemon > > > starts up (and maybe make a little dictionary), then we can do > > > prefix tricks on the absolute path after the path_k = paths[k]; step > > > in > > > fsevent_callback() to extract a relative path rather than an > > > absolute path. > > > > > > Then call fsmonitor_classify_path_relative() instead of _absolute(). > > > > > > Watch out though, there are several places that do rel = path+k + > > > ...state->path_worktree_watch.len... > > > that would need to be adjusted. > > > > > > Hope this helps, > > > Jeff > > > > > > > > Thanks for the insights, I'll dig around and test things out some more. > > > > -Eric > > This is informative: > > https://developer.apple.com/forums/thread/120665 > > -Eric > Conversation with one of our macOS admins: Q: Is /mathworks a symbolic link or some sort of firmlink or synthetic firmlink? A: It’s a “synthetic” link, per apple. Config is managed in /etc/synthetic.conf The auto mounter will also automatically create it based on the contents of /etc/auto_master even without the synthetic.conf file. We went back and forth with apple for a while on the “right” way to do it and now I’m not sure what we settled on, off the top of my head I don’t know if they’re the same under the hood, but it seemed to be able to create root level links even though you can’t with ln anymore I suspect apple wants users to use the synthetic.conf file in all cases... ---- It appears that the automounter is creating "synthetic firmlinks", a la https://derflounder.wordpress.com/2020/01/18/creating-root-level-directories-and-symbolic-links-on-macos-catalina/ When I look in /etc/auto_master I see several entries that appear as symlinks in the output of "ls -l /" but in reality they are synthetic firmlinks. -Eric
