On Tue, Aug 24, 2021 at 10:38 PM Christoph Hellwig <hch@xxxxxx> wrote: > > On Wed, Aug 25, 2021 at 01:51:32AM +0000, Neeraj Singh via GitGitGadget wrote: > > From: Neeraj Singh <neerajsi@xxxxxxxxxxxxx> > > > > When adding many objects to a repo with core.fsyncObjectFiles set to > > true, the cost of fsync'ing each object file can become prohibitive. > > > > One major source of the cost of fsync is the implied flush of the > > hardware writeback cache within the disk drive. Fortunately, Windows, > > MacOS, and Linux each offer mechanisms to write data from the filesystem > > page cache without initiating a hardware flush. > > > > This patch introduces a new 'core.fsyncObjectFiles = 2' option that > > takes advantage of the bulk-checkin infrastructure to batch up hardware > > flushes. > > Another interesting way to flush on linux would be the syncfs call, > which syncs all files on a file system. Once you write more than > handful or two of files that tends to win out over a batch of fsync > calls. I'd expect syncfs to suffer from the noisy-neighbor problem that Linus alluded to on the big thread you kicked off. The equivalent call on Windows currently requires administrative privileges (I'm really not sure exactly why, perhaps we should change that). If someone adds a more targeted bulk sync interface to the Linux kernel, I'm sure Git could be changed to use it. Maybe an fcntl(2) interface that initiates writeback and registers completion with an eventfd.
