> > What's the easiest way to "try it and see", to add tooling and find out > > whether the config files would be reopened during the second parse? > > Because I suspect that we won't actually reopen those files, due to the > > config cache. > > strace -f? Thanks - this might work. > > So couldn't we do something like.... > > > > pass #1: > > if (include) > > if (not hasRemoteUrl) > > open up path & parse > > put config into in-memory cache normally > > pass #2: (and this pass would need to be added to repo_config() probably) > > if (include) > > if (hasRemoteUrl) > > open up path & parse > > insert in-order into in-memory cache > > don't touch existing configs otherwise > > > > I think it's in practice similar to the approach you're using (getting > > around the weird ordering with a cache in memory), but we could reuse > > the existing config cache rather than creating a new and different one. > > I don't know enough to say if this two-step approach is better (although > I'm slightly biased in that direction, since it seems simpler), but this > just seems like premature optimization. > > I.e. let's just read the files twice, they'll be in the OS's FS cache, > which is unlikely to be a bottleneck for the amount of files involved. OK - let me try this. > That being said we do have exactly this cache already. See [1] and > 3c8687a73ee (add `config_set` API for caching config-like files, > 2014-07-28). > > But I think that was added due to *very* frequent re-parsing of the > entire config every time someone needed a config variable, not due to > the I/O overhead (but I may be wrong). > > So if we've got 100 config variables we need and 10 config files then > 10*100 is probably starting to hurt, but if for whatever reason we > needed 2*10 here that's probably no big deal, and in any case would only > happen if this new include mechanism was in play. > > 1. https://lore.kernel.org/git/1404631162-18556-1-git-send-email-tanayabh@xxxxxxxxx/ This might not work for the reasons I described in my reply to Emily [1]. I'll try the read-twice version first. [1] https://lore.kernel.org/git/20211109002255.1110653-1-jonathantanmy@xxxxxxxxxx/
