Hi Eric, On Tue, 26 Nov 2019, Eric Wong wrote: > Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote: > > On Tue, 26 Nov 2019, Eric Wong wrote: > > > Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote: > > > > The biggest obstacle is that at least one of those Pipelines requires > > > > access to a clone of public-inbox.org/git, and cloning that is rather > > > > expensive. Even a shallow fetch would be super expensive, by virtue of > > > > _all_ the mails being blobs reachable from the tip commit's tree. > > > > > > Fwiw, lore.kernel.org/git/$EPOCH.git ought to be somewhat cheaper, > > > but it's a different (more scalable) format which requires SQLite: > > > > > > https://public-inbox.org/public-inbox-v2-format.html > > > > Is this incremental? GitGitGadget needs this to be incremental ;-) > > Incremental as far as "git fetch" goes? Of course :> > The "m" file is overwritten with every commit, so the tree size > stays at 1 (tree growth was a major scalability problem in v1). Let me try again: GitGitGadget "reads" the mail via the incremental clone, remembering the hash of the latest processed commit. When the Azure Pipeline runs, it first fetches, and if the commit is still the same, does nothing but exit with success. If the commit is different, it looks at the mails that were added, via `git log -p <previous-tip-commit>..<tip-commit>`. Is that possible with the v2 format? Ciao, Dscho
