Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote: > Hi Eric, > > On Tue, 26 Nov 2019, Eric Wong wrote: > > > Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote: > > > On Tue, 26 Nov 2019, Eric Wong wrote: > > > > Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote: > > > > > The biggest obstacle is that at least one of those Pipelines requires > > > > > access to a clone of public-inbox.org/git, and cloning that is rather > > > > > expensive. Even a shallow fetch would be super expensive, by virtue of > > > > > _all_ the mails being blobs reachable from the tip commit's tree. > > > > > > > > Fwiw, lore.kernel.org/git/$EPOCH.git ought to be somewhat cheaper, > > > > but it's a different (more scalable) format which requires SQLite: > > > > > > > > https://public-inbox.org/public-inbox-v2-format.html > > > > > > Is this incremental? GitGitGadget needs this to be incremental ;-) > > > > Incremental as far as "git fetch" goes? Of course :> > > The "m" file is overwritten with every commit, so the tree size > > stays at 1 (tree growth was a major scalability problem in v1). > > Let me try again: > > GitGitGadget "reads" the mail via the incremental clone, remembering the > hash of the latest processed commit. When the Azure Pipeline runs, it > first fetches, and if the commit is still the same, does nothing but exit > with success. If the commit is different, it looks at the mails that were > added, via `git log -p <previous-tip-commit>..<tip-commit>`. > > Is that possible with the v2 format? Of course, yes. The Xapian and SQLite indexing also works the same way "git log prev..tip" and storing the latest commit hash.
