Re: Git monorepo - recommendation regarding usage of sparse-checkout

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



"Mor, Gil (DXC Luxoft)" <gil.mor@xxxxxxx> writes:
> Hello, we are experimenting with migrating a large-ish code base from
> SVN to a Git Monorepo and it would help us if we can get some input
> regarding the usage of sparse-checkout.

We're in the same boat. I haven't been able to keep up with the list as
well as I would like, but I can share our experience so far. We're
writing developer tooling for a team of ~2k devs.

> From our timing experiments sparse-checkout is the only method so far
> that reduces our times to good results.

You should also look into sparse-index.

> The only issue might be the Disclaimer that the sparse-checkout
> feature is experimental, and that the behavior will change.

It seems vanishingly unlikely that the feature will go away at this
point (even if the CLI changes). We have automated integration tests set
up for our automation and have near-term plans to start running those
against `git.git:main` and `git.git:next`. This way, we'll get advance
notice if something we're relying on starts breaking.

> The command is already featured in GitHub and GitLab articles about
> reducing Monorepos size but we are still not sure how un/stable the
> feature is or how commonly used the feature is already.

We haven't encountered many issues with stability. There was one issue
a few months back where the pattern syntax changed, but as I recall that
was more of a problem with one of our developers going off the beaten
path and trying to write to GIT_DIR directly instead of using `git
sparse-checkout set` or similar.

> So, we thought we'll write an email to see if we can get a bit more
> nuanced answer about the safety of real-world usage so that we can
> make an informed decision whether or not to start using
> sparse-checkout, despite it being experimental.

One of the goals of our tooling is to teach people how to actually use
Git (i.e., use our tooling to automate the boring stuff -- not to
replace Git itself). To meet this goal, we're using the more 'ergonomic'
git-switch command instead of git-checkout. In our case, as long as we
can react to changes in git-switch syntax (which we haven't seen since
the project started a few years ago) and as long as we can get the same
side-effects, we'll be fine. This comfort is largely driven by the
existence of integration tests.

> We are not looking for 100% assurance, we know the responsibility is
> eventually totally ours and there are no guarantees, but it seems like
> a game changer so we are just looking for a bit more information so
> that we can make a decision.

Sparse checkout is not a silver bullet, but it does make a difference.
We still see commits take several seconds on Windows (even with sparse
index). This is *several orders of magnitude* better than SVN on our
repository (where naive commits on top-level folders can take tens of
minutes), but it's not what folks are going to be expecting from Git. In
the long-term, we're looking at what would be involved in splitting up
our monorepo and seeing whether the rewards really outweigh the costs
(both of which reach far beyond source control).

Best of luck!

--
Sean Allred



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux