We actually pursued trying to make submodules work for some time and even built tooling around trying to work around some of the issues we ran into (not repo.py but along a similar line) before determining that we would be better served by having a single repo and solving the scale issues. I don't want to rehash the arguments for/against a single repo - suffice it to say, we have opted for a single large repo. :) Thanks, Ben > -----Original Message----- > From: Stefan Beller [mailto:sbeller@xxxxxxxxxx] > Sent: Tuesday, January 17, 2017 5:24 PM > To: Martin Fick <mfick@xxxxxxxxxxxxxx> > Cc: Ben Peart <peartben@xxxxxxxxx>; Shawn Pearce > <spearce@xxxxxxxxxxx>; git <git@xxxxxxxxxxxxxxx>; > benpeart@xxxxxxxxxxxxx > Subject: Re: [RFC] Add support for downloading blobs on demand > > On Tue, Jan 17, 2017 at 2:05 PM, Martin Fick <mfick@xxxxxxxxxxxxxx> > wrote: > > On Tuesday, January 17, 2017 04:50:13 PM Ben Peart wrote: > >> While large files can be a real problem, our biggest issue today is > >> having a lot (millions!) of source files when any individual > >> developer only needs a small percentage of them. Git with 3+ million > >> local files just doesn't perform well. > > > > Honestly, this sounds like a problem better dealt with by using git > > subtree or git submodules, have you considered that? > > > > -Martin > > > > I cannot speak for subtrees as I have very little knowledge on them. > But there you also have the problem that *someone* has to have a whole > tree? (e.g. the build bot) > > submodules however comes with a couple of things attached, both positive > as well as negative points: > > * it offers ACLs along the way. ($user may not be allowed to > clone all submodules, but only those related to the work) > * The conceptual understanding of git just got a lot harder. > ("Yo dawg, I heard you like git, so I put git repos inside > other git repos"), it is not easy to come up with reasonable > defaults for all usecases, so the everyday user still has to > have some understanding of submodules. > * typical cheap in-tree operations may become very expensive: > -> moving a file from one location to another (in another > submodule) adds overhead, no rename detection. > * We are actively working on submodules, so there is > some momentum going already. > * our experiments with Android show that e.g. fetching (even > if you have all of Android) becomes a lot faster for everyday > usage as only a few repositories change each day). This > comparision was against the repo tool, that we currently > use for Android. I do not know how it would compare against > single repo Git, as having such a large repository seemed > complicated. > * the support for submodules in Git is already there, though > not polished. The positive side is to have already a good base, > the negative side is to have support current use cases. > > Stefan