On Fri, Jun 12, 2015 at 12:52:44PM +0200, Bernd Naumann wrote: > Hello again, > > After digging the code I may have got a clue where to start but I > would still appreciate some help from a developer, cause I have never > learned to write C. (Some basics at school which happened over a > decade ago.) > > Currently I have questions on: > > * How to patch clone: would cmd_clone() a good place? Or are there > other calls which might be better. I think about to insert the check > if a mirror will be setup or just updated, right after dest_exists. If you'd still like to modify "git clone" itself, then the "cmd_clone" entry point is certainly the place to start. I would suggest exploring other alternatives, though. Is it possible to use a caching HTTP proxy, so that "git clone" goes through a local caching proxy? I haven't tried this myself, so maybe it's not even possible, but that seems like a natural http-ish solution. Another idea is to use Git's URL rewriting feature. If your clone URLs all follow a similar pattern then they can automatically be rewritten to point to some other URL. e.g. in ~/.gitconfig: [url "file:///home/git/mirror/github.com/"] insteadOf = "https://github.com/"; This will make git clone from /home/git/mirror/github.com/ whenever it sees https://github.com/ URLs. This is not perfect because it ends up cloning from your local copies rather than setting up the references via --mirror, but at least it avoids hitting the network. You'll need to periodically update your local mirrors, though. If you prefer to keep ~/.gitconfig pristine then you could do it in a wrapper script by injecting e.g. the "-c" config flags, git \ -c url.file://foo/bar/.insteadOf=https://github.com/ \ clone ... > [...snip...] > > > > I often build in example 'openwrt' with various build-scripts which > > depends heavily on a fresh or clean environment and they omit many > > sources via `git clone`, which results sometimes in over 100 MB of > > traffic just for one build. /* Later needed .tar.gz source archives > > are stored in a symlinked download directory which is supported by > > 'openwrt/.config' since a few months... to reduce network traffic. > > */ Why does a rebuild delete existing Git repositories? That seems like a bad practice, and shouldn't be needed. If possible, it would be worth improving the build scripts. For example, a clone can be made pristine by doing "git reset --hard && git clean -fdx". Deleting a repository just so that it can be re-cloned is very wasteful. > > My connection to the internet is not the fastest in world and > > sometimes unstable, so I wanted to have some kind of local bare > > repository mirror, which is possible with `git clone --mirror`. > > > > From these repositories I can later clone from, by calling `git > > clone --reference /path/to.git <url>`, but I do not wish to edit > > all the build-scripts and Makefiles. Maybe it'd be possible to make just the "git clone" part of the build scripts configurable? That'd make it really easy to inject a wrapper script that scans the arguments and injects the needed --mirror arguments, in the case that the above options won't work. > > So I wrote a git wrapper script (`$HOME/bin/git`), which checks if > > `git` was called with 'clone', and if so, then it will first > > clones the repository as a mirror and then clones from that local > > mirror. If the mirror already exists, then it will only be updated > > (`git remote update`). This works for now. > > > > [...snip...] > > > > Ok, so far, so good, but the implementation of the current > > shell-prototype looks way too hacky [0] and I have found some edge > > cases on which my script will fail: The script depends on the > > fact that the last, or at least the second last argument is a > > valid git-url, but the following is a valid call, too : > > > > `git --no-pager \ clone git@xxxxxxxxxx:openwrt/packages.git > > openwrt-packages --depth 1` > > > > But this is not valid: > > > > `git clone https://github.com/openwrt/packages.git --reference > > packages.git packages-2` or `git clone --verbose > > https://github.com/openwrt/packages.git packages-2 --reference > > packages.git` > > > > > > I found out that git-clone actually also can only make a guess > > what is the url and what not. Another option is to rewrite the wrapper script in a better language. For example, Python's argparse module can handle the above cases with minimal fuss. Anyways, as I said before, the root problem is really the build scripts. I bet modifying the build scripts to reuse existing git repositories is easier than modifying "git clone". cheers, -- David -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html