Hi,
a few folks from the Git LFS project and I try to make cloning of repositories
with a lot of LFS files faster.
The core problem is that Git LFS uses a Git smudge filter to replace LFS
pointers with the actual file content. Right now, a smudge filter can only
be executed on an individual file which makes the operation slow for many
files [1].
We solved this issue by temporarily disabling the smudge filter for the clone
command via Git config (optimized in 1a8630 [2]):
git -c filter.lfs.smudge= -c filter.lfs.required=false clone <url> <path>
Afterwards Git LFS runs a special command to download and replace all LFS
content in bulk [3]. This works great for LFS repositories.
However, I noticed that git config command line instructions such as
"-c filter.lfs.smudge=" are not passed to Git submodule operations. Thus
this does not work as expected:
git -c filter.lfs.smudge= -c filter.lfs.required=false clone --recursive <url> <path>
I tried to work around that by copying the relevant pieced from the Git
Submodule command [4] and applying the command line Git config
manually (look closely at the modified checkout command):
git -c filter.lfs.smudge= -c filter.lfs.required=false clone $@
if [[ -z $2 ]]; then
CLONE_PATH=$(basename ${1%.git});
else
CLONE_PATH=$2;
fi
pushd "$CLONE_PATH"
git submodule init
wt_prefix=$(git rev-parse --show-prefix)
git submodule--helper list --prefix "$wt_prefix" | {
while read mode sha1 stage sm_path
do
name=$(git submodule--helper name "$sm_path") || exit
url=$(git config submodule."$name".url)
if ! test -d "$sm_path"/.git && ! test -f "$sm_path"/.git
then
git submodule--helper clone --prefix "$wt_prefix" --path "$sm_path" --name "$name" --url "$url"
pushd "$sm_path"
git -c filter.lfs.smudge= -c filter.lfs.required=false checkout -q $sha1 || exit
git-lfs pull || exit
popd
fi
done
}
popd
Do you see an easier way to pass command line Git config instructions to the
underlaying Git Submodule commands? If not, do you think a patch adding this
would be worth working on?
I also started a discussion about that on the Git LFS issue page [5].
Thanks,
Lars
[1] https://github.com/github/git-lfs/issues/931
[2] https://github.com/git/git/commit/1a8630dc3b1cc6f1361a4e5d94630133c24c97d9
[3] https://developer.atlassian.com/blog/2016/04/git-lfs-12-clone-faster/
[4] https://github.com/git/git/blob/6a6636270fbaf74609cd3e1bd207dd2c420d640a/git-submodule.sh#L686-L731
[5] https://github.com/github/git-lfs/issues/1172
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html