[TOPIC 5/17] Partial Clone

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



1. Stolee: what is the status, who is deploying it, what issues need to be handled? Example, downloading tags. Hard to highly recommend it.

2. Taylor: we deployed it. No activity except for internal testing. Some more activity, but no crashes. Have been dragging our feet. Chicken egg, can’t deploy it because the client may not work, but hoping to hear about problems.

3. ZJ: dark launched for a mission critical repos. Internal questions from CI team, not sure about performance. Build farm hitting it hard with different filter specs.

4. Taylor: we have patches we are promising to the list. Blob none and limit, for now, but add them incrementally. Bitmap patches are on the list

5. James: I’ve been talking to customers who have high interest in this. But they are hesitant. Do people have similar situations, like shallow clones?

6. Jonathan N: we’re not using it en masse with server farms (see Terry’s stats). Performance issues with catchup, long periods of downloading and no progress. Missing progress display means a user waits and gets worried. On server side, reachability check can be expensive, in part because enumerating refs is expensive.

7. Peff: client experience sucks with N+1 situations. If the server operator side is tolerable, that way it’s easier to move the client side forward. By default, v2 just serves them up, no reachability check. Not sure if we’ll do that forever. Often have to inflate blobs that are deltas, and then delta compression which is not needed.

8. Stolee: Jonathan built a batch download when changing trees. Possible to improve by sending haves.

9. Jonathan N: if you’re in the blob none filter, and say I have a commit, I might not actually have what the server expects.

10. Peff: could enumerate blobs

12. Demetr: Partial clones are dangerous for DoS attacks

12. Jonathan: JGit forbids most filters that can't use bitmaps.

13. Peff: just blob filters? Yes, so far.

14. Jonathan: as far as the client experience goes, we’re not batching often enough and not showing progress on catch-up fetches. Any other UX issues?

15. Jeff: no, those two are what I meant.

16. James: another question for git service providers: Is it a replacement for LFS?

17. Brian: some files can compress, others don’t. Repacking can blow up if you try to compress something that can’t be compressed. How do we identify which objects we compress, and which we don’t.

18. Jonathan N: if you see something already compressed, tell zlib to do passthrough compression.

19. Taylor: two problems - which projects do you want to quarantine, where do you put them. CDN offloading would be nice.

20. Stolee: reachability bitmaps are tied to a single packfile. Becomes more and more expensive. Even just having them in another file requires a lot of work.

21. Taylor: we’re looking at some heuristics so that some parts of the pack can just be moved over verbatim.

22. Peff: I see three problems: multi pack lookups, bitmaps,

23. Jonathan N: we never generate on the fly deltas

24. Peff: there are pathological cases.

25. Terry: we are seeing 89k partial clones per day. Majority is clone. Shallow clone equivalent.

26. Peff: why? Is it better?

27. Jonathan N: initial clone is about the same as shallow. One reason we encourage, if you do a follow up, with shallow clone it is expensive for the server.

28. Stolee: if you persist the previous shallow clone, it is much much cheaper to do incremental fetch.

29. Terry: JGit has enough shallow clone bugs that we often just send everything. Make shallow clone obsolete

30. Jonathan N: Jenkins style CI, option for shallow clone. Want to run diff or git describe, have to turn it off. Partial clone is simpler.

31. Minh: could the server force the client to partial clone?

32. Brian: risks, working on an airplane. I don’t want to do any kind of fetch operation on poor connection. Could be good for CI, but don’t want to break things for humans.

33. Jonathan N: if I am going to get on an airplane, is there a way to fill it in the background. There are workarounds, like run `git show` which needs everything.

34. Elijah: I want to fetch a bunch more stuff, but don’t fetch anymore, throw an error rather than hanging.

35. Jonathan: filter blob:none is people's first experience of the feature. Make it a first class ui concept, present a user oriented UI like git sparse-checkout?

36. Taylor: It looks like it’s simple to use, but there’s a lot to do to actually use it. And Scalar is doing that for you.

37. James: Some of our customers would be interested to have a feature that pushes down configuration to all the users. It would give them LFS by default, without the end-users doing something.

38. Jonathan: We considered enabling a global config at Google. For example for 1+GB files.



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux