Re: Resumable clone/Gittorrent (again)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, Jan 9, 2011 at 8:55 PM, Luke Kenneth Casson Leighton
<luke.leighton@xxxxxxxxx> wrote:
> Âyou still have to come up with a mapping from "chains" to "pieces".
> in the bittorrent protocol the mapping is done *entirely* implicitly
> and algorithmically.

Given a commit SHA-1, the mapping can be done algorithmically because
the graph from the commit tip is fixed. Perhaps not mapping all at
once, but as you have more pieces in the graph, you can map more.

> the "meta" info in the .torrent contains
> filenames and file lengths. Âstack the files one after the other in a
> big long data block, get a chopper and just go "whack, whack, whack"
> at regular piece-long points, that's your "pieces". Âso, reassembly is
> a complete bitch, and picking just _one_ file to download rather than
> the whole lot becomes a total pain.

Well, there won't be .torrent files. Torrent files serve as checksums
for file pieces (let's forget the tracker part). We do sha-1 checksum
on every objects in git. The object graph without real content _is_
"info" part in .torrent files. Instead of passing around torrent
files, I only need to pass around the sha-1 of the commit tip(s). That
should be enough for any peer to discover the rest.

Reassembling, in its simplest way, is to just dump loose objects to
$GIT_DIR/objects. But it's been six years since git's birth now, I'll
pack them instead.

> Âthat's hard work and you're now into "unproven" territory. Âthe
> successful R&D proof-of-concept code that i wrote i _deliberately_
> stayed away from "adapting" a proven bittorrent protocol, and as a
> result managed to get that proof-of-concept up and running within ...
> i think it was... 3 days. Âmost of the time was spent arseing about
> adding in a VFS layer into bittornado, in order to libratise it.
>
> i mention that just to give you something to think about. Âif you're
> up to the challenge of writing your own p2p protocol, however, GREAT!
> you'll become a world expert on _both_ peer-to-peer protocols _and_
> git :)

Maybe I have gone insane ;) But I have another aim for this work: to
adjust narrow clone area (pretty much path-based clones). So while it
may not become real torrent for git (i.e p2p exchanging, depends on my
needs), restartable clone from multiple sources is still worth it.
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]