Re: [PATCH 09/11] streaming: read non-delta incrementally from a pack

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, May 15, 2011 at 17:30, Junio C Hamano <gitster@xxxxxxxxx> wrote:
> +static read_method_decl(pack_non_delta)

I am not a huge fan of these decl macros... but I can see how writing
out the same function prototype 3 times is annoying.

> +       switch (in_pack_type) {
> +       default:
> +               return -1; /* we do not do deltas for now */

Haha. Deltas are going to be painful. Very, very painful.

We actually try to stream deltas in JGit. Our implementation isn't
useful. It only works if the base object is only accessed in
sequential order by the delta instruction sequences.

I had plans to add an external delta base cache to
$GIT_DIR/objects/delta-base-cache using a block file format that is
random accessible, but has CRC-32 checksums on each block to still
ensure there isn't silent data corruption on reads. You can read more
about it here http://egit.eclipse.org/r/1724 but the patch is probably
stalled and will get abandoned.

I think the better strategy is to avoid delta compression altogether
for objects that are so big we cannot materialize them as a contiguous
buffer. What a reasonable limit is, I don't know... but its probably
got to be around 25-50 MB. The Android project (as an example) has 6+
MB XML documents in their source code repository that are very delta
compressible.

-- 
Shawn.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]