On Monday 16 May 2011, Shawn Pearce wrote:
> On Sun, May 15, 2011 at 14:37, Johan Herland <johan@xxxxxxxxxxx> wrote:
> > Currently we refuse combining --max-pack-size with --stdout since
> > there's no way to make multiple packs when the pack is written to
> > stdout. However, we want to be able to limit the maximum size of the
> > pack created by --stdout (and abort pack-objects if we are unable to
> > meet that limit).
> >
> > Therefore, when used together with --stdout, we reinterpret
> > --max-pack-size to indicate the maximum pack size which - if exceeded
> > - will cause pack-objects to abort with an error message.
>
> ...
>
> > if (pack_to_stdout) {
> > + if (nr_written != nr_remaining)
> > + die("unable to make pack within the pack size"
> > + " limit (%lu bytes)", pack_size_limit);
>
> I think this is too late. We have already output a bunch of data, up
> to the size limit at this point. If the size limit is non-trivial
> (e.g. 5 MB) we have already sent most of that to the remote side, and
> its already written some of that out to disk.
>
> I'd like this to be a soft limit derived from the reused object sizes.
> When planning the pack by looking at where we will reuse an object
> from, sum those sizes. If the sum of these sizes would break this
> limit, then we abort before even writing the pack header out.
I agree, but it's currently late Sunday (early Monday), and after
looking at this for a while, I'm no longer thinking straight.
If someone that groks the pack-objects internal could help out, I'd be
really grateful. AFAICS, we need to drill into prepare_pack() to find
the details needed to estimate the total pack size, but I don't know
exactly which data structure(s) holds the data needed. We probably need
to accumulate a pack size estimate in find_deltas(), and then sum those
across the threads, before we finally compare the total estimate to
pack_size_limit prior to calling write_pack_file().
...Johan
--
Johan Herland, <johan@xxxxxxxxxxx>
www.herland.net
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html