Patrick Steinhardt <ps@xxxxxx> writes: > When calling `inflateInit()` and `inflate()`, the zlib library will > allocate several data structures for the underlying `zstream` to keep > track of various information. Thus, when inflating repeatedly, it is > possible to optimize memory allocation patterns by reusing the `zstream` > and then calling `inflateReset()` on it to prepare it for the next chunk > of data to inflate. > > This is exactly what the reftable code is doing: when iterating through > reflogs we need to potentially inflate many log blocks, but we discard > the `zstream` every single time. Instead, as we reuse the `block_reader` > for each of the blocks anyway, we can initialize the `zstream` once and > then reuse it for subsequent inflations. > > Refactor the code to do so, which leads to a significant reduction in > the number of allocations. The following measurements were done when > iterating through 1 million reflog entries. Before: > > HEAP SUMMARY: > in use at exit: 13,473 bytes in 122 blocks > total heap usage: 23,028 allocs, 22,906 frees, 162,813,552 bytes allocated > > After: > > HEAP SUMMARY: > in use at exit: 13,473 bytes in 122 blocks > total heap usage: 302 allocs, 180 frees, 88,352 bytes allocated > Really nice how just reusing the data structure has such a significant impact.
Attachment:
signature.asc
Description: PGP signature
