Hi,
this is the second patch series that converts the reftable library to
start handling allocation errors. This is done such that the reftable
library can truly behave like a library and let its callers handle such
conditions.
Changes compared to v1:
- Remove our use of `st_mult()` in `reftable_calloc()`.
- Initialize the `*dest` pointer with `NULL` in case creating a new
indexed table iterator fails.
Thanks!
Patrick
Patrick Steinhardt (22):
reftable/error: introduce out-of-memory error code
reftable/basics: merge "publicbasics" into "basics"
reftable: introduce `reftable_strdup()`
reftable/basics: handle allocation failures in `reftable_calloc()`
reftable/basics: handle allocation failures in `parse_names()`
reftable/record: handle allocation failures on copy
reftable/record: handle allocation failures when decoding records
reftable/writer: handle allocation failures in `writer_index_hash()`
reftable/writer: handle allocation failures in `reftable_new_writer()`
reftable/merged: handle allocation failures in
`merged_table_init_iter()`
reftable/reader: handle allocation failures for unindexed reader
reftable/reader: handle allocation failures in `reader_init_iter()`
reftable/stack: handle allocation failures on reload
reftable/stack: handle allocation failures in `reftable_new_stack()`
reftable/stack: handle allocation failures in `stack_compact_range()`
reftable/stack: handle allocation failures in auto compaction
reftable/iter: handle allocation failures when creating indexed table
iter
reftable/blocksource: handle allocation failures
reftable/block: handle allocation failures
reftable/pq: handle allocation failures when adding entries
reftable/tree: handle allocation failures
reftable: handle trivial allocation failures
Makefile | 1 -
refs/reftable-backend.c | 39 ++++--
reftable/basics.c | 92 ++++++++++++++-
reftable/basics.h | 13 +-
reftable/block.c | 23 +++-
reftable/block.h | 4 +-
reftable/blocksource.c | 25 +++-
reftable/error.c | 2 +
reftable/iter.c | 20 +++-
reftable/iter.h | 2 +-
reftable/merged.c | 84 ++++++++-----
reftable/merged.h | 6 +-
reftable/pq.c | 7 +-
reftable/pq.h | 2 +-
reftable/publicbasics.c | 66 -----------
reftable/reader.c | 68 ++++++++---
reftable/reader.h | 6 +-
reftable/record.c | 164 +++++++++++++++++++-------
reftable/record.h | 6 +-
reftable/reftable-basics.h | 18 +++
reftable/reftable-error.h | 3 +
reftable/reftable-malloc.h | 18 ---
reftable/reftable-merged.h | 8 +-
reftable/reftable-reader.h | 8 +-
reftable/reftable-stack.h | 8 +-
reftable/reftable-writer.h | 12 +-
reftable/stack.c | 177 ++++++++++++++++++++++------
reftable/tree.c | 42 +++++--
reftable/tree.h | 21 +++-
reftable/writer.c | 150 +++++++++++++++--------
t/helper/test-reftable.c | 10 +-
t/unit-tests/lib-reftable.c | 8 +-
t/unit-tests/t-reftable-basics.c | 11 +-
t/unit-tests/t-reftable-block.c | 24 ++--
t/unit-tests/t-reftable-merged.c | 16 ++-
t/unit-tests/t-reftable-readwrite.c | 61 ++++++----
t/unit-tests/t-reftable-stack.c | 4 +-
t/unit-tests/t-reftable-tree.c | 10 +-
38 files changed, 853 insertions(+), 386 deletions(-)
delete mode 100644 reftable/publicbasics.c
create mode 100644 reftable/reftable-basics.h
delete mode 100644 reftable/reftable-malloc.h
Range-diff against v1:
1: 8c99ecc3255 = 1: 8c99ecc3255 reftable/error: introduce out-of-memory error code
2: 4dcdf1d48ec = 2: 4dcdf1d48ec reftable/basics: merge "publicbasics" into "basics"
3: 21fa9b15d96 = 3: 21fa9b15d96 reftable: introduce `reftable_strdup()`
4: e6ded75f630 ! 4: f6ad92ffd01 reftable/basics: handle allocation failures in `reftable_calloc()`
@@ Commit message
Handle allocation failures in `reftable_calloc()`.
+ While at it, remove our use of `st_mult()` that would cause us to die on
+ an overflow. From the caller's point of view there is not much of a
+ difference between arguments that are too large to be multiplied and a
+ request that is too big to handle by the allocator: in both cases the
+ allocation cannot be fulfilled. And in neither of these cases do we want
+ the reftable library to die.
+
+ While we could use `unsigned_mult_overflows()` to handle the overflow
+ gracefully, we instead open-code it to further our goal of converting
+ the reftable codebase to become a standalone library that can be reused
+ by external projects.
+
Signed-off-by: Patrick Steinhardt <ps@xxxxxx>
## reftable/basics.c ##
-@@ reftable/basics.c: void *reftable_calloc(size_t nelem, size_t elsize)
+@@ reftable/basics.c: void reftable_free(void *p)
+
+ void *reftable_calloc(size_t nelem, size_t elsize)
{
- size_t sz = st_mult(nelem, elsize);
- void *p = reftable_malloc(sz);
+- size_t sz = st_mult(nelem, elsize);
+- void *p = reftable_malloc(sz);
+- memset(p, 0, sz);
++ void *p;
++
++ if (nelem && elsize > SIZE_MAX / nelem)
++ return NULL;
++
++ p = reftable_malloc(nelem * elsize);
+ if (!p)
+ return NULL;
- memset(p, 0, sz);
++
++ memset(p, 0, nelem * elsize);
return p;
}
+
5: 1f98abe9812 = 5: ad028020df7 reftable/basics: handle allocation failures in `parse_names()`
6: fa32be01e50 = 6: df713fbe08c reftable/record: handle allocation failures on copy
7: 372e36f880e = 7: 870bb003c0a reftable/record: handle allocation failures when decoding records
8: 7404d648a9d = 8: 1d47e425009 reftable/writer: handle allocation failures in `writer_index_hash()`
9: d6afb30221e = 9: caa71f0a775 reftable/writer: handle allocation failures in `reftable_new_writer()`
10: 349fd785a81 = 10: a84e9cadae4 reftable/merged: handle allocation failures in `merged_table_init_iter()`
11: c4985e64ce2 = 11: 20d38330141 reftable/reader: handle allocation failures for unindexed reader
12: 94c85ffd5ec = 12: e35c3a705d5 reftable/reader: handle allocation failures in `reader_init_iter()`
13: 6ef25b6c655 = 13: ca3b57f151e reftable/stack: handle allocation failures on reload
14: f5583cc2300 = 14: 7377421a632 reftable/stack: handle allocation failures in `reftable_new_stack()`
15: 3393a59ce42 = 15: 244e8667c5d reftable/stack: handle allocation failures in `stack_compact_range()`
16: 2c19f258fe7 = 16: 99f4868c38f reftable/stack: handle allocation failures in auto compaction
17: 32fead57de9 ! 17: 271839a6260 reftable/iter: handle allocation failures when creating indexed table iter
@@ reftable/iter.c: int new_indexed_table_ref_iter(struct indexed_table_ref_iter **
itr->offset_len = offset_len;
err = indexed_table_ref_iter_next_block(itr);
-- if (err < 0) {
+ if (err < 0)
+ goto out;
+
@@ reftable/iter.c: int new_indexed_table_ref_iter(struct indexed_table_ref_iter **
+ err = 0;
+
+out:
-+ if (err < 0)
+ if (err < 0) {
++ *dest = NULL;
reftable_free(itr);
- } else {
- *dest = itr;
-- }
+ }
return err;
}
-
## reftable/iter.h ##
@@ reftable/iter.h: void iterator_from_indexed_table_ref_iter(struct reftable_iterator *it,
18: 4d672ab9b15 = 18: c41808e9d75 reftable/blocksource: handle allocation failures
19: 4fa47a559b2 = 19: 9348b0a4575 reftable/block: handle allocation failures
20: a569c39fd8e = 20: 445daf9464d reftable/pq: handle allocation failures when adding entries
21: 99bca688f2a = 21: c31f9e53f52 reftable/tree: handle allocation failures
22: 02073dafb96 = 22: d0fe9993716 reftable: handle trivial allocation failures
--
2.46.0.551.gc5ee8f2d1c.dirty
