On Fri, Dec 20, 2024 at 02:23:49PM -0500, rsbecker@xxxxxxxxxxxxx wrote: > On December 20, 2024 2:08 AM, Patrick Steinhardt wrote: > >On Thu, Dec 19, 2024 at 03:46:20PM -0500, rsbecker@xxxxxxxxxxxxx wrote: > >> On December 18, 2024 11:07 AM, I wrote: > >> >All tests, actually. > >> > > >> >$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true sh > >> >t0000- basic.sh --verbose -i -x > >> >error: reftable: transaction prepare: out of memory > >> >error: cannot run git init > >> > >> Any updates or hypothesis on this? Our test system has loads of memory > >> - I cannot figure out where the allocation failure takes place. There > >> is a limit to how much memory can be allocated, but it is very high > >> and our virtual memory is extensive, but this is a 32-bit build. > > > >My hypothesis is that this is caused by ps/reftable-alloc-failures, but I > am unable to > >tell where exactly the error comes from. So I'm dependent on your input. > > > >Could you please bisect the error? Finding out where the error is raised > would also > >be quite helpful. It has to be one of the reftable functions that returns > >REFTABLE_OUT_OF_MEMORY_ERROR, but other than that I do not have any more > >gut feeling right now. > > This is what bisect shows: > > git bisect start > # status: waiting for both good and bad commits > # good: [777489f9e09c8d0dd6b12f9d90de6376330577a2] Git 2.47 > git bisect good 777489f9e09c8d0dd6b12f9d90de6376330577a2 > # status: waiting for bad commit, 1 good commit known > # bad: [063bcebf0c917140ca0e705cbe0fdea127e90086] Git 2.48-rc0 > > git bisect bad 063bcebf0c917140ca0e705cbe0fdea127e90086 > # bad: [2037ca85ad93ec905b46543df6df4080f6ca258b] worktree: refactor > `repair_worktree_after_gitdir_move()` > git bisect bad 2037ca85ad93ec905b46543df6df4080f6ca258b > # bad: [6a11438f43469f3815f2f0fc997bd45792ff04c0] The fifth batch > git bisect bad 6a11438f43469f3815f2f0fc997bd45792ff04c0 > # bad: [f004467b042d735a2fe8bd5706b053b04b1aec65] Merge branch > 'jh/config-unset-doc-fix' > git bisect bad f004467b042d735a2fe8bd5706b053b04b1aec65 > # bad: [e29296745dc92fb03f8f60111b458adc69ff84c5] Merge branch > 'sk/doc-maintenance-schedule' > git bisect bad e29296745dc92fb03f8f60111b458adc69ff84c5 > # bad: [5b67cc6477ce88c499caab5ebcebd492ec78932d] reftable/stack: handle > allocation failures in auto compaction > git bisect bad 5b67cc6477ce88c499caab5ebcebd492ec78932d > # good: [31f5b972e0231d4211987775dd58e67815734989] reftable/record: handle > allocation failures when decoding records > git bisect good 31f5b972e0231d4211987775dd58e67815734989 > # bad: [18da60029319733e2d931f2758a8e47b8b25b117] reftable/reader: handle > allocation failures for unindexed reader > git bisect bad 18da60029319733e2d931f2758a8e47b8b25b117 > # good: [74d1c18757d1a45b95e46836adf478193a34c42c] reftable/writer: handle > allocation failures in `reftable_new_writer()` > git bisect good 74d1c18757d1a45b95e46836adf478193a34c42c This is missing the last step for git-bisect(1). Right now it could be caused by one of these commits: 18da600293 (reftable/reader: handle allocation failures for unindexed reader, 2024-10-02) 802c0646ac (reftable/merged: handle allocation failures in `merged_table_init_iter()`, 2024-10-02) The first commit seems quite unlikely to be the root cause. The second commit is rather interesting though. I wonder whether NonStop's malloc returns a NULL pointer when given a size of 0? A quick stab into the dark, but does below patch on top of `master` make things work for you? Patrick -- >8 -- diff --git a/reftable/merged.c b/reftable/merged.c index bb0836e344..7ae6f78d45 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -244,7 +244,7 @@ int merged_table_init_iter(struct reftable_merged_table *mt, struct merged_iter *mi = NULL; int ret; - REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len); + REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len + 1); if (!subiters) { ret = REFTABLE_OUT_OF_MEMORY_ERROR; goto out; diff --git a/reftable/stack.c b/reftable/stack.c index 59fd695a12..1b6b8cc9ea 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -1612,7 +1612,7 @@ static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st) int overhead = header_size(version) - 1; uint64_t *sizes; - REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len); + REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len + 1); if (!sizes) return NULL;