[RFC PATCH 00/10] range-diff: fix segfault due to integer overflow

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The difference between "master" and "git-for-windows/main" is large
enough that comparing the two will segfault on my system. This is
because the range-diff code does some expensive calculations and will
overflow the "int" type.

Fixing this wasn't trivial, our st_*() macros only work for unsigned
types, but as signed overflow is undefined in C detecting it is a lot
more painful. The range-diff code needs to store -1 in various places
(not inherently, but changing that looked a lot more painful).

Furthermore even if the range-diff.c and linear-assignment.c code were
fixed we'd still segfault due to the used string-list.c code using an
"int" type. Or rather, we tracked its "unsigned int" with an "int". If
we used "unsigned" we won't segfault on master..git-for-windows/main,
but we would soon enough on slightly larger divergent history.

This series changes the relevant APIs to use size_t where possible,
but as noted we need to use a signed type for range-diff.c. That's now
intmax_t.

We detect signed overflow first 8/10 with a GCC and Clang-specific
__builtin*() function, and then in the subsequent commit portably by
importing intprops.h from Gnulib.

Perhaps there's an easier way to do this, but this works for me, and
using the portable intprops.h (see its source for all the hard-won
lessons encdoded therein) seems like a good way forward. It *is*
slightly slower than our current unsigned-only detection, but as
discussed in 09/10 we should probably just switch to it anyway. This
series does not make that switch.

We still have various st_*() calls in the codebase where we use signed
types, presumably this is as broken as the not-really-working
detection discussed in 02/10, but I didn't looki into those in any
detail.

This is an RFC because there's some rather painful merge conflicts
in-flight due to the changeover from "unsigned int nr" to "size_t nr"
in the string-list.h API. There's also a CI failure on 32 bit linux
due to a format in 03/10, but it's an easily fixable bug.

And more generally it's an RFC perhaps this direction is a good one
for fixing that and any similar segfauls we have, and maybe it
isn't. I'm not all that sure about it, but this seems to work, and
certainly beats segfaulting...

Ævar Arnfjörð Bjarmason (10):
  string-list API: change "nr" and "alloc" to "size_t"
  range-diff.c: don't use st_mult() for signed "int"
  range-diff.c: use "size_t" to refer to "struct string_list"'s "nr"
  range-diff: zero out elements in "cost" first
  linear-assignment.c: split up compute_assignment() function
  linear-assignment.c: take "size_t", not "int" for *_count
  linear-assignment.c: convert a macro to a "static inline" function
  linear-assignment.c: detect signed add/mul on GCC and Clang
  linear-assignment.c: add and use intprops.h from Gnulib
  linear-assignment.c: use "intmax_t" instead of "int"

 builtin/receive-pack.c       |   6 +-
 builtin/shortlog.c           |   8 +-
 bundle.c                     |   4 +-
 commit-graph.c               |   4 +-
 compat/gnulib/.gitattributes |   1 +
 compat/gnulib/intprops.h     | 637 +++++++++++++++++++++++++++++++++++
 linear-assignment.c          | 149 +++++---
 linear-assignment.h          |   9 +-
 mailmap.c                    |   4 +-
 merge-ort.c                  |   2 +-
 range-diff.c                 |  32 +-
 string-list.h                |   2 +-
 t/helper/test-run-command.c  |   4 +-
 wt-status.c                  |   8 +-
 14 files changed, 781 insertions(+), 89 deletions(-)
 create mode 100644 compat/gnulib/.gitattributes
 create mode 100644 compat/gnulib/intprops.h

-- 
2.34.1.930.g0f9292b224d




[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux