On Sat, Dec 18, 2021 at 08:50:02PM +0100, René Scharfe wrote: > compile_pcre2_pattern() currently uses the option PCRE2_UTF only for > patterns with non-ASCII characters. Patterns with ASCII wildcards can > match non-ASCII strings, though. Without that option PCRE2 mishandles > UTF-8 input, though -- it matches parts of multi-byte characters. Fix > that by using PCRE2_UTF even for ASCII-only patterns. > > This is a remake of the reverted ae39ba431a (grep/pcre2: fix an edge > case concerning ascii patterns and UTF-8 data, 2021-10-15). The change > to the condition and the test are simplified and more targeted. > > Original-patch-by: Hamza Mahfooz <someguy@xxxxxxxxxxxxxxxxxxx> > Signed-off-by: René Scharfe <l.s.r@xxxxxx> > --- > grep.c | 2 +- > t/t7812-grep-icase-non-ascii.sh | 6 ++++++ > 2 files changed, 7 insertions(+), 1 deletion(-) > > diff --git a/grep.c b/grep.c > index fe847a0111..5badb6d851 100644 > --- a/grep.c > +++ b/grep.c > @@ -382,7 +382,7 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt > } > options |= PCRE2_CASELESS; > } > - if (!opt->ignore_locale && is_utf8_locale() && has_non_ascii(p->pattern) && > + if (!opt->ignore_locale && is_utf8_locale() && > !(!opt->ignore_case && (p->fixed || p->is_fixed))) > options |= (PCRE2_UTF | PCRE2_MATCH_INVALID_UTF); > I tried to use 'git grep -P' for the first time ever, and it hung right away, spinning all CPUs at 100%. I could narrow it down, both the complexity of the pattern and the size of the input, see the test below, and it bisects to this patch. --- >8 --- #!/bin/sh test_description='test' . ./test-lib.sh test_expect_success PCRE 'test' ' # LC_ALL=C works LC_ALL=en_US.UTF-8 && cat >ascii <<-\EOF && foo bar baz EOF cat >utf8 <<-\EOF && foo bar báz EOF git add ascii utf8 && # These all work as expected: git grep --threads=1 -P " " ascii && git grep --threads=1 -P "^ " ascii && git grep --threads=1 -P "\s" ascii && git grep --threads=1 -P "^\s" ascii && git grep --threads=1 -P " " utf8 && git grep --threads=1 -P "^ " utf8 && git grep --threads=1 -P "\s" utf8 && # This hangs (but it does work with basic and extended regexp): git grep --threads=1 -P "^\s" utf8 ' test_done
