Thank you for looking into this so quickly.
I’m unable to reproduce locally reliably but I created a custom pipeline to reproduce more quickly.
Here are the things I found out.
* With the NO_JIT flag and limited to only python files (in our case we only want to grep on py files anyways):
- git grep -c -P '(*NO_JIT)^[[:alnum:]_]+ = json.load' -- '*.py’
This is snappy and works, no more error.
* Without the flag and the *.py restriction:
- git grep -c -P '^[^ #][^#]+sys[.]argv’
This did not fail but took almost 3m, big performance regression.
- git grep -c -P '^[[:alnum:]_]+ = json.load’
Crashed and returned -11. Stderr was empty so I have no idea on what file it failed.
* With NO_JIT on all the files:
- git grep -c -P '(*NO_JIT)^[[:alnum:]_]+ = json.load’
This worked, that pattern is snappy but other patterns are very slow:
- git grep -c -P '(*NO_JIT)^[^ #][^#]+sys[.]argv’
Took 8m to complete.
* Without the flag but only *.py.
- git grep -c -P '^[[:alnum:]_]+ = json.load' -- '*.py’
All the patterns run fast (under 1s), and no errors.
Note that I was trying -E and replaced \w with [[:alnum:]_] … I’ll need to revert that, but I don’t thing \w is the issue.
Overall I would say that the issue is likely because the patterns are run against a non ASCII file somewhere in the repo.
Our repo is fairly large with files in various formats, including potentially some binaries that would definitely not be proper UTF-8.
For now I have a good workaround which is to only check for *.py files, which we should have done in the first place. The NO_JIT flag slows down things significantly so we will not use it here.
Do you have any recommendation on how to identify which file(s) is causing the crash considering there is nothing in stderr?
Thanks!
