On Fri, Jun 2, 2023 at 12:27 PM Paul Smith <paul@xxxxxxxxxxxxxxxxx> wrote:
>
> I have some scripting on my CI/CD servers that invokes git ls-remote
> and parses the output. The scripting is in Python. Sometimes, but not
> always, the output of this command is corrupted. I've enhanced the
> error handling and I see this:
>
> >> git ls-remote --heads origin
> *** INTERNAL: remote branch lookup failed:
> Output:
> -----
> 8431d80571dea5cc8e6d0848f27124f66346dcc4 refs/heads/foo1
> aaec1feb1167cf3fbd39a36cdd7736679a9f4fae refs/heads/foo2
> 6167c73fbaded389ff54d52a01878975f4a6d5e5 refs/heads/foo3
> ...
> 3a2e8036a6f6605d4dd14c72bd395298bff9d80e refs/heads/xxx1
> 3a2e8036a6f6605d4dd14c72bd395298bff9d80e refs/heads/xxx2
> 795d2ff669041fc91341cf5bf820aibab79dc92bd741e77a7dcf71d94285a6ae494dc0 refs/heads/yyy1
> 1496ea0ddab29ae3935754fced4bd5858cff7940 refs/heads/yyy2
> 1496ea0ddab29ae3935754fced4bd5858cff7940 refs/heads/yyy3
> -----
Can you trigger this problem with just `git ls-remote --heads origin`,
or do you only see it after processing by your python script?
If you can trigger with the former, what does
GIT_TRACE_PACKET=1 git ls-remote --heads origin
report?
If the latter, can you find a way to minimize your python script or
find some equivalent shell commands with the same buggy behavior?
> Also a bunch of the heads are missing. It's pretty clear that right in
> the middle of printing one of the SHAs we suddenly lost a bunch of
> output, and started printing stuff from later (in the last instance 66
> out of 131 heads were missing). Breaking down the output above you can
> see:
>
> 3a2e8036a6f6605d4dd14c72bd395298bff9d80e refs/heads/xxx2
> 795d2ff669041fc91341cf5bf820aibab79dc92bd741e77a7dcf71d94285a6ae494dc0 refs/heads/yyy1
> ^
>
> where the "795d2ff669041fc91341cf5bf820a" before the "i" char is a
> valid start of a SHA for a head (not shown), then the "i", then a fully
> valid SHA for heads/yyy1 which is 66 heads later.
Sounds kind of like
https://lore.kernel.org/git/6786526.72e2EbofS7@devpool47/ which also
triggered for some other tooling and then was reduced down to some
shell commands. Unfortunately, the thread ended without a lot of
resolution other than "don't mix stdout and stderr" and "if we slow
down the network connection somehow, that'll avoid the problem".
