Re: Cloning does not work on available download bandwidth changes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



W dniu 22.05.2024 o 23:52, brian m. carlson pisze:

On 2024-05-22 at 15:02:51, Paweł Bogusławski wrote:
Started to clone big git repo on not-too-fast, _stable_ VDSL link (up to
20mbps down)...

     $ git clone https://github.com/googleapis/google-api-go-client
     Cloning into 'google-api-go-client'...
     remote: Enumerating objects: 644446, done.
     remote: Counting objects: 100% (6922/6922), done.
     remote: Compressing objects: 100% (2904/2904), done.
     Receiving objects:   0% (3859/644446), 20.82 MiB | 1.01 MiB/s

...and then started to watch a VOD movie on same link; when VOD buffers
data, eats almost all available down bandwidth and leaves only about 100
kB/s for git...

     Receiving objects:   1% (7111/644446), 44.49 MiB | 130.00 KiB/s

...and when VOD stops buffering and whole bandwith is available for git
again, git transfer starts to grow...

     Receiving objects:   1% (7660/644446), 50.56 MiB | 575.00 KiB/s

...but finally git throws an error

     error: 181 bytes of body are still expected5 MiB | 1015.00 KiB/s
     fetch-pack: unexpected disconnect while reading sideband packet
     fatal: early EOF
     fatal: index-pack failed

or sometimes:

     error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly:
CANCEL (err 8)
     error: 6109 bytes of body are still expected
     fetch-pack: unexpected disconnect while reading sideband packet
     fatal: early EOF
     fatal: fetch-pack: invalid index-pack output

No such problems when downloading bigger file (i.e. linux kernel source)
with wget or curl instead of git clone (wget/curl transfer drops to about
100 kB/s when VOD buffers and increases to full speed when VOD is not
transferring and transfer finishes successfully).

Sounds like a bug in git; should not throw an error on available download
bandwidth changes as wget and curl do and should not require any params
tuning (to stop users flooding bugtrackers).
I don't believe this is a bug in Git.  The problem here is a network
issue of some sort that's causing the connection to be interrupted or
dropped.  This is very common, but the possible causes are many.

A lot of times we see this it's some sort of proxy, and that can be a
non-default antivirus or firewall or TLS MITM device, but that's usually
on Windows, and you're using Debian.  It can also be just a bad
connection, poor traffic management by your ISP, or a flaky wireless or
wired network card.

My _guess_ about what's happening here is poor traffic management:
because video is typically flagged as low-latency in DSCP, some device
on the path is prioritizing it to the exclusion of all other traffic,
which is causing the Git connection to be dropped.  I have no proof of
this, though.

Try to run

git clone https://github.com/googleapis/google-api-go-client

and limit interface speed somehow i.e.

tc qdisc add dev eth0 root handle 1: htb default 12
tc class add dev eth0 parent 1:1 classid 1:12 htb rate 20kbps ceil 20kbps
tc qdisc add dev eth0 parent 1:12 netem delay 1000ms

and after a while restore speed with i.e.

tc qdisc del dev eth0 parent 1:12 netem delay 1000ms

I tried this in Debian 11 and  git 2.30.2 died with:

error: 5723 bytes of body are still expected MiB | 920.00 KiB/s
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: index-pack failed








[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux