Re: git fast-import leaks memory drastically, so crashes with out of memory by attempt to import 22MB export dump

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Jeff,

well, I don't know how you were trying to reproduce it.

My first attempt with a git-repository (cloned from https://github.com/git/git.git) showed that immediately to me.
Here you go (I used git bash here):

```
# clone or copy git repository (we'll use it for export and import):
git clone https://github.com/git/git.git
cd git

# make 1st fast-export in order to generate marks (don't need the dump, only marks are needed): git fast-export --reencode=yes --export-marks=.tmp.exp-1.tmp e83c5163316f89bfbde7d9ab23ca2e25604af290..dc04167d378fb29d30e1647ff6ff51dd182bc9a3 > /dev/null

# make 2nd fast-export in order to generate partial export dump (file ".tmp.dump" will be ca. 87MB): git fast-export --reencode=yes --import-marks=.tmp.exp-1.tmp --export-marks=.tmp.exp-2.tmp 61addb841f2a6d74a1737a01e03df1f773e04944..master > .tmp.dump

# now try to import this dump, using first marks as import marks (we have all revs in git-repo): git fast-import --import-marks=.tmp.exp-1.tmp --export-marks=.tmp.imp.tmp < .tmp.dump

```

And see how git-fast-import eating your whole memory and enjoy the crash :)

```
fatal: Out of memory, malloc failed (tried to allocate 2097152 bytes)
fast-import: dumping crash report to .git/fast_import_crash_6684
```

Regards,
Sergey

15.10.2020 03:26, Jeff King wrote:

On Wed, Oct 14, 2020 at 11:22:03AM +0200, Dipl. Ing. Sergey Brester wrote:

I did not see any issues with (even much larger) imports, before I upgraded git to 2.28.0 (from 2.25.1, I guess).

I wasn't able to reproduce, but it sounds like there's something special
about this repo (since the dump is not that big, and you said much
larger dumps have succeeded).

Is it possible to make either the fossil repo, or the fast-import input
file available?

If not, since you said it works with 2.14.4, is it possible to bisect in
Git to see when it stopped working? You'll need to be able to build Git
from source.

-Peff



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux