Git chokes on large file

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I've discovered a problem using Git.  It's not clear to me what the
"correct" behavior should be, but it seems to me that Git is failing
in an undesirable way.

The problem arises when trying to handle a very large file.  For
example:

    $ git --version
    git version 1.8.3.1
    $ mkdir $$
    $ cd $$
    $ git init
    Initialized empty Git repository in /common/not-replicated/worley/temp/5627/.git/
    $ truncate --size=20G big_file
    $ ls -l
    total 0
    -rw-rw-r--. 1 worley worley 21474836480 May 27 11:59 big_file
    $ time git add big_file

    real	4m48.752s
    user	4m31.295s
    sys	0m16.747s
    $

At this point, either 'git fsck' or 'git commit' fails:

    $ git fsck --full --strict
    notice: HEAD points to an unborn branch (master)
    Checking object directories: 100% (256/256), done.
    fatal: Out of memory, malloc failed (tried to allocate 21474836481 bytes)

    $ git commit -m Test.
    [master (root-commit) 3df3655] Test.
    fatal: Out of memory, malloc failed (tried to allocate 21474836481 bytes)

The central problem is that one can accidentally add a file that
leaves the repository in a "broken" state, where various normal
commands simply don't work.  The most worrying aspect is that "git
fsck" fails -- of all the commands, the one that verifies the validity
of the repository (and diagnoses errors) should be the most robust!

Even doing a 'git reset' does not put the repository in a state where
'git fsck' will complete:

    $ git reset
    $ git fsck --full --strict
    notice: HEAD points to an unborn branch (master)
    Checking object directories: 100% (256/256), done.
    fatal: Out of memory, malloc failed (tried to allocate 21474836481 bytes)

Dale
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]