Re: Summer of Code project ideas due this Friday

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Comments on the "Better big-file support" section:


"While git can handle arbitrary-sized binary content [...]"

This is very much not true. Git tries at many places to load the complete file into memory and usually fails with "out of memory" if it can't. With the 32bit msysGit client this places the upper file size limit, from purely empirical observation, at 600-700 MByte. When a file is to large git fails late, adding and committing works (as long as there are no filters or other complications), but you can forget about pushing, rebasing or otherwise manipulating that commit. Even worse yet, commits consisting of smaller files but with a combined size over the limit will also cause out-of-memories.

Thus a main focus should be the memory problem, e.g. by using stream-like file handling everywhere, since not working at all is orders of magnitude worse than working slowly :)



"In some cases, this may be as simple as having a "large file" codepath that avoids pulling whole files into memory (e.g., during "git add")."

Ironically git add is one of the few things that work with large files, as mentioned above. Presumably the stream-oriented zlib enforced/encouraged a steam-like handling here :)
Slow as hell though and of course it is usually not sensible to compress a 1.5 GByte file.




I'm very willing to work on this topic. Though I'm not a student and as a git code newbie I also don't have the skills for mentoring yet.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]