On Tue, Jul 30, 2013 at 12:05 AM, Brandon Casey <bcasey@xxxxxxxxxx> wrote: > When the number of open packs exceeds pack_max_fds, unuse_one_window() > is called repeatedly to attempt to release the least-recently-used > pack windows, which, as a side-effect, will also close a pack file > after closing its last open window. If a pack file has been opened, > but no windows have been allocated into it, it will never be selected > by unuse_one_window() and hence its file descriptor will not be > closed. When this happens, git may exceed the number of file > descriptors permitted by the system. > > This latter situation can occur in show-ref or receive-pack during ref > advertisement. During ref advertisement, receive-pack will iterate > over every ref in the repository and advertise it to the client after > ensuring that the ref exists in the local repository. If the ref is > located inside a pack, then the pack is opened to ensure that it > exists, but since the object is not actually read from the pack, no > mmap windows are allocated. When the number of open packs exceeds > pack_max_fds, unuse_one_window() will not able to find any windows to s/not able/not be able/ ...or... s/not able to find/not find/ > free and will not be able to close any packs. Once the per-process > file descriptor limit is exceeded, receive-pack will produce a warning, > not an error, for each pack it cannot open, and will then most likely > fail with an error to spawn rev-list or index-pack like: > > error: cannot create standard input pipe for rev-list: Too many open files > error: Could not run 'git rev-list' > > This is not likely to occur during upload-pack since upload-pack > reads each object from the pack so that it can peel tags and > advertise the exposed object. So during upload-pack, mmap windows > will be allocated for each pack that is opened and unuse_one_window() > will eventually be able to close unused packs after freeing all of > their windows. > > When we have file descriptor pressure, in contrast to memory pressure, > we need to free all windows and close the pack file descriptor so that > a new pack can be opened. Let's introduce a new function > close_one_pack() designed specifically for this purpose to search > for and close the least-recently-used pack, where LRU is defined as > > * pack with oldest mtime and no allocated mmap windows or > * pack with the least-recently-used windows, i.e. the pack > with the oldest most-recently-used window > > Signed-off-by: Brandon Casey <drafnel@xxxxxxxxx> -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html