Re: [MAKESHIFT PATCH] Cope better with a _lot_ of packs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote:
> 
> You might end up with a situation where you have tons of pack files, e.g.
> when using hg2git.  In this situation, all kinds of operations may 
> end up with a "too many files open" error.  Let's recover gracefully from 
> that.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@xxxxxx>

Yea, this looks right to me.  JGit had a similar problem and now
maintains a core.packedGitOpenFiles parameter, set to 128 by default,
as one of the rules it uses to limit the size of its pack cache area.

> diff --git a/sha1_file.c b/sha1_file.c
> index 28bd908..bd5edd8 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -720,6 +720,8 @@ static int open_packed_git_1(struct packed_git *p)
>  		return error("packfile %s index unavailable", p->pack_name);
>  
>  	p->pack_fd = open(p->pack_name, O_RDONLY);
> +	while (p->pack_fd < 0 && errno == EMFILE && unuse_one_window(p, -1))
> +		p->pack_fd = open(p->pack_name, O_RDONLY);
>  	if (p->pack_fd < 0 || fstat(p->pack_fd, &st))
>  		return -1;
>  
> @@ -937,6 +939,8 @@ static void prepare_packed_git_one(char *objdir, int local)
>  	sprintf(path, "%s/pack", objdir);
>  	len = strlen(path);
>  	dir = opendir(path);
> +	while (!dir && errno == EMFILE && unuse_one_window(packed_git, -1))
> +		dir = opendir(path);
>  	if (!dir) {
>  		if (errno != ENOENT)
>  			error("unable to open object pack directory: %s: %s",
> @@ -2339,6 +2343,8 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
>  
>  	filename = sha1_file_name(sha1);
>  	fd = create_tmpfile(tmpfile, sizeof(tmpfile), filename);
> +	while (fd < 0 && errno == EMFILE && unuse_one_window(packed_git, -1))
> +		fd = create_tmpfile(tmpfile, sizeof(tmpfile), filename);
>  	if (fd < 0) {
>  		if (errno == EACCES)
>  			return error("insufficient permission for adding an object to repository database %s\n", get_object_directory());
> -- 

-- 
Shawn.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]