> On Oct 18, 2014, at 19:33, Eric Wong <normalperson@xxxxxxxx> wrote:
>
> Eric Wong <normalperson@xxxxxxxx> wrote:
>> This reduces hash lookups for looking up cache data and will
>> simplify tying data to disk in the next commit.
>
> I considered the following, but GDBM might not be readily available on
> non-POSIX platforms. I think the other problem is the existing caches
> are still in memory (whether YAML or Storable) even if disk-backed,
> causing a large amount of memory usage anyways.
If cached_mergeinfo is using too much memory, you can probably drop that cache entirely. IIRC, it didn't give that much of a speed up.
I am surprised that it is using a lot of memory, though. There is only one entry per SVN branch.
> (Both patches on top of Jakob's)
> -------------------------
> Subject: [RFC] git-svn: tie cached_mergeinfo to a GDBM_File store
>
> This should reduce per-instance memory usage by allowing
> serialization to disk. Using the existing Memoize::Storable
> or YAML backends does not allow fast lookups.
>
> GDBM_File should be available in most Perl installations
> and should not pose unnecessary burden
> ---
> perl/Git/SVN.pm | 19 ++++++++++++++++---
> 1 file changed, 16 insertions(+), 3 deletions(-)
>
> diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
> index 25dbcd5..3e477c7 100644
> --- a/perl/Git/SVN.pm
> +++ b/perl/Git/SVN.pm
> @@ -14,6 +14,7 @@ use IPC::Open3;
> use Memoize; # core since 5.8.0, Jul 2002
> use Memoize::Storable;
> use POSIX qw(:signal_h);
> +use Storable qw(freeze thaw);
>
> use Git qw(
> command
> @@ -1713,10 +1714,21 @@ sub mergeinfo_changes {
>
> # Initialize cache on the first call.
> unless (defined $cached_mergeinfo) {
> - $cached_mergeinfo = $self->{cached_mergeinfo} = {};
> + my %hash;
> + eval '
> + require File::Temp;
> + use GDBM_File;
> + my $fh = File::Temp->new(TEMPLATE => "mergeinfo.XXXXXXXX");
> + $self->{cached_mergeinfo_fh} = $fh;
> + $fh->unlink_on_destroy(1);
> + tie %hash => "GDBM_File", $fh->filename, GDBM_WRCREAT, 0600;
> + ';
> + $cached_mergeinfo = $self->{cached_mergeinfo} = \%hash;
> }
>
> my $cached = $cached_mergeinfo->{$old_path};
> + $cached = thaw($cached) if defined $cached;
> +
> if (defined $cached && $cached->[0] == $old_rev) {
> $old_minfo = $cached->[1];
> } else {
> @@ -1735,11 +1747,12 @@ sub mergeinfo_changes {
> $props->{"svn:mergeinfo"};
> $old_minfo = \%omi;
> }
> - $cached_mergeinfo->{$old_path} = [ $old_rev, $old_minfo ];
> + $cached_mergeinfo->{$old_path} =
> + freeze([ $old_rev, $old_minfo ]);
> }
>
> # Cache the new mergeinfo.
> - $cached_mergeinfo->{$path} = [ $rev, \%minfo ];
> + $cached_mergeinfo->{$path} = freeze([ $rev, \%minfo ]);
>
> my %changes = ();
> foreach my $p (keys %minfo) {
> --
> EW
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html