Jan-Benedict Glaw <jbglaw@xxxxxxxxxx> wrote: > On Wed, 2006-03-22 14:33:37 +0100, Jan-Benedict Glaw <jbglaw@xxxxxxxxxx> wrote: > > Since it seems nobody looked at the GCC import run (which means to use > the svnimport), I ran it again, under strace control: If you don't care for automated branch handling, how about trying git-svn? under the contrib/ directory in git.git git-svn init svn://gcc.gnu.org/svn/gcc git-svn fetch > > GCC > > ~~~ > > $ /home/jbglaw/bin/git svnimport -C gcc -v svn://gcc.gnu.org/svn/gcc > > > Committed change 3936:/ 1993-03-31 05:44:03) > > Commit ID ceff85145f8671fb2a9d826a761cedc2a507bd1e > > Writing to refs/heads/origin > > DONE: 3936 origin ceff85145f8671fb2a9d826a761cedc2a507bd1e > > ... 3937 trunk/gcc/final.c ... > > Can't fork at /home/jbglaw/bin/git-svnimport line 379. > > ... 4279 trunk/gcc/config/i386/xm-sco.h ... > > This time it broke at a different revision, so I guess it's not a SVN > or git / git-svnimport problem, but rather a problem of my Perl > installation or the kernel itself? I've known of SVN library bindings leaking memory in the past, but I thought it's been solved. Afaik, any memory allocated by the Perl interpreter is never released back to the kernel, either. (At least that seems to be the case with my setup (Debian unstable, Perl 5.8.8, 2.6 kernel, x86 machine). > What are possible reasons for clone() to fail with -ENOMEN? I have to > admit that the box _is_ loaded a bit all the time: > > jbglaw@bixie:~/vax/git-conversion$ uptime > 19:23:58 up 136 days, 7:46, 20 users, load average: 4.45, 4.25, 3.05 > jbglaw@bixie:~/vax/git-conversion$ free > total used free shared buffers cached > Mem: 507308 501760 5548 0 2184 16900 > -/+ buffers/cache: 482676 24632 > Swap: 2441872 1295512 1146360 Some importers (my own git-svn included) aren't amazingly efficient when handling lots of history which gcc has. It looks like (from what I understand of the SVN api used in git-svnimport) is that the entire log for the 100k+ revisions in the tree is slurped down into memory before any processing is done. git-svn does this too, but by parsing the output of the svn binary instead of using the library, so at least it won't have issues with the svn bindings and libraries to worry about. My git-svn process running on the SVN tree just finished parsing the svn log output, and it's maxed out at 74M RSS (on a 32-bit x86). It'll probably take a while to import it all (which I won't do), but I could have just as easily done the following to reduce memory usage by ~half: git-svn fetch -r0:50000 # import the first 50000k git-svn fetch # now import the remaining Afaik, there's no way to do something like the above with git-svnimport for memory-starved setups. -- Eric Wong - : send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html