Hi, On Wed, Apr 8, 2009 at 3:45 AM, Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote: > On Tue, 7 Apr 2009, Elijah Newren wrote: > >> Just thought I'd make this available, in case there's others with niche >> needs that find it useful... > > Have you seen > > http://thread.gmane.org/gmane.comp.version-control.git/52323 > > I was rather disappointed that skimo left the patch series in a rather > half-useful state. No, I had not. Looks useful, though it appears to be missing a number of options from git-filter-branch, such as --subdirectory-filter, --tree-filter, and --prune-empty. I'm guessing that's related to your "half-useful" comment? I was particularly interested in --tree-filter for my rewrite, since I needed it (to remove all cvs keywords and a few other touch-ups) and it was the slowest of the filters I'd be using. Problem is, in a repository with 40,000 commits and 70,000 files in the latest commits, --tree-filter is unacceptably slow. On average, a commit would have about 35,000 files (assuming approximately linear growth over the commit history), meaning that I'd have to modify 40,000 x 35,000 files = 1,400,000,000 files[1]. However, on average, less than a dozen files changed per commit, so there are less than 40,000 x 12 = 480,000 unique files that actually need to be rewritten. git-fast-export provides (and git-fast-import expects) just those half million files, and rewriting half a million files instead of 1.4 billion files is the difference between a 45 minute rewrite and a 3 month one. I didn't see a way to easily avoid the 1.4 billion file rewrite using git-filter-branch (or git-rewrite-commits had I known about it), and writing something to parse and modify git-fast-export output seemed like the easiest solution. Perhaps I could have written some fancy index-filter script that recorded original and modified file sha1sums somewhere and used that to only check out certain files and rewrite them, but such an idea hadn't occurred to me (and I'm not sure it would have been the better route even if it had). Maybe there's something I missed that would have made this easy, though? Elijah [1] For simplicity, I'm ignoring the 'binary' files that should not have any cvs-keyword unmunging performed on them. However, it does present an issue, particularly with extra process forks, since you need to determine which files are safe to modify. I used libmagic (the library behind the unix 'file' command) to avoid the need to run 'file' repeatedly. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html