Hi Drew, Drew Northup writes: > On Thu, 2011-01-20 at 10:20 +0530, Ramkumar Ramachandra wrote: > > Hi, > > Jonathan Nieder writes: > > > Junio C Hamano wrote: > > > > Ramkumar Ramachandra <artagnon@xxxxxxxxx> writes: > > > > Just thinking aloud, but is it possible to write a filter that converts an > > > > arbitrary G-F-I stream with referenced blobs into a G-F-I stream without > > > > referenced blobs by inlining all the blobs? > > > > > > to avoid complexity in the svn fast-import backend itself. > > > (Complicating detail: such a filter would presumably take responsibility > > > for --export-marks, so it might want a way to retrieve commit marks > > > from its downstream.) > > > > This filter will need to persist every blob for the entire lifetime of > > the program. We can't possibly do it in-memory, so we have to find > > some way to persist them on-disk and retrieve them very > > quickly. Jonathan suggested using something like ToyoCabinet earlier- > > I'll start working and see what I come up with. > > Is it worth including the extra dependency? Most systems that I'm in > frequent contact with already have some lightweight BDB implementation > already. I don't currently know of any with TokyoCabinet (or > KyotoCabinet for that matter) already in place. Besides, if all you're > doing is persisting blobs that you're likely to write out to disk > eventually anyway you might as well just do so once you have them and > keep an "index" (not to be confused with the Git Index, just lacking a > better word right now) of what you have in some standard in-memory > format (a heap?). From there you can build each commit into the Git > Index in the proper order once you have the required parts for > each--perhaps even re-using the blobs you've already dumped to disk > (mv'ing them or something). Agreed. I wouldn't like to introduce an extra dependency either. I was talking about using it for prototyping- if the final version includes an extra dependency, it's unlikely to get merged into git.git :) The final design will probably use an in-memory B+ tree, but I haven't thought about that hard enough. -- Ram -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html