On Friday 01 January 2010 @ 15:45, Paul Dekkers wrote: > > similar processes were killed. And the new archive-folder now ended up > with several duplicates, taking about millions instead of tens of > thousands. (We'll have to see how to dedup that, any ideas are > appreciated otherwise I'll write something for that.) I forgot to add, we used to use a perl script called dupseek to clean these up. It has some nice optimization that make it quite fast. http://freshmeat.net/projects/dupseek/ -Brian ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
