Re: Tool to split huge archive before using MHonarc

["Christopher P. Lindsey" <lindsey@mallorn.com>]

> I would like to use MHonarc to process a huge 50meg file of mailing list
> messages that go back to 1995. Before doing so I'd like to break the large
> file up into separate monthly files in the form of YYYYMM. Are there any
> fairly recent, stable tools that will do this? I tried an old program called
> "spmail" but it craps out with a segmentation fault.

Hi Kevin,

I see that you're using Outlook Express as your mailer, so I don't
know if this will help you or not.  It's strictly UNIX based, but if
not, maybe it will help others on the list.

I also don't know what format your mail archive is in; if it's in Berkeley
mbox format the UNIX tools procmail and formail will probably work for you.
If not, let me know and we can probably still devise a scheme based on 
formail to split up your mailbox.

So...  The assumptions here are that you have access to UNIX and your 
mailbox is in Berkeley mbox format.

The first thing to do is create a procmailrc file that looks like this 
(call the file whatever you want):

   SHELL=/bin/sh
   DIR=/tmp/splitboxes
   :0 Wic
   * ? test ! -d $DIR
   | mkdir $DIR

   :0
   * ^From.*\/(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[ ]+[0-9]+[ ]+[0-9:]+[ ]+(19|20)[0-9][0-9]
   {
      FROM=$MATCH

      :0
      * FROM ?? ^^\/[^   ]+
      { MONTH = $MATCH }
   
      :0
      * FROM ?? .*(19|20)\/[0-9][0-9]
      { YEAR = $MATCH }
   
      :0:
      $DIR/$YEAR$MONTH
   }
   
   :0:
   $DIR/Unknown

Change DIR to be the directory where you want your mailboxes to 
appear (I recommend putting them in their own subdirectory).  Now run
this command:

   formail -s procmail -m procmailrc < mbox.in

Be sure to replace procmailrc with the name of the file that you created
above.  Replace mbox.in with the name of your mailbox archive.

There's one main difference between what you asked for and what you're
getting here -- the filenames will be in the format of YYYYmmm, where
mmm is the three letter abbreviation for the month.  

If your system has the GNU date program installed (part of the GNU
sh-utils package and most likely installed on Linux systems) you can
change things even further.  Your procmailrc file could look like this:

   SHELL=/bin/sh
   DIR=/tmp/splitboxes
   :0 Wic
   * ? test ! -d $DIR
   | mkdir -p $DIR

   :0
   * ^From [^ ]+[ ]+\/.*
   { 
      DATE=`date --date="$MATCH" +%Y%m`

      :0:
      $DIR/$DATE
   }
   
   :0:
   $DIR/Unknown

If all went well, your mailboxes will be split up the way that you asked.
You can change the format by tweaking the format sequence passed to the
date command (%Y%m in this case).  If there are messages in $DIR/unknown,
they will have failed this process and may need to be handled manually.

Chris

---------------------------------------------------------------------
To sign-off this list, send email to majordomo@mhonarc.org with the
message text UNSUBSCRIBE MHONARC-USERS