MHonArc DOWNGRADE ALERT! (was Re: corruption problem )

[Earl Hood <ehood@hydra.acs.uci.edu>]

On November 12, 2001 at 21:27, Jeff Breidenbach wrote:

> I seem to be experiencing a systematic corruption problem.  For
> example, in the last half-dozen more recent entries of this date index
> [1], the message pages are non-existant and listed under the same URL.
> 
> The version of mhonarc running on this archive has been
>    2.early -> 2.49 -> 2.5.0 ->  2.4.9 -> 2.5.0
> 
> MHonArc 2.5.0 is not generating any warnings and is returning a good
> return code. What steps are suggested for diagnosis, and are there any
> suggestions for a fix? Rebuilding the archive from scratch is possible
> but not desirable, due to the large number of archives
> affected. Currently my top priority is stabilizing the system.

I just remembered that when upgrading to v2.5, there are some
data format changes to some items in the .mhonarc.db file.  Pre-v2.5
archives are automatically upgraded.  From the release notes:

  * If updgrading from v2.4.x, or earlier, reference and follow-up
    information of a message is now stored in a different format in the
    database (and internally). MHonArc will auto-update older archives to
    the new format, so no action should be required on your part.

However, if you downgrade, you will have problems.  I.e. Some data
in the .mhonarc.db is incompatible with previous versions.  I do
not know off-hand what would be the expected problems without more
analysis and if this is the cause to the problems you noted above.
Reference and follow-up information will definitely get screwed up.

The only way to safely downgrade is to pre-process the .mhonarc.db
file to convert from v2.5 format to v2.4 format to avoid data loss.

Jeff, I think the only way to properly fix your problem is to
regenerate archives. 

As for the problems for the Cygnus folks, I would avoid downgrading
if possible.  If you feel it is needed, I can help out with what steps
are needed to pre-process your .mhonarc.db files.  Or, you could
run v2.4.9's version of mha-dbrecover for each archive to reconstruct
.mhonarc.db files in v2.4.9 format (Jeff, you cannot do this since
you have already processed message page files and the reference information
contained within them will have been lost for any pages edited).

For those interested in a pre-processing approach to .mhonarc.db, 
what would need to be done can be determined by the update routine
in mhopt.pl that updates pre-v2.5 archives.  Here is the routine:

sub update_data_2_4_to_later {
    my($index, $value);
    while (($index, $value) = each(%Refs)) {
        next  if ref($value);
        $Refs{$index} = [ split(/$X/o, $value) ];
    }
    while (($index, $value) = each(%FollowOld)) {
        next  if ref($value);
        $FollowOld{$index} = [ split(/$bs/o, $value) ];
    }
    while (($index, $value) = each(%Derived)) {
        next  if ref($value);
        $Derived{$index} = [ split(/$X/o, $value) ];
    }
}

You will notice that list data has been converted from the old Perl 4
hack of storing lists within another data structure to using Perl 5
anonymous arrays.  The reverse of this process will have to be done
if you want to patch the .mhonarc.db files directly vs trying the
v2.4.9 mha-dbrecover script.

I apologize for the problems that exist.
When it rains, it pours.

--ewh