Thanks for taking the time to send me that patch. It looks like there are several areas where mi_offset needs to be checked. I tested it on 4.2.1 (had to hand apply a couple rejects, but that was no big real). But it still fails in the same way. I am not 100% surprised since I too tried using dbiByteSwapped(dbi) as an indicator as to when to swap, but found it didn't think things were swapped even when they were. For example, I've added the two lines (marked with +): memcpy(&mi_offset, keyp, sizeof(mi_offset.ui)); if (dbiByteSwapped(dbi) == 1) _DBSWAP(mi_offset); + if (dbiByteSwapped(dbi) != 1) + printf("Not swapped, but offset=%d\n", mi_offset.ui); mi->mi_offset = mi_offset.ui; and I still see large (i.e. unswapped) offsets as follows: ---------------------------------------------------- root@SBC8265:/root> rpm -qa Not swapped, but offset=0x2000000 acl-2.2.23-1 Not swapped, but offset=0x4000000 at-3.1.8_11-1 Not swapped, but offset=0x6000000 [...] minicom-2.1-1 Not swapped, but offset=0x3e000000 memory alloc (260046852 bytes) returned NULL. root@SBC8265:/root> file /var/lib/rpm/Packages /var/lib/rpm/Packages: Berkeley DB (Hash, version 8, little-endian) root@SBC8265:/root> --------------------------------------------------- Note that file shows it as little endian and not native endian. So you'd think that dbiByteSwapped() would report the right thing, but it doesn't seem to. This is why gave up on dbiByteSwapped(dbi) and used something like: if (mi->mi_offset > 0x00ffffff) { /* swap mi_offset here */ } even though dbiByteSwapped() would have been a cleaner way to do it. Paul. On 4/28/05, Paul Nasrat <pnasrat@xxxxxxxxxx> wrote: > On Wed, 2005-04-27 at 19:03 -0400, Paul Gortmaker wrote: > > [Yet more info, and a work-around] > > > > I found rpmdbNextIterator in rpmdb/rpmdb.c does a dbiGet(...key...) > > and then memcpy(&mi->mi_offset, key->data ...). The mi_offset is the > > value that has unaccounted for endian issues, and since it is used in > > calculating the allocation size for PBM_REALLOC, we get huge > > allocation requests in PBM_REALLOC. > > Thanks for the research - I'm pretty sure that this should be fixed > already on HEAD. > > revision 1.113 > date: 2004/03/25 19:41:22; author: jbj; state: Exp; lines: +24 -10 > Endian neutral join keys for rpmdbAdd() and rpmdbRemove(). > ---------------------------- > revision 1.112 > date: 2004/03/25 18:14:03; author: jbj; state: Exp; lines: +51 -31 > - endian neutral rpmdb join keys (finally). > ---------------------------- > revision 1.111 > date: 2004/03/18 15:29:23; author: jbj; state: Exp; lines: +18 -6 > Resurrect "other endian" rpmdb functionality. > > cvs diff -u -r 1.110 -r 1.113 rpmdb.c > > That applies to the rpm 4.2 branch - untested diff against 4.2 branch > attached. If you get the opportunity to test that'd be great. > > > I did an endian swap on mi_offset whenever I found it was a huge > > value, and also did an "rpm --rebuilddb" (which used to fail with the > > same huge allocation problem) and now things seem to work sanely. > > There is a function to see if db is other endian so you can do eg: > > if (dbiByteSwapped(dbi) == 1) > _DBSWAP(mi_offset) > > Paul > > > _______________________________________________ Rpm-list mailing list Rpm-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/rpm-list