Hi Tom,
On Tue, 4 Jun 2019 at 05:22, Tom Hughes <tom@xxxxxxxxxx> wrote:
>
> On 04/06/2019 10:03, Elliott Sales de Andrade wrote:
>
> > I have narrowed this down to the mmap call at [6] which attempts to
> > map the backing file into memory. AFAICT, the mapped size is far below
> > the RAM on the build machine as well as far below the normal 32-bit VM
> > limit. So I don't know why the call is failing.
>
> It's not really about the physical memory because just doing the
> mmap won't actually read it all in and in any case the kernel will
> be quite happy to page bits in and out as needed.
>
> The issue will be whether there is that much contiguous address
> space available - on a 32 bit machine 2**28 is, at best, one
> sixteenth of the potential address space.
>
> I say at best because even with the best possible user:kernel
> split at least some address space is reserved to the kernel
> although that can be fairly small with some kernel configurations.
>
> Then you have to take the memory map into account though, and
> how address randomisation may have placed other mappings in
> locations that mean such a large contiguous mapping is not in
> fact available.
>
> Take a look an /proc/NNN/maps for the process when the mmap
> fails and see if it looks like there it a range of that size
> that is actually available...
>
Thanks for the hint. I ran another test and patched it to print
/proc/self/maps whenever loading the lmdb file failed [1] (ignore
other arch failures; I just forgot to disable debuginfo.) Flattening
down the ranges, the largest free space is only about 50% of what the
test would need.
However, looking at the maps also hinted at a way out. There were a
_lot_ of lmdb mappings and looking at the tests, they never explicitly
close the files and rely on garbage collection to clean them up. So I
wrote a somewhat hacky patch to do:
try:
self.db = lmdb.open(...)
except lmdb.MemoryError:
gc.collect()
self.db = lmdb.open(...)
and now it passes [2]. I will report something upstream so that they
can modify the tests to be more explicit about closing things.
That being said, this does not explain why it fails in s390x mock (and
continues to fail even with gc). But I guess getting it to work on
koji is good enough for me.
[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=35282558
[2] https://koji.fedoraproject.org/koji/taskinfo?taskID=35291525
--
Elliott
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
