BinaryDataContainer swapping ? ...

[Michael Meeks <michael.meeks@xxxxxxxxxxxxx>]

Hi there,

	I've been looking at some memory profiles, and it seems to me that we're storing too much bulk image data in memory. I understand that previously we had real problems with image lifecycle and loss due to optimistic attempts to use the original document as a reliable source for fetching data from; so it made good sense to brute-force this by holding all images in memory: fair enough - I've not seen us loose an image while copy/pasting/closing/opening etc. since that was done - which is awesome, and I don't want to go back to that =)

	Nevertheless, the memory profiles show that for large documents the compressed image data can also be large: hundreds of Mb (and yes we have some customers that love copy/pasting lots of large complex images into their files ;-)

	Anyhow - I was doing some re-factoring of the BinaryDataContainer to encapsulate it better; and I was thinking of adding the ability to swap these out (which should be async), and to then read them back in on demand which (I hope) in the era of SSDs - should be extremely fast, although synchronous.

	Then again - since I've not touched these code paths for a long time; and lost any deep understanding I may once have had, I'd love some advice:

https://git.libreoffice.org/core/+log/refs/heads/private/mmeeks/swapdatacontainer

	Would be good to have two forms of that: advice for long-term star-trek futures, and also advice for quick-fix-that-goes-in-the-right-direction =)

	Beyond that - I'd -really- like to get more de-compressed images out of memory; the 20 MPixel image that turns into 80Mb of RAM - and then is interpolated down to some 1080p screen - so ~2Mpixel once, then cached, and re-rendered from the cache. I wonder - will our existing usage of eg. cairo's scaling cache allow us to push the giant backing bitmap out of memory, and leave the 8Mb alternative in the scaling cache ? ;-)

	Thoughts appreciated,

		Michael.

--
michael.meeks@xxxxxxxxxxxxx <><, GM Collabora Productivity
Hangout: mejmeeks@xxxxxxxxx, Skype: mmeeks
(M) +44 7795 666 147 - timezone usually UK / Europe