On 2024-11-13 02:59, Zdenek Kabelac wrote:
Dne 12. 11. 24 v 20:40 Shao Miller napsal(a):
Good day, Device Mapper Folks.
In the format of an X Y Problem:
[Almost] Real goal (X): I would like to read storage-blocks locally
from a far-away (remote, high latency) source and keep costs (for
local storage, for network-traffic) to a minimum. 95% of the remote
source will never be read and 5% will be read frequently. If I read a
storage-block 1,000,000 times over 10 years, ideally it should only
traverse the network once during that time.
There is no mention of writes, so far.
Sub-goal (Y): Identify whether or not there is a device-mapper target
with copy-on-read, read-only behaviour. This seems like it could be
useful in certain forensic situations in which "accesses" ought to be
recorded. "We" (device-mapper users) have the "write half" of
"accesses" covered, already, with snapshots.
dm-cache might work out, except that it appears to've been designed
with considerations for writes, which are irrelevant to the goal(s).
[...]
dm-cache should be likely your 'favorite' since it's designed to
handle different policies - for some reason not too many policies were
ever written (maybe existing one is good enough for most to bother),
but the idea behind was to be modular and handle different need of
users - if they simply 'know more' upfront then the current
'heuristic' smq policy might ever learn from observing IO traffic
within cached device.
So I'd highly recommend to read about those policies and eventually
write a better one for you specific case (like instantly promote any
blocks until cache is 100% full)
Although you should probably try to use the existing one first -and
you might be surprised how good it will get over the time in picking
your hot spots of your disks - although there might be some 'extra'
reads from your origin device before the cache considers a block to be
'promoted' to cache.
As for your 'write' concerns - they are actually irrelevant itself -
since if there is no write - there is no need to maintain the place
for it....
[...]
OK, thanks. I guess dm-cache "writeback" could work: reads will
hopefully populate the cache and since no writes will ever happen, the
details about writes are irrelevant. That didn't seem to occur to me,
the first time I read about it:
- https://www.kernel.org/doc/Documentation/device-mapper/cache.txt
- Shao Miller
Synthetel Corporation
