A part your whole description Our 15 years experience on hard disks data recovery, suggests to avoid the spin down and spin up of the storage's HDDs Just make sure you have a vibrations free environment, yes, that is a must Any noise in any fan , mean vibrations. Contemporary HDDs (mean manufactured after 2016-2017) , are high density and heads sliders are now at few air molecules of distance from the platters surfaces Just that Meanwhile the drives are really low power consumption, so for just 2 or 4 drives, it is better to avoid those spin cycles. We don't put standby neither our 12 and 24 bays storages. Kind regards Roberto Gini Technical Manager https://www.RecuperoDatiRAIDFAsTec.it Il giorno 19 dic 2023, 23:28, alle ore 23:28, Wolf480pl <wolf480@xxxxxxxxxx> ha scritto: >Hello, > >I have an x86-64 PC acting as a NAS, it has 2x 3TB HDDs in md(4) RAID1, >with LVM on top, and LVs for various types of data. It also has >an NVME SSD with rootfs and 16GB of RAM. I spin down the HDDs to >minimize idle power draw, and to make them spin up less often I tried >to use lvmcache for one of the LVs, with cache volume on the SSD. > >However, barely anything gets cached in the lvmcache. Blocks aren't >getting promoted (based on dmsetup status), despite the cache volume >being mostly empty. > >When a file is read for the first time, it gets cached in RAM >by page cache, but not on SSD by lvmcache. Subsequent reads never hit >the block layer, because the data is already in RAM, until I need to >reboot the machine. After a reboot, when something tries to read >that file again, HDDs will have to be spun up again, because it never >got promoted to SSD by lvmcache. > >I looked into dm-cache's smq policy code[1], and it looks like >for a block to be promoted to the cache volume, it needs to be read >at least twice: > >- first read - block gets added to the bottom of the hotspot queue >- some time passes, queue tick triggers a redistribute, some blocks > get moved to the top of the queue >- second read - block is found to be on top of the queue, gets promoted > >Unfortunately in my case the second read never comes. > >I'm working it around by setting very low memory limit in cgroups that >often read small files, but I'm curious if a better solution exists. > >Also, I'm guessing I'm not the first one discovering this - is there >any bugtracker where I can find or submit an issue like this? > >I have a few ideas on how this could be fixed in the kernel, but >I don't know if such patches would be welcome or where to reach >device-mapper developers to ask them about it. > >I'd be thankful for guidance on how to proceed with this. > >Here are more details about the setup: > > $ uname -a >Linux roy 5.10.0-20-amd64 #1 SMP Debian 5.10.158-2 (2022-12-13) x86_64 >GNU/Linux > $ head -n1 /etc/os-release > PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" > $ sudo lvs -a >LV VG Attr LSize Pool >Origin Data% Meta% Move Log Cpy%Sync Convert >archive greed Cwi-aoC--- 250,00g [archive_cache_cvol] >[archive_corig] 1,83 24,09 0,00 > [archive_cache_cvol] greed Cwi-aoC--- 50,00g > [archive_corig] greed owi-aoC--- 250,00g > [...] > $ sudo pvs > PV VG Fmt Attr PSize PFree > /dev/md126 greed lvm2 a-- <2,00t <473,00g > /dev/nvme0n1p3 greed lvm2 a-- <200,00g <150,00g > > >I have graphs showing IO metrics from /proc/diskstats >and `dmsetup status`, but I don't know if/how one should send pictures >to the list. > >This is my first time posting to the list so please let me know >if I made any mistakes. I can provide more details about my usecase, >or my ideas for fixes, but I wasn't sure if this is the right place or >how much detail is appropriate. I'll appreciate any guidance. > >Regards > >Wolf480pl > >[1]: >https://elixir.bootlin.com/linux/v5.10.200/source/drivers/md/dm-cache-policy-smq.c
