On 8/29/20 5:36 PM, Drew wrote:
I know what you and Wols are talking about and I think it's actually
two separate things. Wol's is referring to traditional read caching
where it only benefits if you are reading the same thing over and over
again, cache hits. For streaming it won't help as you'll never hit the
cache.
What you are talking about is a write cache, something I have seen
implemented before. Basically the idea is for writes to hit the SSD's
first, the SSD acting as a cache or buffer between the filesystem and
the slower RAID array. To the end process they're just writing to a
disk, they don't see the SSD buffer/cache. QNAP implements this in
their NAS chassis, just not sure what the exact implementation is in
their case.
On Fri, Aug 28, 2020 at 9:14 PM R. Ramesh <rramesh@xxxxxxxxxxx> wrote:
On 8/28/20 7:01 PM, Roger Heflin wrote:
Something I would suggest, I have found improves my mythtv experience
is: Get a big enough SSD to hold 12-18 hours of the recording or
whatever you do daily, and setup the recordings to go to the SSD. i
defined use the disk with the highest percentage free to be used
first, and since my raid6 is always 90% plus the SSD always gets used.
Then nightly I move the files from the ssd recordings directory onto
the raid6 recordings directory. This also helps when your disks start
going bad and getting badblocks, the badblocks *WILL* cause mythtv to
stop recording shows at random because of some prior choices the
developers made (sync often, and if you get more than a few seconds
behind stop recording, attempting to save some recordings).
I also put daily security camera data on the ssd and copy it over to
the raid6 device nightly.
Using the ssd for recording much reduces the load on the slower raid6
spinning disks.
You would have to have a large number of people watching at the same
time as the watching is relatively easy load, compared to the writes.
On Fri, Aug 28, 2020 at 5:42 PM Ram Ramesh <rramesh2400@xxxxxxxxx> wrote:
On 8/28/20 5:12 PM, antlists wrote:
On 28/08/2020 18:25, Ram Ramesh wrote:
I am mainly looking for IOP improvement as I want to use this RAID in
mythtv environment. So multiple threads will be active and I expect
cache to help with random access IOPs.
???
Caching will only help in a read-after-write scenario, or a
read-several-times scenario.
I'm guessing mythtv means it's a film server? Can ALL your films (or
at least your favourite "watch again and again" ones) fit in the
cache? If you watch a lot of films, chances are you'll read it from
disk (no advantage from the cache), and by the time you watch it again
it will have been evicted so you'll have to read it again.
The other time cache may be useful, is if you're recording one thing
and watching another. That way, the writes can stall in cache as you
prioritise reading.
Think about what is actually happening at the i/o level, and will
cache help?
Cheers,
Wol
Mythtv is a sever client DVR system. I have a client next to each of my
TVs and one backend with large disk (this will have RAID with cache). At
any time many clients will be accessing different programs and any
scheduled recording will also be going on in parallel. So you will see a
lot of seeks, but still all will be based on limited threads (I only
have 3 TVs and may be one other PC acting as a client) So lots of IOs,
mostly sequential, across small number of threads. I think most cache
algorithms should be able to benefit from random access to blocks in SSD.
Do you see any flaws in my argument?
Regards
Ramesh
I was hoping SSD caching would do what you are suggesting without daily
copying. Based on Wol's comments, it does not. May be I misunderstood
how SSD caching works. I will try it any way and see what happens. If
it does not do what I want, I will remove caching and go straight to disks.
Ramesh
After thinking through this, I really like the idea of simply recording
programs to SSD and move one file at a time based on some aging
algorithms of my own. I will move files back and forth as needed during
overnight hours creating my own caching effect. As long as I keep the
original (renamed) and cache the ones needed with correct name, mythtv
will find the cached copy. When mythtv complains about something
missing, I can manually look at the renamed backup copy and make the
corrections. Unless my thinking is badly broken, this should work.
I really wished overlay fs had a nice merge/clean feature that will
allow us to move overlay items to underlying file system and start over
the overlay. All I need is file level caching and not block level caching.
Ramesh
