>> Use case is long-term storage of many small files and a few large ones >> (family photos and videos, backups of other systems, working copies of >> photo, audio, and video edits, etc.)? Current usable space is about >> 10TB but my end state vision is probably upwards of 20TB. I'll >> probably consign the slowest working disks in the server to an archive >> filesystem, either RAID 1 or RAID 5, for stuff I care less about and >> backups; the archive part can be ignored for the purposes of this >> exercise. >> >> My question is: what filesystem type would be best practice for my use >> case and size requirements on the big array? (I have reviewed >> https://raid.wiki.kernel.org/index.php/RAID_and_filesystems, but am >> looking for practitioners' recommendations.) I've run ext4 >> exclusively on my arrays to date, but have been reading up on xfs; is >> there another filesystem type I should consider? Finally, are there >> any pitfalls I should know about in my high-level design? > > Whichever filesystem you choose, you will end up with a huge single point of > failure, and any trouble with that FS or the underlying array put all your > data instantly at risk. "But RAID6" -- what about a SATA controller failure, > or a flaky cabling/PSU/backplane, which disconnects, say, 4 disks at once "on > the fly". What about a sudden power loss amidst heavy write load. And so on. If that happens, you just connect the drives to another controller and reassemble. If you are really, really unlucky and lose more than two of the drives in a RAID-6, you restore from backup. > First of all, ask yourself -- is all of this backed up? If no, then go and buy > more drives until the answer is yes. With current drive prices, or as you say, > with having lots of spare old drives lying around, there's no excuse to leave > anything non-trivial not backed up. > > Secondly -- if all of this... is BACKED UP ANYWAY, why even run RAID? And with > RAID6, even waste 2 more drives for redundancy. Do you need 24x7 uptime of your > home NAS, do you have hotswap cages, do you require that the server absolutely > stays online while a disk is being replaced. Simply because if you lose 10-20TiB of data on a disk failure, you also lose a week or two with troubleshooting instead of just replacing that disk. > (blablbla) > > For the FS considerations, the dealbreaker of XFS for me is its inability to > be shrunk. And how many times have you had the need to shrink a large filesystem used for the mentioned purposes? To me that's zero. It's practical, yes, but the adverse effect of ext4 and large filesystems are worse. But hey - go on with your "good" advice. I just stick to my own so far. - Remember that RAID is not backup, it's redundancy. It may fail any day at any time, but normally just one drive fails, sometimes two. With sufficient redundancy, you just swap those drives out. - Redundancy has a cost, but time also has a cost, even at home. If you need to spend hours or days to restore a system, the time you spent could be used for family or friends. And so on… Vennlig hilsen roy -- Roy Sigurd Karlsbakk (+47) 98013356 http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- Hið góða skaltu í stein höggva, hið illa í snjó rita.
