> [...] enough cheap 2TB disks to get started. I'm planning on
> creating a RAID 6 array due to the age and consumer-grade
> quality of my 16 2TB disks. [...] Use case is long-term
> storage of many small files and a few large ones (family
> photos and videos, backups of other systems, working copies of
> photo, audio, and video edits, etc.)? Current usable space is
> about 10TB but my end state vision is probably upwards of
> 20TB.
As other people have remarked a single RAID is not long-term
storage, RAID redundancy is designed for *online continuity*
(that is storage systems to remain available despite media
failures), not for "backup" or "integrity" (even if LVM RAID has
got some recent additions for that).
Also RAID6 is terrible for writing small files, small files are
bad for any filesystem type, and recovery times on a large RAID6
are risky. Also it is much better BTW to use 'ar' or 'zip' etc.
('zip -0' for already compressed files) to avoid many small
files, esepcially if they are part of a read-only collection;
most GUI tools can access archive files as if directories. Some
tools have checksums or even redundancy codes for members.
But overall the idea is that if you are doing archival a single
large storage pool is a terrible idea (it is a terrible idea in
general, but especially for archival):
http://www.sabi.co.uk/blog/0805may.html?080516#080516
Probably your best options is to have a series of separate
smaller "silos" (or "data drawers" or "data shelves"), where you
fill one "silo" at a time, and when full it can become read-only
mounted, and then you fill the next and so on. An alternative is
to have two RW silos: one for archival, and one permanently RW
for home directories.
For backup you can buy some large disk drives, at least 3 (well
2 is the minimum but it is rather riskier), to use in rotation,
and partition them in the size of the "silos", and dump the
currently RW "silo"; with 'tar' or 'dumpe2fs' (or even 'dd' but
careful about duplicate UUIDs or labels), but not RSYNC if you
have many small files. The backups can be easily encrypted even
if the live "silos" are not.
As the currently RW "silo" fills up you keep backing it up in to
distinct backup disks, and once it is full you can stop backing
it up, and keep its backup disks on a shelf.
For example you could have a number of 2+1 or 3+1 or 4+1 RAID5s
as "silos", with a 4TB or 6TB or 8TB raw capacity and fill one
after the other, and then get 8-12TB drives, partitioned
accordingly. 4TB is for me is a good limit for "live" silos.
The filesystem type for the silos does not matter much, but I
like for archiving the checksumming COW filesystem types, like
NILFS2/Btrfs/ZFS (but I never use the Btrfs volume manager, I
prefer MD RAID by far, while the ZFS volume manager is
acceptable). Otherwise I like JFS (or F2FS or even UDF).
