Note with NVME drives, well any NAND FLASH, you have to know what
technology is in use when writing data to the drive.
In particular we have noticed that some of the latest, large storage
devices, use TLC (three bits per cell) based technology. Writing to TLC
cells is relatively slow.
So most NVME "drives" actually write PCIe -> RAM -> SLC first, then in
the background SLC -> TLC. An area of the NAND flash is configured/used
as SLC (1 bit per cell) which can be written at a fast speed. Then later
(or when this SLC area is full) the "drive" starts moving this to TLC
(probably the same SLC cells and now used in TLC mode).
The results of this is that you can see a fast burst for a few 100
MBytes, and then the drive slows dramatically depending on the "drive"
type, the size of it, how full it is and how the manufacturer's firmware
does this. This is fine for typical uses by for streaming large amounts
of data or data tests this can rear its head. Typically modern MLC based
drives don't see this drop off in write speed.
Terry
On 11/03/2022 12:26, Patrick O'Callaghan wrote:
On Thu, 2022-03-10 at 19:47 -0400, George N. White III wrote:
Oops. It actually has "compress=zstd:1" in the fstab line.
Apologies. That completely invalidates the numbers.
Not completely invalid, they still say something about a real-world
use
case, (I work with optical remote sensing where many images have big
blocks of "missing" data codes, e.g., clouds) but the interpretation
changes. We have been using netcdf4 files with internal
compression,
but now I'm motivated to compare without compression on btrfs for
"scratch"
files that don't move on a network.
I'm calling it invalid because the data is a stream of zeroes, i.e.
it's pretty much maximally compressible.
This might be more realistic, using /dev/urandom:
$ time dd if=/dev/urandom bs=1M count=23000 of=Big
23000+0 records in
23000+0 records out
24117248000 bytes (24 GB, 22 GiB) copied, 81.9688 s, 294 MB/s
real 1m22.106s
user 0m0.040s
sys 1m21.753s
poc
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
