On Fri, May 28, 2021 at 7:04 AM Colin Walters <walters@xxxxxxxxxx> wrote: > > On this topic though, if Fedora CoreOS didn't exist, this proposal to change Cloud would be significantly more consequential. The defaults *really matter* here in particular, even more than Workstation. But, I think because CoreOS does exist, this change matters less. Well it's sorta ancient history now, but CoreOS started out on Btrfs by default because of the feature set, including the early native btrfs driver support taking advantage of cheap snapshots. CoreOS moved off of Btrfs because of ENOSPC bugs, and performance impact of the various work arounds (all predating ticketed enospc infrastructure). "Source of truth" related features that I think could be relevant for cloud use cases: * (immutable) read-only subvolumes, root can't write to them * (immutable) read-only Btrfs seed device images, truly resettable just by dropping the writable device(s) * checksums for metadata and data: crc32c, xxhash, blake2, sha256 * currently in-development: fsverity support * currently in-development: fscrypt support * currently in-development: authenticated (keyed) checksumming support And generally usable, whether image used for provisioning, or installed sysroot, or workload data: native compression. And out of band deduplication. > > One big advantage CoreOS has is Ignition, which allows provisioning filesystems in the initramfs, including the root partition. It works today to boot up a stock Fedora CoreOS AMI, OpenStack qcow2 etc. and provide an Ignition config that changes it: > https://docs.fedoraproject.org/en-US/fedora-coreos/storage/#_reconfiguring_the_root_filesystem > And it works to use btrfs too! Just s/ext4/btrfs/ in the first example. (And that *exact same* Ignition config also works for bare metal) > > That's not true of cloud-init based systems - instead of e.g. you wanted to use xfs/ext4 for a high performance database-like workload, in cloud you'd need to attach a separate instance volume or so for `/var/lib/$whatever` (That said separate volumes can actually be a better approach anyways, it depends). Some traditional Cloud image users won't notice this or care (just like Workstation users) but others definitely will. Not all databases are cow-unfriendly. RocksDB and sqlite with WAL enabled do fine on Btrfs, with datacow. There's also been a bunch of database+fsync related performance enhancements in ever kernel release for the past year since Btrfs because the default file system on the desktop. Last I checked (a year ago) postgreSQL did have some performance issues on Btrfs, but I don't know the significance, in particular with today's kernel. Interested parties could create a micro-SIG, I'm happy to help coordinate, and do some testing and document best practices. -- Chris Murphy _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
