Why not zfs? On 6/26/2020 10:42 AM, Ben Cotton wrote: > https://fedoraproject.org/wiki/Changes/BtrfsByDefault > > == Summary == > > For laptop and workstation installs of Fedora, we want to provide file > system features to users in a transparent fashion. We want to add new > features, while reducing the amount of expertise needed to deal with > situations like [https://pagure.io/fedora-workstation/issue/152 > running out of disk space.] Btrfs is well adapted to this role by > design philosophy, let's make it the default. > > == Owners == > > * Names: [[User:Chrismurphy|Chris Murphy]], [[User:Ngompa|Neal > Gompa]], [[User:Josef|Josef Bacik]], [[User:Salimma|Michel Alexandre > Salim]], [[User:Dcavalca|Davide Cavalca]], [[User:eeickmeyer|Erich > Eickmeyer]], [[User:ignatenkobrain|Igor Raits]], > [[User:Raveit65|Wolfgang Ulbrich]], [[User:Zsun|Zamir SUN]], > [[User:rdieter|Rex Dieter]], [[User:grinnz|Dan Book]], > [[User:nonamedotc|Mukundan Ragavan]] > * Emails: chrismurphy@xxxxxxxxxxxxxxxxx, ngompa13@xxxxxxxxx, > josef@xxxxxxxxxxxxxx, michel@xxxxxxxxxxxxxxx, dcavalca@xxxxxx, > erich@xxxxxxxxxxxxxxxxxx, ignatenkobrain@xxxxxxxxxxxxxxxxx, > fedora@xxxxxxxxx, zsun@xxxxxxxxxxxxxxxxx, rdieter@xxxxxxxxx, > grinnz@xxxxxxxxx, nonamedotc@xxxxxxxxx > > * Products: All desktop editions, spins, and labs > * Responsible WGs: Workstation Working Group, KDE Special Interest Group > > > == Detailed Description == > > Fedora desktop edition/spin variants will switch to using Btrfs as the > filesystem by default for new installs. Labs derived from these > variants inherit this change, and other editions may opt into this > change. > > The change is based on the installer's custom partitioning Btrfs > preset. It's been well tested for 7 years. > > '''''Current partitioning'''''<br /> > <span style="color: tomato">vg/root</span> LV mounted at <span > style="color: tomato">/</span> and a <span style="color: > tomato">vg/home</span> LV mounted at <span style="color: > tomato">/home</span>. These are separate file system volumes, with > separate free/used space. > > '''''Proposed partitioning'''''<br /> > <span style="color: tomato">root</span> subvolume mounted at <span > style="color: tomato">/</span> and <span style="color: > tomato">home</span> subvolume mounted at <span style="color: > tomato">/home</span>. Subvolumes don't have size, they act mostly like > directories, space is shared. > > '''''Unchanged'''''<br /> > <span style="color: tomato">/boot</span> will be a small ext4 volume. > A separate boot is needed to boot dm-crypt sysroot installations; it's > less complicated to keep the layout the same, regardless of whether > sysroot is encrypted. There will be no automatic snapshots/rollbacks. > > If you select to encrypt your data, LUKS (dm-crypt) will be still used > as it is today (with the small difference that Btrfs is used instead > of LVM+Ext4). There is upstream work on getting native encryption for > Btrfs that will be considered once ready and is subject of a different > change proposal in a future Fedora release. > > === Optimizations (Optional) === > > The detailed description above is the proposal. It's intended to be a > minimalist and transparent switch. It's also the same as was > [[Features/F16BtrfsDefaultFs|proposed]] (and > [https://lwn.net/Articles/446925/ accepted]) for Fedora 16. The > following optimizations improve on the proposal, but are not critical. > They are also transparent to most users. The general idea is agree to > the base proposal first, and then consider these as enhancements. > > ==== Boot on Btrfs ==== > > * Instead of a 1G ext4 boot, create a 1G Btrfs boot. > * Advantage: Makes it possible to include in a snapshot and rollback > regime. GRUB has stable support for Btrfs for 10+ years. > * Scope: Contingent on bootloader and installer team review and > approval. blivet should use <code>mkfs.btrfs --mixed</code>. > > ==== Compression ==== > > * Enable transparent compression using zstd on select directories: > <span style="color: tomato">/usr</span> <span style="color: > tomato">/var/lib/flatpak</span> <span style="color: > tomato">~/.local/share/flatpak</span> > * Advantage: Saves space and significantly increase the lifespan of > flash-based media by reducing write amplification. It may improve > performance in some instances. > * Scope: Contingent on installer team review and approval to enhance > anaconda to perform the installation using <code>mount -o > compress=zstd</code>, then set the proper XATTR for each directory. > The XATTR can't be set until after the directories are created via: > rsync, rpm, or unsquashfs based installation. > > ==== Additional subvolumes ==== > > * <span style="color: tomato">/var/log/</span> <span style="color: > tomato">/var/lib/libvirt/images</span> and <span style="color: > tomato">~/.local/share/gnome-boxes/images/</span> will use separate > subvolumes. > * Advantage: Makes it easier to excluded them from snapshots, > rollbacks, and send/receive. (Btrfs snapshotting is not recursive, it > stops at a nested subvolume.) > * Scope: Anaconda knows how to do this already, just change the > kickstart to add additional subvolumes (minus the subvolume in <span > style="color: tomato">~/</span>. GNOME Boxes will need enhancement to > detect that the user home is on Btrfs and create <span style="color: > tomato">~/.local/share/gnome-boxes/images/</span> as a subvolume. > > == Feedback == > > ==== Red Hat doesn't support Btrfs? Can Fedora do this? ==== > > Red Hat supports Fedora well, in many ways. But Fedora already works > closely with, and depends on, upstreams. And this will be one of them. > That's an important consideration for this proposal. The community has > a stake in ensuring it is supported. Red Hat will never support Btrfs > if Fedora rejects it. Fedora necessarily needs to be first, and make > the persuasive case that it solves more problems than alternatives. > Feature owners believe it does, hands down. > > The Btrfs community has users that have been using it for most of the > past decade at scale. It's been the default on openSUSE (and SUSE > Linux Enterprise) since 2014, and Facebook has been using it for all > their OS and data volumes, in their data centers, for almost as long. > Btrfs is a mature, well-understood, and battle-tested file system, > used on both desktop/container and server/cloud use-cases. We do have > developers of the Btrfs filesystem maintaining and supporting the code > in Fedora, one is a Change owner, so issues that are pinned to Btrfs > can be addressed quickly. > > ==== What about device-mapper alternatives? ==== > > dm-thin (thin provisioning): > [[https://pagure.io/fedora-workstation/issue/152 Issue #152] still > happens, because the installer won't over provision by default. It > still requires manual intervention by the user to identify and resolve > the problem. Upon growing a file system on dm-thin, the pool is over > committed, and file system sizes become a fantasy: they don't add up > to the total physical storage available. The truth of used and free > space is only known by the thin pool, and CLI and GUI programs are > unprepared for this. Integration points like rpm free space checks or > GNOME disk-space warnings would have to be adapted as well. > > dm-vdo: is not yet merged, and isn't as straightforward to selectively > enable per directory and per file, as is the case on Btrfs using > <code>chattr +c</code> on <span style="color: > tomato">/var/lib/flatpaks/</span>. > > Btrfs solves the problems that need solving, with few side effects or > pitfalls for users. It has more features we can take advantage of > immediately and transparently: compression, integrity, and IO > isolation. Many Btrfs features and optimizations can be opted into > selectively per directory or file, such as compression and nodatacow, > rather than as a layer that's either on or off. > > ==== What about UI/UX and integration in the desktop? ==== > > If Btrfs isn't the default file system, there's no commitment, nor > reason to work on any UI/UX integration. There are ideas to make > certain features discoverable: selective compression; systemd-homed > may take advantage of either Btrfs online resize, or near-term planned > native encryption, which could make it possible to live convert > non-encrypted homes to encrypted; and system snapshot and rollbacks. > > Anaconda already has sophisticated Btrfs integration. > > ==== What Btrfs features are recommended and supported? ==== > > The primary goal of this feature is to be largely transparent to the > user. It does not require or expect users to learn new commands, or to > engage in peculiar maintenance rituals. > > The full set of Btrfs features that is considered stable and enabled > by default upstream will be enabled in Fedora. Fedora is a community > project. What is supported within Fedora depends on what the community > decides to put forward in terms of resources. > > The upstream [https://btrfs.wiki.kernel.org/index.php/Status Btrfs > feature status page]. > > ==== Are subvolumes really mostly like directories? ==== > > Subvolumes behave like directories in terms of navigation in both the > GUI and CLI, e.g. <code>cp</code>, <code>mv</code>, <code>du</code>, > owner/permissions, and SELinux labels. They also share space, just > like a directory. > > But it is an incomplete answer. > > A subvolume is an independent file tree, with its own POSIX namespace, > and has its own pool of inodes. This means inode numbers repeat > themselves on a Btrfs volume. Inodes are only unique within a given > subvolume. A subvolume has its own st_dev, so if you use <code>stat > FILE</code> it reports a device value referring to the subvolume the > file is in. And it also means hard links can't be created between > subvolumes. From this perspective, subvolumes start looking more like > a separate file system. But subvolumes share most of the other trees, > so they're not truly independent file systems. They're also not block > devices. > > == Benefit to Fedora == > > Problems Btrfs helps solve: > > * Users running out of free space on either <span style="color: > tomato">/</span> or <span style="color: tomato">/home</span> > [https://pagure.io/fedora-workstation/issue/152 Workstation issue > #152] > ** "one big file system": no hard barriers like partitions or logical volumes > ** transparent compression: significantly reduces write amplification, > improves lifespan of storage hardware > ** reflinks and snapshots are more efficient for use cases like > containers (Podman supports both) > * Storage devices can be flaky, resulting in data corruption > ** Everything is checksummed and verified on every read > ** Corrupt data results in EIO (input/output error), instead of > resulting in application confusion, and isn't replicated into backups > and archives > * Poor desktop responsiveness when under pressure > [https://pagure.io/fedora-workstation/issue/154 Workstation issue > #154] > ** Currently only Btrfs has proper IO isolation capability via cgroups2 > ** Completes the resource control picture: memory, cpu, IO isolation > * File system resize > ** Online shrink and grow are fundamental to the design > * Complex storage setups are... complicated > ** Simple and comprehensive command interface. One master command > ** Simpler to boot, all code is in the kernel, no initramfs complexities > ** Simple and efficient file system replication, including incremental > backups, with <code>btrfs send</code> and <code>btrfs receive</code> > > == Scope == > * Proposal owners: > ** Submit PR's for Anaconda to change <code>default_scheme = > BTRFS</code> to the proper product files. > ** Multiple test days: build community support network > ** Aid with documentation > > * Other developers: > ** Anaconda, review PRs and merge > ** Bootloader team, review PRs and merge > ** Recommended optimization <code>chattr +C</code> set on the > containing directory for virt-manager and GNOME Boxes. > > * Release engineering: [https://pagure.io/releng/issue/9545 #9545] > > * Policies and guidelines: N/A > > * Trademark approval: N/A > > == Upgrade/compatibility impact == > > Change will not affect upgrades. > > Documentation will be provided for existing Btrfs users to "retrofit" > their setups to that of a default Btrfs installation (base plus any > approved options). > > == How To Test == > > '''''Today'''''<br /> > Do a custom partitioning installation; change the scheme drop-down > menu to Btrfs; click the blue "automatically create partitions"; and > install.<br /> > Fedora 31, 32, Rawhide, on x86_64 and ARM. > > '''''Once change lands'''''<br /> > It should be simple enough to test, just do a normal install. > > == User Experience == > > ==== Pros ==== > > * Mostly transparent > * Space savings from compression > * Longer lifespan of hardware, also from compression. > * Utilities for used and free space, CLI and GUI, are expected to > behave the same. No special commands are required. > * More detailed information can be revealed by <code>btrfs</code> > specific commands. > > ==== Enhancement opportunities ==== > > [https://bugzilla.redhat.com/show_bug.cgi?id=906591 updatedb does not > index /home when /home is a bind mount] Also can affected rpm-ostree > installations, including Silverblue. > > [https://gitlab.gnome.org/GNOME/gnome-usage/-/issues/49 GNOME Usage: > Incorrect numbers when using multiple btrfs subvolumes] This isn't > Btrfs specific, happens with "one big ext4" volume as well. > > [https://gitlab.gnome.org/GNOME/gnome-boxes/-/issues/88 GNOME Boxes, > RFE: create qcow2 with 'nocow' option when on btrfs /home] This is > Btrfs specific, and is a recommended optimization for both GNOME Boxes > and virt-manager. > > [https://github.com/containers/libpod/issues/6563 containers/libpod: > automatically use btrfs driver if on btrfs] > > == Dependencies == > > None. > > == Contingency Plan == > > * Contingency mechanism: Owner will revert changes back to LVM+ext4 > * Contingency deadline: Beta freeze > > * Blocks release? Yes > * Blocks product? Workstation and KDE > > == Documentation == > > Strictly speaking no documentation is required reading for users. But > there will be some Fedora documentation to help get the ball rolling. > > For those who want to know more: > > [https://btrfs.wiki.kernel.org/index.php/Main_Page btrfs wiki main > page and full feature list.] > > <code>man 5 btrfs</code> contains: mount options, features, swapfile > support, checksum algorithms, and more<br /> > <code>man btrfs</code> contains an overview of the btrfs subcommands<br /> > <code>man btrfs <nowiki><subcommand></nowiki></code> will show the man > page for that subcommand > > > NOTE: The btrfs command will accept partial subcommands, as long as > it's not ambiguous. These are equivalent commands:<br /> > <code>btrfs subvolume snapshot</code><br /> > <code>btrfs sub snap</code><br /> > <code>btrfs su sn</code> > > You'll discover your own convention. It might be preferable to write > out the full command on forums and lists, but then maybe some folks > don't learn about this useful shortcut? > > For those who want to know a lot more: > > [https://btrfs.wiki.kernel.org/index.php/Main_Page#Developer_documentation > Btrfs developer documentation]<br /> > [https://github.com/btrfs/btrfs-dev-docs/blob/master/trees.txt Btrfs trees] > > == Release Notes == > The default file system on the desktop is Btrfs. > > _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
