On Tue, Jun 26, 2018 at 1:47 PM, Lennart Poettering <mzerqung@xxxxxxxxxxx> wrote: > The logic I am working on is built as an add-on to the boot loader > spec: I simply encode the counter in the file name of the snippet, so > that counting down and marking an entry as good is a simple rename > operation, which has a good chance to be implementable atomically even > in simpler file systems (such as fat), as there should not be the need > to allocate or release blocks. rename() is not atomic on FAT. On VFAT in particular, it actually stores two filenames, 8.3 and LFN - so there are multiple independent writes that need to happen. It's probably find most of the time. But if there's a crash, well it's not a crash safe file system, so if your boot failure includes a crash, either your rename() based counter, or kdump dumping crash dump data onto /boot, might be a problem. TFAT, KFAT, and Window's transaction logs for the BCD on the ESP are meant to work around the limits of FAT in this regard. We're not doing anything like that in this proposal though. >> Let me tell you how totally non-trivial VFAT is for sharing when the >> driver is in firmware. Digital camera vendors have had vfat drivers in >> both consumer and professional cameras for over a decade. The one sure >> way you can corrupt your CF/SD card file system, is transferring it >> between cameras *even of the same make and model with different >> firmware versions* and doing basic file operations like create and >> delete. Boom! Fuck all your files! Hahaha! (Yes the camera maniacally >> laughs in your face as it corrupts the file system.) The manufacturer >> recommendation, even on professional gear? Format the card in-camera >> before each use. Shoot. Do not ever delete files. When you're ready >> suck the images off the card, back them up, put the card back in the >> camera, reformat. If you switch cards to different cameras, reformat >> the card. You can't do that? Expect data loss is possible. > > Dunno. We are not talking about digital cameras here. Already for > licensing/patent reasons firmware tends to stick to the intel uefi > fat driver. Which might bad and they might have patched around in it > to make it worse, but I think it's certainly not as bad as you assume > it to be. The digital camera case should be even more reliable than our case. They never write to high latency devices, we do. They also aren't doing writes in the context of boot failures, including crashes. >> I've lost count how many times I personally have experienced such data >> loss, with all sorts of consumer and professional gear, let alone the >> number of stories I've heard from professional photographers and from >> camera and SD/CF card engineers. > > I did not assume the goal was to run fedora on a digital camera. This > is borderline FUD... Nope. Your argument is that FAT is somehow this simple, well understood thing, by all kinds of UEFI firmware and the kernel. And doing daily (Rawhide) and weekly (either updates or updates-testing) modification of this critical file system is completely safe, no more risky than doing it on a log based file system. I think that's an open question. > But given that the exclusive (or almost exclusive) write side of > things is done from Linux (and not from the boot loader) it's under > our control, and hence can be made as safe as we can. or to say this > differently: it's the crappy firmware fs code that *reads* primarily, > and our Linux code that *writes*. That's systematically different from > the camera word, where it's the crappy device code that *writes* > primarily and we on our Linux PC's mostly only *read* the CF cards... Yes, true. It's more reliable in that sense. But it's less reliable in that there could be modifications, possibly quite a lot of them, to the $BOOT file system in the middle of a crash using a file system known to not be crash safe. >> > Oh, right. this approach already failed with Grub, with it's >> > relatively large commercial support, and now you want pile on? >> >> Oh please stop. It's not a failed approach. Windows and macOS have >> been doing this reliably for 20 years because, holy crap! they > > Well, still, the fs situation with grub is bad, see the xfs mess. This > is not going to be any better if you pick some random fs nobody uses > (such as udf or all the other exotic stuff that was mentioned). I don't like it either, but at least it's a crash safe file system. It's just not crash safe for the bootloader, and neither is VFAT. It's hardly a mess though, it doesn't affect any Fedora default installations. What you're proposing is a default Fedora installation. > I understand you have some love for niche file systems, but seriously, > a boot process is not a place where you want to try something new and > shiny, but where you stick to the boring stuff you one can manage. That's funny. I haven't advocated anything. I'm merely wondering why the spec should define something other than ext3/4 when Linux distros have been using one of them for ~18 years. Meanwhile booting kernels off VFAT hasn't been done for a decade by the company who invented that file system, and you're saying you want to go back to that. I seriously doubt any distro will ever use VFAT for $BOOT, including Fedora. I can guarantee you SUSE will not follow it, they boot from a Btrfs subvolume for years now (very similar to Apple, who just started doing this). I think the two biggest things dragging BLS down: the insistence $BOOT needs to be vfat (directly firmware readable, not through extension); and the merged $BOOT and ESP. I just don't see it happening except for, actual niche use cases. What you're seeing with this feature proposal, is a split. The BLS file format is going to get used, the other parts aren't. > Also, let me stress one thing: windows and macos are in a very > different position: they care only about their respective islands and > they control the hardware to much higher degrees that we. I mean, for > apple it's easy to support hfs+ in the uefi firmware, because their > stuff only needs to run on macs. We do not have that luxury, we need > to work with what we got. We are the ones who have to make multi-boot > work. And thus we should really pick something that is simply, and > understood by everybody and can bridge the gap to other OSes, > firmwares and boot loaders. Define everybody and then also why we care about literally everybody. On the one hand you've lamented the emphasis on Windows dual boot having such an affect on the design. But are you suggesting we should care whether Windows and macOS can read write to $BOOT? Why? Is anyone really complaining about this? > > Well, as far as I know the xfs mess is not caused by a simple > "bug". It's caused by philisophical differences, that the xfs kernel > folks made clear that the in-kernel xfs implementation is the only one > they care for and that other implementations, such as those in grub or > so are not supportable. I am pretty sure the other general purpose > file system folks aren't too far off in their thinking. FAT bypasses > this philosophical, social problem: the FAT driver is pretty well > supported, but as the format is set pretty much in stone and no new > features are continously added it's relatively easy to support between > multiple peers that all want to support it separately. Yep. I expect any complex file system (not FAT, UDF, ISO 9660), their developers will say this same thing, they assume no 3rd party writers, that all metadata on disk is intentional and written by (ultimately) upstream code rather than a reimplementation. The only file systems that can potentially deal with intentional subterfuge by an outside writer are OpenZFS and Btrfs both of which support cryptocraphic hashing for data and metadata. > >> > UDF? When's the last time you actually used that? I mean, I don't even >> > have a DVD drive anymore, where I could find an UDF file system on... >> >> I use it regularly on large flash media because there's no file size >> limit like vfat, it supports Unix permissions, symlinks, hardlinnks, >> and also isn't proprietary like vfat or exfat. Since ancient times >> it's been intended for random read/write support on flash media and >> even hard drives, it's not a DVD only thing. It has way better cross >> platform support than exFAT, NTFS, or HFS+ - and without the vfat >> limitations. > > I must say I haven't seen an UDF file system anywhere in ages. I am > not sure it's really as well supported and universally used as you > suggest. I've used the same UDF formatted stick for a couple of years now, variably written by Linux, macOS, and Windows of various versions. As for firmware and bootloaders, yeah it may be a small problem. GRUB supports it, and UEFI firmware supports it but possibly only for optical media because the spec only requires it be supported for optical media. > >> > Also, it's read-only afair, hence stuff like boot counting is not >> > going to work, it's a dead end. >> >> It is not a read only affair. You're confusing UDF with ISO 9660. They >> aren't the same thing. UDF has offered random read write support for >> hard drives before flash drives were even a thing, since its >> inception. >> >> https://en.wikipedia.org/wiki/Universal_Disk_Format > > Hmm, looking at the wikipedia page it appears that support on Linux > has languished. the current versions of udf (2.50, 2.60, from 2003) > are only supported read-only? Is there an active community around udf > on Linux? Yes. Jan Kara <jack@xxxxxxx> is maintaining, and I see patches regularly, and as recently as a couple months ago. #define UDF_MAX_READ_VERSION 0x0250 #define UDF_MAX_WRITE_VERSION 0x0201 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree/fs/udf/udf_sb.h?h=v4.17.3 > Quite frankly udf appears entirely unmaintained... I tried to track > down the location where mkudffs is maintained, but all google finds > are dead sourceforge pages. And links like > "https://www.reddit.com/r/linux/comments/6giwvi/why_isnt_udf_given_more_attentiondeveloped_fully/";, > which don't precisely make me want to trust this. > > Again, stick to something supportable. udffs is not it. Don't make a > barely supported fringe fs a core piece of our boot process. I mean, > seriously! I think you'd get more buy off of UDF for $BOOT than VFAT which has gone exactly nowhere since the BLS was initially published, despite the prevailing support for the file system. And still more buy off with ext2. And still more with ext4. VFAT for static binaries and files, fine. Get off that as soon as possible onto something people *are actually using* -- Chris Murphy _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/message/L5G5INIOSJ3CYE4LCR3AKCZX7IORHF5Q/
