Re: F29 System Wide Change: Make BootLoaderSpec the default

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, Jun 26, 2018 at 1:47 PM, Lennart Poettering
<mzerqung@xxxxxxxxxxx> wrote:

> The logic I am working on is built as an add-on to the boot loader
> spec: I simply encode the counter in the file name of the snippet, so
> that counting down and marking an entry as good is a simple rename
> operation, which has a good chance to be implementable atomically even
> in simpler file systems (such as fat), as there should not be the need
> to allocate or release blocks.

rename() is not atomic on FAT. On VFAT in particular, it actually
stores two filenames, 8.3 and LFN - so there are multiple independent
writes that need to happen.

It's probably find most of the time. But if there's a crash, well it's
not a crash safe file system, so if your boot failure includes a
crash, either your rename() based counter, or kdump dumping crash dump
data onto /boot, might be a problem.

TFAT, KFAT, and Window's transaction logs for the BCD on the ESP are
meant to work around the limits of FAT in this regard. We're not doing
anything like that in this proposal though.



>> Let me tell you how totally non-trivial VFAT is for sharing when the
>> driver is in firmware. Digital camera vendors have had vfat drivers in
>> both consumer and professional cameras for over a decade. The one sure
>> way you can corrupt your CF/SD card file system, is transferring it
>> between cameras *even of the same make and model with different
>> firmware versions* and doing basic file operations like create and
>> delete. Boom! Fuck all your files! Hahaha! (Yes the camera maniacally
>> laughs in your face as it corrupts the file system.) The manufacturer
>> recommendation, even on professional gear? Format the card in-camera
>> before each use. Shoot. Do not ever delete files. When you're ready
>> suck the images off the card, back them up, put the card back in the
>> camera, reformat. If you switch cards to different cameras, reformat
>> the card. You can't do that? Expect data loss is possible.
>
> Dunno. We are not talking about digital cameras here. Already for
> licensing/patent reasons firmware tends to stick to the intel uefi
> fat driver. Which might bad and they might have patched around in it
> to make it worse, but I think it's certainly not as bad as you assume
> it to be.

The digital camera case should be even more reliable than our case.
They never write to high latency devices, we do. They also aren't
doing writes in the context of boot failures, including crashes.


>> I've lost count how many times I personally have experienced such data
>> loss, with all sorts of consumer and professional gear, let alone the
>> number of stories I've heard from professional photographers and from
>> camera and SD/CF card engineers.
>
> I did not assume the goal was to run fedora on a digital camera. This
> is borderline FUD...

Nope. Your argument is that FAT is somehow this simple, well
understood thing, by all kinds of UEFI firmware and the kernel. And
doing daily (Rawhide) and weekly (either updates or updates-testing)
modification of this critical file system is completely safe, no more
risky than doing it on a log based file system. I think that's an open
question.


> But given that the exclusive (or almost exclusive) write side of
> things is done from Linux (and not from the boot loader) it's under
> our control, and hence can be made as safe as we can. or to say this
> differently: it's the crappy firmware fs code that *reads* primarily,
> and our Linux code that *writes*. That's systematically different from
> the camera word, where it's the crappy device code that *writes*
> primarily and we on our Linux PC's mostly only *read* the CF cards...

Yes, true. It's more reliable in that sense. But it's less reliable in
that there could be modifications, possibly quite a lot of them, to
the $BOOT file system in the middle of a crash using a file system
known to not be crash safe.


>> > Oh, right. this approach already failed with Grub, with it's
>> > relatively large commercial support, and now you want pile on?
>>
>> Oh please stop. It's not a failed approach. Windows and macOS have
>> been doing this reliably for 20 years because, holy crap! they
>
> Well, still, the fs situation with grub is bad, see the xfs mess. This
> is not going to be any better if you pick some random fs nobody uses
> (such as udf or all the other exotic stuff that was mentioned).

I don't like it either, but at least it's a crash safe file system.
It's just not crash safe for the bootloader, and neither is VFAT.

It's hardly a mess though, it doesn't affect any Fedora default
installations. What you're proposing is a default Fedora installation.


> I understand you have some love for niche file systems, but seriously,
> a boot process is not a place where you want to try something new and
> shiny, but where you stick to the boring stuff you one can manage.

That's funny.

I haven't advocated anything. I'm merely wondering why the spec should
define something other than ext3/4 when Linux distros have been using
one of them for ~18 years.

Meanwhile booting kernels off VFAT hasn't been done for a decade by
the company who invented that file system, and you're saying you want
to go back to that.

I seriously doubt any distro will ever use VFAT for $BOOT, including
Fedora. I can guarantee you SUSE will not follow it, they boot from a
Btrfs subvolume for years now (very similar to Apple, who just started
doing this).

I think the two biggest things dragging BLS down: the insistence $BOOT
needs to be vfat (directly firmware readable, not through extension);
and the merged $BOOT and ESP. I just don't see it happening except
for, actual niche use cases.

What you're seeing with this feature proposal, is a split. The BLS
file format is going to get used, the other parts aren't.




> Also, let me stress one thing: windows and macos are in a very
> different position: they care only about their respective islands and
> they control the hardware to much higher degrees that we. I mean, for
> apple it's easy to support hfs+ in the uefi firmware, because their
> stuff only needs to run on macs. We do not have that luxury, we need
> to work with what we got. We are the ones who have to make multi-boot
> work. And thus we should really pick something that is simply, and
> understood by everybody and can bridge the gap to other OSes,
> firmwares and boot loaders.

Define everybody and then also why we care about literally everybody.
On the one  hand you've lamented the emphasis on Windows dual boot
having such an affect on the design. But are you suggesting we should
care whether Windows and macOS can read write to $BOOT? Why? Is anyone
really complaining about this?

>
> Well, as far as I know the xfs mess is not caused by a simple
> "bug". It's caused by philisophical differences, that the xfs kernel
> folks made clear that the in-kernel xfs implementation is the only one
> they care for and that other implementations, such as those in grub or
> so are not supportable. I am pretty sure the other general purpose
> file system folks aren't too far off in their thinking. FAT bypasses
> this philosophical, social problem: the FAT driver is pretty well
> supported, but as the format is set pretty much in stone and no new
> features are continously added it's relatively easy to support between
> multiple peers that all want to support it separately.

Yep.

I expect any complex file system (not FAT, UDF, ISO 9660), their
developers will say this same thing, they assume no 3rd party writers,
that all metadata on disk is intentional and written by (ultimately)
upstream code rather than a reimplementation. The only file systems
that can potentially deal with intentional subterfuge by an outside
writer are OpenZFS and Btrfs both of which support cryptocraphic
hashing for data and metadata.



>
>> > UDF? When's the last time you actually used that? I mean, I don't even
>> > have a DVD drive anymore, where I could find an UDF file system on...
>>
>> I use it regularly on large flash media because there's no file size
>> limit like vfat, it supports Unix permissions, symlinks, hardlinnks,
>> and also isn't proprietary like vfat or exfat. Since ancient times
>> it's been intended for random read/write support on flash media and
>> even hard drives, it's not a DVD only thing. It has way better cross
>> platform support than exFAT, NTFS, or HFS+ - and without the vfat
>> limitations.
>
> I must say I haven't seen an UDF file system anywhere in ages. I am
> not sure it's really as well supported and universally used as you
> suggest.

I've used the same UDF formatted stick for a couple of years now,
variably written by Linux, macOS, and Windows of various versions.

As for firmware and bootloaders, yeah it may be a small problem. GRUB
supports it, and UEFI firmware supports it but possibly only for
optical media because the spec only requires it be supported for
optical media.

>
>> > Also, it's read-only afair, hence stuff like boot counting is not
>> > going to work, it's a dead end.
>>
>> It is not a read only affair. You're confusing UDF with ISO 9660. They
>> aren't the same thing. UDF has offered random read write support for
>> hard drives before flash drives were even a thing, since its
>> inception.
>>
>> https://en.wikipedia.org/wiki/Universal_Disk_Format
>
> Hmm, looking at the wikipedia page it appears that support on Linux
> has languished. the current versions of udf (2.50, 2.60, from 2003)
> are only supported read-only? Is there an active community around udf
> on Linux?

Yes. Jan Kara <jack@xxxxxxx> is maintaining, and I see patches
regularly, and as recently as a couple months ago.

#define UDF_MAX_READ_VERSION 0x0250
#define UDF_MAX_WRITE_VERSION 0x0201
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree/fs/udf/udf_sb.h?h=v4.17.3


> Quite frankly udf appears entirely unmaintained... I tried to track
> down the location where mkudffs is maintained, but all google finds
> are dead sourceforge pages. And links like
> "https://www.reddit.com/r/linux/comments/6giwvi/why_isnt_udf_given_more_attentiondeveloped_fully/";,
> which don't precisely make me want to trust this.
>
> Again, stick to something supportable. udffs is not it. Don't make a
> barely supported fringe fs a core piece of our boot process. I mean,
> seriously!

I think you'd get more buy off of UDF for $BOOT than VFAT which has
gone exactly nowhere since the BLS was initially published, despite
the prevailing support for the file system.

And still more buy off with ext2. And still more with ext4.

VFAT for static binaries and files, fine. Get off that as soon as
possible onto something people *are actually using*

-- 
Chris Murphy
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/message/L5G5INIOSJ3CYE4LCR3AKCZX7IORHF5Q/




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux