On Mon, Aug 24, 2020 at 05:30:18PM +0200, Christian Schoenebeck wrote:
> Ok, maybe I should make this more clear with another example: one major use
> case for forks/ADS is extending (e.g. proprietary) binary file formats with
> new features. Say company B is developing an editor application that supports
> working directly with a binary media file (format) of another company A. And
> say that company B's application has some feature that don't exist in app of
> company A.
But that's going to happen today (company B's feature silently getting
dropped) when using data forks/ADS if the file is sent via zip,
http/https, compressed using gzip, xz, bzip2, etc. I remember that
world when I had to deal with with MacOS files decades ago, and it was
a total mess.
> > Keep in mind that you are not going to get universal support for ADS
> > any time soon as most filesystems will require on-disk format
> > changes to support them. Further, you are goign to have to wait for
> > the entire OS ecosystem to grow support for ADS (e.g. cp, tar,
> > rsync, file, etc) before you can actually use it sanely in
> > production systems. Even if we implement kernel support right now,
> > it will be years before it will be widely available and supported at
> > an OS/distro level...
>
> Sure, that's a chicken egg problem.
>
> Being realistic, I don't expect that forks are something that would be landing
> in Linux very soon. I think it is an effort that will take its time, probably
> as a Linux-test-fork / PoC for quite a while, up to a point where a common
> acceptance is reached.
We're talking *decades*. It's not enough for new protocol specs for
https, rsync, nfs, etc., to be modified, and then implemented. It's
not enough for file formats for zip, xz, gzip, etc., to be created;
all of this new software has to be deployed throughout the entire
ecosystem. People don't upgrade server software quickly; look up long
IPv6 has taken to be adopted!
In that amount of time, it's going to be easier to implement a more
modular application container format which allows for new features to
be added into a file --- for example, such as ISO/IEC 26300....
> But file forks already exist on other systems for multiple good reasons. So I
> think it makes sense to thrive the effort on Linux as well.
They aren't actually used all that often with Windows/Windows Office.
That's why you can upload/upload a docx file via https, or check it
into git, etc. without it being broken. (Trying doing that with an
old-style MacOS file with resource forks; what a nightmare....)
The only place where you really see use of forks/ADS is in places
where interoperability isn't a big deal, such as MacOS executables, or
back when a certain company with monopolistic tendencies was trying to
lock desktop users into their OS....
On Mon, Aug 24, 2020 at 09:26:56PM +0000, Frank van der Linden wrote:
> I agree with him and Linus that the Solaris interface of:
>
> ffd = open("foo", O_RDONLY);
> afd = openat(ffd, "attrpath", O_XATTR|O_RDWR);
>
> ..is the best starting point. It's simple, it's clean, it doesn't overload
> path separators. And hey, if you like doing it with path separators, put
> a library function on top of it that uses them :-)
The Solaris interface is pretty clean, but there if we really want to
do this (and from above, I'm not a fan), there is one thing that I
would drop from the Solaris API, and that's the ability to use chdirat
to cd into a directory which is inside a file's ADS. There were
malware authors who were using this to go to town, since most shells
didn't know about ADS, and so it was a *great* way to hide setuid root
binaries, files that were being prepared for exfiltration from the
corporation's intranet, etc.
So let's learn from Solaris's mistake, and let's not. Solaris may
have that feature, for Windows compatibility, but I'm not aware of any
enterprise Unix software that has used it, for all of the reasons
discussed above.
- Ted
