Le 25/05/2024 à 09:32, Amir Goldstein a écrit :
On Thu, May 23, 2024 at 11:56 AM Edouard Gaulué <edouard@xxxxxxxxxxxx> wrote:
Thanks a lot Amir,
Here is a proposal, but consider it as a draft:
"
Changes to the underlying filesystems while part of a mounted overlay filesystem are not supported. Thought Overlayfs will try to handle those changed files in a way it may not result in a crash or deadlock, you shouldn't do it. Due to multiple reasons involving caches, attributes, and others, if the underlying filesystem is changed, the behavior of the overlay gets "undefined", so you can't trust it anymore.
Offline changes (i.e. when the overlay is not mounted) are allowed to the upper tree. But beware of remount after offline changes to the lower tree. They are almost supported if the “metacopy”, “index”, “xino” and “redirect_dir” features have not been used. If the lower tree is modified and any of these features has been used before on this overlay, the behavior can also get "undefined".
Edouard,
I am sorry to be discouraging, but I personally don't see much value
in this rephrasing
and I also don't think that the current documentation is lacking in this point.
This is my personal opinion and review is a community procedure.
If there are proponents for this rewrite let them speak up.
Amir,
I agree with you, there is no new ideas, it's just rephrasing of the
current documentation plus the information you brought in this thread.
It just makes no way of misinterpretation, reason why I initialy opened
this thread. For us, moving to overlayfs, it was important we undstand
better those "offline changes" notions. For the kernel user community, I
can't say if this rephrasing helps. That is just a CC0 proposal.
"
I came to overlayfs, because of chatGPT. It easily proposes to bind mount between upper and lower. Just say: "I want the feature of overlayfs, but for this specific directory, I want it to write on lower". The provided solution writes on the underlying filesystems (through bind), even if the result is quite predictable and almost works. Now I understand better the way overlayfs is working, I think there should be a warning in the documentation (that chatGPT or others may read next time) regarding this:
Overalyfs is not the only way to merge directories. This is out of scope.
"
Overlayfs will never write on the lower filesystems, so it will never arm them. But mind the interactions you could create outside of overlayfs using tools like bind mounts, "rsync" or even "cp" between upper filesystem (or merged) and lower ones. Those lead to changes to the underlying filesystems and should be avoided as already stated.
Sorry. This feels out of scope to me.
I think the introduction sections describe overlayfs and lower and
upper layers well enough.
Again, I agree with you. The documentation is quite clear, and once
again, this proposal is more an important reminder for us, as users,
than something describing the service.
We tried to look for a good place to get strong informations regarding
overlayfs usages, but the best we found is the kernel documentation (in
its last version). All the rest (blogs, "unix.stackexchange.com" or
"stackoverflow.com") sometime gives misinformation regarding usage.
Unfortunatly those looks to weight more than the kernel documentation in
the IA algrotihms (or neural networks) and, of course, that is not the
kernel documentation responsibility.
In a futur, where indexation wil be replace by IA, I just wonder if RFC,
documentations, man pages and others shouldn't be more verbose and IA
oriented. Of course, it's out of the scope of this thread.
"
Finally, I think it would be great to have an option to clean dirs of all previous xattrs set by overlayfs at mount time. Or a command line in the documentation to explain how to get the same. In the meanwhile, I would add:
"
Note: in those specific cases where data written to the overlay can be recreated without significant effort (like in volatile), you can always recreate an empty upperdir and workdir before remount.
"
But it doesn't handle the case of those who had bound upper and lower, and decide one day, to use the lower as an upper.
Sorry, but I am not sure if those details belong in the scope of this document,
because I don't think we would like to commit to any specific procedure of
cleaning the upper layer.
I do hear your concerns as a user, but I don't think that better documentation
alone is going to solve them.
Fully agree.
What overlayfs has always been missing is a counterpart library and user tools
to deal with those things.
There has been an attempt in the past to start overlayfs-progs [1] and later
overlayfs-tool2 project [2] to work on offline overlayfs layers.
I even contributed the "overlay deref" command [3] which partly does what
you are looking for, but it does not look like this project is
actively developed
except for a recent merge of the fsck tool from overlayfs-progs.
Really interesting! I will look at it.
Thanks,
Amir.
[1] https://github.com/hisilicon/overlayfs-progs
[2] https://github.com/kmxz/overlayfs-tools
[3] https://github.com/kmxz/overlayfs-tools/pull/11
Thanks a lot, Amir, your answers have always been helpful. The time you
can give to answer ours questions will ever be better than the
documentation.
Regards, Edouard
