Hello Gürkan, On Sun, Mar 2, 2025 at 4:20 PM Gürkan G <ceph@xxxxxxxxx> wrote: > > Hi again Alexander, > > Thanks for taking the time. > > > Subvolumes exist to implement a notion of managed mountable > directories with a given maximum size, as required, e.g., by > Kubernetes RWX Persistent Volumes. > > I highly doubt that the main reason was this, since (afaik) snapshots > feature predate Kubernetes. > > > However, if Ceph permitted snapshots at arbitrary points within > the volume, a malicious pod could have created a snapshot, deleted > everything (not for real, "thanks" to the snapshot), written new > files, and thus evaded the quota. > > Then the admin can opt to not use allow_new_snaps in that FS, or provide > key without "s" flag so client would be unable to create snapshots. > > I am not proficient with cpp, but looks like even MDS code has special > handling of snapshots+subvolumes, even within quota-restricted ones. > > But again, this might be an oversight and conflict of features on MDS side. > > > And yes, the documentation you mention does need to be corrected. > > Could you point for the place to report this to the documentation? I > would've designed my implementation completely different if this was not > implied. The proper place to report documentation bugs is https://pad.ceph.com/p/Report_Documentation_Bugs, linked from the top of every Ceph documentation page. > > Thanks again, > > Gürkan > > On 02/03/2025 02.10, Alexander Patrakov wrote: > > Hello Gürkan, > > > > Let me clarify and correct my answer. > > > > I incorrectly assumed that you use Kubernetes, because its CSI driver > > is, by far, the main consumer of subvolumes. Still, let me explain > > this use case, as the limitations you observe naturally follow from > > it. > > > > Subvolumes exist to implement a notion of managed mountable > > directories with a given maximum size, as required, e.g., by > > Kubernetes RWX Persistent Volumes. In Kubernetes, the CSI driver, when > > it needs to, creates a subvolume, sets a quota on it, creates a dummy > > subdirectory, and mounts it in the pod that needs the Persistent > > Volume. As such, the pod has no access to the top directory of the > > persistent volume and thus cannot increase the quota by changing the > > xattr. However, if Ceph permitted snapshots at arbitrary points within > > the volume, a malicious pod could have created a snapshot, deleted > > everything (not for real, "thanks" to the snapshot), written new > > files, and thus evaded the quota. Thus, the only point where snapshots > > are allowed is the top directory of the subvolume, where the CSI > > driver can do it. > > > > Therefore, the answer to your original question is: if you want > > client-managed snapshots, do not use subvolumes, they are the wrong > > abstraction for you. Just create plain old directories outside of the > > /volumes path and mount them on the client. > > > > And yes, the documentation you mention does need to be corrected. > > > > On Sun, Mar 2, 2025 at 5:44 AM Gürkan G <ceph@xxxxxxxxx> wrote: > >> Hi, > >> > >>> This is deliberate, as otherwise they would become a mechanism for quota evasion. > >> This.. does not make much sense. If I give the setfattr command, everything works fine. Plus the documentation says following: > >> > >> Arbitrary subtrees. Snapshots are created within any directory you choose, and cover all data in the file system under that directory. > >> Ref: https://docs.ceph.com/en/squid/dev/cephfs-snapshots/ > >> > >>> In any case, please also try asking in Kubernetes forums. On Ceph side, unfortunately, everything works as intended. > >> I am also not using Kubernetes. This is a deployment over Debian bookworm VMs. Never mentioned a pod, the client is another Debian VM. > >> _______________________________________________ > >> ceph-users mailing list -- ceph-users@xxxxxxx > >> To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > -- Alexander Patrakov _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |