Hi Maged,
Thanks for taking the time to go into a detailed explanation. It's
certainly not as easy as working out the appropriate object to get via
rados. As you suggest, I'll have to look into ceph-objectstore-tool and
perhaps librados to get any further.
Thanks again,
Chris
On Mon, Oct 24, 2022 at 02:41:07PM +0200, Maged Mokhtar wrote:
On 18/10/2022 01:24, Chris Dunlop wrote:
Hi,
Is there anywhere that describes exactly how rbd data (including
snapshots) are stored within a pool?
Hi Chris,
snaphots are stored on the same OSD as current object.
rbd snapshots are self managed rather than rados pool managed, the rbd
client tales responsibility for passing the correct snapshot
environment/context to the OSDs in i/o operations via librados.
First to create a snapshot, rbd client requests a unique snap id number
from the mons.
This number and snap name are persisted/stored in rbd_header.xx object
for the rbd image. it is added to a list of prev snaps if any.
When rbd client writes to rbd_data.xx rados object, it passes the list
of snaps.
The OSD will look at the snap list and perform a lot of logic like
create and copy original data to the snap before writing in case it did
not have this snao before or copy data if snap did not have this
offset/extent before...etc. The OSD will keep track of what snapshots it
is storing for the object, their blob offset/extent in the object as
well as their physical location on the OSD block device all in the
rocksdb database. The physical location on block device can be far apart
allocated by the allocator from free space on device. You can use
ceph-objectstore-tool on the OSD to examine snapshot location and get
its data.
When reading, the rbd client passes the snap id to read, or default id
for head/current.
I do not believe you can use rados get command on the rbd_data.xx as you
were doing to get snap shot data, even if you specify the snapshot
parameter to the command as i think this works with rados pool snapshots
and not self managed.
As a user, if you wanted to access rbd snap data, you can rbd map the
snap and read from it via kernel rbd.
If you want to fiddle with reading snapshots at rados level on the
rbd_data.xx, you can write a librados app that first read the snap id
from rbd_header.xx based on snap name then pass this id in the context
to the librados read function.
/Maged
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
