Re: [ERR] OSD_FULL: 1 full osd(s) - with 73% used

[Eugen Block <eblock@xxxxxx>]

Without having an answer to the question why the OSD is full I'm  
wondering why the OSD has a crush weight of 1.29999 while its size is  
only 1 TB. Was that changed on purpose? I'm not sure if that would  
explain the OSD full message, though.


Zitat von Rodrigo Werle <rodrigo.werle@xxxxxxxxx>:

> Hi everyone!
> I'm trying to understand why Ceph changed the state of one osd as full if
> it is 73% used and full_ratio is 0.97.
>
> Follow some information:
>
> # ceph health detail
> (...)
> [ERR] OSD_FULL: 1 full osd(s)
>     osd.11 is full
> (...)
>
> # ceph osd metadata
> (...)
>
> > {
> >         "id": 11,
> >         "arch": "x86_64",
> >         "back_iface": "",
> >         "bluefs": "1",
> >         "bluefs_dedicated_db": "0",
> >         "bluefs_dedicated_wal": "0",
> >         "bluefs_single_shared_device": "1",
> >         "bluestore_bdev_access_mode": "blk",
> >         "bluestore_bdev_block_size": "4096",
> >         "bluestore_bdev_dev_node": "/dev/dm-41",
> >         "bluestore_bdev_devices": "nvme0n1",
> >         "bluestore_bdev_driver": "KernelDevice",
> >         "bluestore_bdev_partition_path": "/dev/dm-41",
> >         "bluestore_bdev_rotational": "0",
> >         "bluestore_bdev_size": "1000200994816",
> >         "bluestore_bdev_support_discard": "1",
> >         "bluestore_bdev_type": "ssd",
> >         "ceph_release": "octopus",
> >         "ceph_version": "ceph version 15.2.16
> > (d46a73d6d0a67a79558054a3a5a72cb561724974) octopus (stable)",
> >         "ceph_version_short": "15.2.16",
> >         "cpu": "Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz",
> >         "default_device_class": "ssd",
> >         "device_ids":
> > "nvme0n1=Samsung_SSD_970_EVO_Plus_1TB_S59ANJ0N308050F",
> >         "device_paths":
> > "nvme0n1=/dev/disk/by-path/pci-0000:01:00.0-nvme-1",
> >         "devices": "nvme0n1",
> >         "distro": "ubuntu",
> >         "distro_description": "Ubuntu 18.04.6 LTS",
> >         "distro_version": "18.04",
> >         "front_iface": "",
> >         "hostname": "pf-us1-dfs6",
> >         "journal_rotational": "0",
> >         "kernel_description": "#182-Ubuntu SMP Fri Mar 18 15:53:46 UTC
> > 2022",
> >         "kernel_version": "4.15.0-173-generic",
> >         "mem_swap_kb": "117440508",
> >         "mem_total_kb": "181456152",
> >         "network_numa_unknown_ifaces": "back_iface,front_iface",
> >         "objectstore_numa_node": "0",
> >         "objectstore_numa_nodes": "0",
> >         "os": "Linux",
> >         "osd_data": "/var/lib/ceph/osd/ceph-11",
> >         "osd_objectstore": "bluestore",
> >         "osdspec_affinity": "",
> >         "rotational": "0"
> >     } (...)
>
>
> # cat /var/lib/ceph/osd/ceph-11/type
>
> > bluestore
>
>
> # ls -l /var/lib/ceph/osd/ceph-11 | grep block
>
> > lrwxrwxrwx 1 ceph ceph  50 Mar 22 21:44 block ->
> > /dev/mapper/YpB2cx-HlyU-VPqT-Abaz-Dutx-iMrz-Tty6o1
>
>
> # lsblk
>
> > nvme0n1                                                      259:0    0
> > 931.5G  0 disk
> > └─ceph--7ef8f83a--d055--4a59--8d6b--c564544c5a55-osd--block--c7c03dfd--a1b1--4182--9e61--bf264f293f2b
> >                                                            253:0    0
> > 931.5G  0 lvm
> >   └─YpB2cx-HlyU-VPqT-Abaz-Dutx-iMrz-Tty6o1                   253:41   0
> > 931.5G  0 crypt
>
>
> # ceph osd dump | grep full_ratio
>
> > full_ratio 0.97
> > backfillfull_ratio 0.95
> > nearfull_ratio 0.9
>
>
> # ceph osd df | head
> > ID   CLASS  WEIGHT    REWEIGHT  SIZE     RAW USE  DATA     OMAP      
> >  META     AVAIL     %USE   VAR   PGS  STATUS
> >  11   nvme   1.29999   1.00000  932 GiB   694 GiB  319 GiB   301  
> > GiB   74 GiB  238 GiB  74.49  1.09   96      up
>
>
> Any ideas?
> Thanks!
> --
>
> Rodrigo Werle
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx



_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx