Hi,
can you share the auth caps for your k8s client?
ceph auth get client.<K8S_CLIENT>
And maybe share the yaml files as well (redact sensitive data) so we
can get a full picture.
Zitat von Kushagr Gupta <kushagrguptasps.mun@xxxxxxxxx>:
Hi Team,
Components:
Kubernetes, Ceph
Problem statement:
We are trying to integrate Ceph with kubernetes.
We are unable to utilize the block volume mode in the pod.
Description:
OS: Almalinux 8.8
Ceph version: 18.2
Kubernetes version: 1.28.2
We have deployed a single node kubernetes cluster and a 3 node ceph cluster.
We are trying to integrate kubernetes with ceph using csi-rbd plugin.
We have used the following link to do the same:
https://docs.ceph.com/en/reef/rbd/rbd-kubernetes/
We are strugling with the usage of the pvc we created in the block mode.
Kindly refer the "raw-block-pvc-4.yaml".
We created a pod using the file "raw-block-pod-4.yaml".
The pod was created successfully. The device path we used was /dev/xvda.
We are unable to use this path inside the pod. Its not gettting shown in
the lsblk of the pod and its not even shown in the df -kh of the pod.
When we try to mount /dev/xvda on /tmp but we get permission denied error.
We get a similar error when we try to make partiotion of the disk. The
partition gets created but partiotion table is not updated.
At the same time we are able to format the disk using the command mkfs.ext4
/dev/xvda.
Kindly refer the output:
"
[root@kube-cluster-1 ceph_provisioner]# k exec -it
pod-with-raw-block-volume-2 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future
version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.4# hostname
pod-with-raw-block-volume-2
sh-4.4# df -kh
Filesystem Size Used Avail Use% Mounted on
overlay 70G 14G 57G 19% /
tmpfs 64M 0 64M 0% /dev
tmpfs 32G 0 32G 0% /sys/fs/cgroup
/dev/mapper/almalinux-root 70G 14G 57G 19% /etc/hosts
shm 64M 0 64M 0% /dev/shm
tmpfs 63G 12K 63G 1% /run/secrets/
kubernetes.io/serviceaccount
tmpfs 32G 0 32G 0% /proc/acpi
tmpfs 32G 0 32G 0% /proc/scsi
tmpfs 32G 0 32G 0% /sys/firmware
sh-4.4# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 1G 0 loop
loop1 7:1 0 500G 0 loop
sda 8:0 0 446.6G 0 disk
|-sda1 8:1 200M 0 part
|-sda2 8:2 1M 0 part
|-sda3 8:3 1G 0 part
`-sda4 8:4 0 445.4G 0 part
sdb 8:16 0 893.3G 0 disk
rbd0 252:0 0 1G 0 disk
rbd1 252:16 0 500G 0 disk
rbd2 252:32 0 100G 0 disk
sh-4.4# ls /dev
core fd full mqueue null ptmx pts random shm stderr stdin stdout
termination-log tty urandom xvda zero
sh-4.4#
"
Altough we are able to see a volume corresponding to the PV.
But we are not able to use the block device.
Could anyone help me out.
Thanks and Regards,
Kushagra Gupta
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
