Re: Issue with using the block device inside a pod.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

can you share the auth caps for your k8s client?

ceph auth get client.<K8S_CLIENT>

And maybe share the yaml files as well (redact sensitive data) so we can get a full picture.

Zitat von Kushagr Gupta <kushagrguptasps.mun@xxxxxxxxx>:

Hi Team,

Components:
Kubernetes, Ceph

Problem statement:
We are trying to integrate Ceph with kubernetes.
We are unable to utilize the block volume mode in the pod.

Description:
OS: Almalinux 8.8
Ceph version: 18.2
Kubernetes version: 1.28.2

We have deployed a single node kubernetes cluster and a 3 node ceph cluster.
We are trying to integrate kubernetes with ceph using csi-rbd plugin.
We have used the following link to do the same:
https://docs.ceph.com/en/reef/rbd/rbd-kubernetes/

We are strugling with the usage of the pvc we created in the block mode.
Kindly refer the "raw-block-pvc-4.yaml".
We created a pod using the file "raw-block-pod-4.yaml".

The pod was created successfully. The device path we used was /dev/xvda.
We are unable to use this path inside the pod. Its not gettting shown in
the lsblk of the pod and its not even shown in the df -kh of the pod.
When we try to mount /dev/xvda on /tmp but we get permission denied error.
We get a similar error when we try to make partiotion of the disk. The
partition gets created but partiotion table is not updated.
At the same time we are able to format the disk using the command mkfs.ext4
/dev/xvda.
Kindly refer the output:
"
[root@kube-cluster-1 ceph_provisioner]# k exec -it
pod-with-raw-block-volume-2 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future
version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.4# hostname
pod-with-raw-block-volume-2
sh-4.4# df -kh
Filesystem                  Size  Used Avail Use% Mounted on
overlay                      70G   14G   57G  19% /
tmpfs                        64M     0   64M   0% /dev
tmpfs                        32G     0   32G   0% /sys/fs/cgroup
/dev/mapper/almalinux-root   70G   14G   57G  19% /etc/hosts
shm                          64M     0   64M   0% /dev/shm
tmpfs                        63G   12K   63G   1% /run/secrets/
kubernetes.io/serviceaccount
tmpfs                        32G     0   32G   0% /proc/acpi
tmpfs                        32G     0   32G   0% /proc/scsi
tmpfs                        32G     0   32G   0% /sys/firmware
sh-4.4# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0    7:0    0     1G  0 loop
loop1    7:1    0   500G  0 loop
sda      8:0    0 446.6G  0 disk
|-sda1   8:1        200M  0 part
|-sda2   8:2          1M  0 part
|-sda3   8:3          1G  0 part
`-sda4   8:4    0 445.4G  0 part
sdb      8:16   0 893.3G  0 disk
rbd0   252:0    0     1G  0 disk
rbd1   252:16   0   500G  0 disk
rbd2   252:32   0   100G  0 disk
sh-4.4# ls /dev
core  fd  full  mqueue  null  ptmx  pts  random  shm  stderr  stdin  stdout
 termination-log  tty  urandom  xvda  zero
sh-4.4#
"

Altough we are able to see a volume corresponding to the PV.
But we are not able to use the block device.

Could anyone help me out.

Thanks and Regards,
Kushagra Gupta
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux