Hi all,
I have create a pod using rbd image as backend storage, then map rbd image
to local block device, and mount it with ext4 filesystem. The `df`
displays the disk usage much larger than the available space displayed
after disabling ext4 journal. The following is the steps to reproduce,
thanks in advance.
Environment details
- Image/version of Ceph CSI driver : cephcsi:v3.5.1
- Kernel version :
🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) uname -a
Linux k1 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020
x86_64 x86_64 x86_64 GNU/Linux
- Mounter used for mounting PVC (for cephFS its fuse or kernel. for rbd
its
krbd or rbd-nbd) : krbd
- Kubernetes cluster version :
🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) kubectl version
Client Version: version.Info{Major:"1", Minor:"22",
GitVersion:"v1.22.7",
GitCommit:"b56e432f2191419647a6a13b9f5867801850f969",
GitTreeState:"clean", BuildDate:"2022-02-16T11:50:27Z",
GoVersion:"go1.16.14", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22",
GitVersion:"v1.22.7",
GitCommit:"b56e432f2191419647a6a13b9f5867801850f969",
GitTreeState:"clean", BuildDate:"2022-02-16T11:43:55Z",
GoVersion:"go1.16.14", Compiler:"gc", Platform:"linux/amd64"}
- Ceph cluster version :
🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel)ceph --version
ceph version 15.2.16 (d46a73d6d0a67a79558054a3a5a72cb561724974) octopus (stable)
Steps to reproduce
Steps to reproduce the behavior:
1. Create a storageclass with storageclass
<https://github.com/ceph/ceph-csi/blob/devel/examples/rbd/storageclass.yaml>
2. Then create pvc, and test pod like below
➜ /root ☞ cat csi-rbd/examples/pvc.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: rbd-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
storageClassName: csi-rbd-sc
🍺 /root ☞cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: csi-rbd-demo-pod
spec:
containers:
- name: web-server
image: docker.io/library/nginx:latest
volumeMounts:
- name: mypvc
mountPath: /var/lib/www/html
volumes:
- name: mypvc
persistentVolumeClaim:
claimName: rbd-pvc
readOnly: false
1. The following steps are executed in ceph cluster
🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) rbd ls -p
pool-51312494-44b2-43bc-8ba1-9c4f5eda3287
csi-vol-ad0bba2a-49fc-11ed-8ab9-3a534777138b
🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) rbd map
pool-51312494-44b2-43bc-8ba1-9c4f5eda3287/csi-vol-ad0bba2a-49fc-11ed-8ab9-3a534777138b
/dev/rbd0🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) lsblk -f
NAME FSTYPE LABEL UUID
MOUNTPOINT
sr0
vda
├─vda1 xfs a080444c-7927-49f7-b94f-e20f823bbc95 /boot
├─vda2 LVM2_member jDjk4o-AaZU-He1S-8t56-4YEY-ujTp-ozFrK5
│ ├─centos-root xfs 5e322b94-4141-4a15-ae29-4136ae9c2e15 /
│ └─centos-swap swap d59f7992-9027-407a-84b3-ec69c3dadd4e
└─vda3 LVM2_member Qn0c4t-Sf93-oIDr-e57o-XQ73-DsyG-pGI8X0
└─centos-root xfs 5e322b94-4141-4a15-ae29-4136ae9c2e15 /
vdb
vdc
rbd0 ext4 e381fa9f-9f94-43d1-8f3a-c2d90bc8de27
🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) mount /dev/rbd0
/mnt/ext4🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) df -hT | egrep
'rbd|Type'
Filesystem Type Size Used Avail Use% Mounted on
/dev/rbd0 ext4 49G 53M 49G 1% /mnt/ext4🍺
/root/go/src/ceph/ceph-csi ☞ git:(devel) umount /mnt/ext4🍺
/root/go/src/ceph/ceph-csi ☞ git:(devel) tune2fs -o
journal_data_writeback /dev/rbd0
tune2fs 1.46.5 (30-Dec-2021)🍺 /root/go/src/ceph/ceph-csi ☞
git:(devel) tune2fs -O "^has_journal" /dev/rbd0 * <= disable
ext4 journal*
tune2fs 1.46.5 (30-Dec-2021)🍺 /root/go/src/ceph/ceph-csi ☞
git:(devel) e2fsck -f /dev/rbd0
e2fsck 1.46.5 (30-Dec-2021)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/rbd0: 11/3276800 files (0.0% non-contiguous), 219022/13107200
blocks🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) mount /dev/rbd0
/mnt/ext4🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel) df -hT | egrep
'rbd|Type'
Filesystem Type Size Used Avail Use% Mounted on
/dev/rbd0 ext4 64Z 64Z 50G 100% /mnt/ext4
🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel)mount | grep rbd
/dev/rbd0 on /mnt/ext4 type ext4 (rw,relatime,stripe=1024)
Actual results
The disk usage much larger than the available space displayed by the df command
after disabling ext4 journal
🍺 /root/go/src/ceph/ceph-csi ☞ git:(devel)df -T | egrep 'rbd|Type'
Filesystem Type 1K-blocks
Used Available Use% Mounted on
/dev/rbd0 ext4 73786976277711028224
73786976277659475512 51536328 100% /mnt/ext4
Expected behavior
The df command could show disk usage normally.
Best Regards,
Liang Zheng
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
