Re: rm: cannot remove dir and files (cephfs)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



>> rm: cannot remove '/design/4695/8/6-50kb.jpg': No space left on device

“No space left on device” issue typically in ceph FS might be caused if you have files > 1million(100000) in “single directory”. To mitigate this try increasing the "mds_bal_fragment_size_max" to a higher value, example 7 million (700000)

 

[mds]

mds_bal_fragment_size_max = 700000 

 

I am not going detail here, that said, there are many other tuning params including enabling dir frag, multiple MDS etc. It seems your CEPH version is 10.2.10(Jewel). Luminous(12.2.x) support better with multiple MDS and dir_frag etc. Jewel some of these options might be experimental features you might need todo bit reading related to cephFS.  Just a note of advise based on my experience, cephFS is ideal for large files in MB/GB is not great for small files(in kbs). Also split your files into multiple sub-dirs to avoid the similar issue or any likewise.

 

--

Deepak

 

 

 

From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of ??????
Sent: Friday, February 09, 2018 2:35 AM
To: ceph-users@xxxxxxxxxxxxxx
Subject: [ceph-users] rm: cannot remove dir and files (cephfs)

 

ceph version 10.2.10

ceph -s
cluster 97f833aa-cc6f-41d5-bf82-bda5c09fd664
health HEALTH_OK
monmap e3: 3 mons at {web23=192.168.65.24:6789/0,web25=192.168.65.55:6789/0,web26=192.168.65.56:6789/0}
election epoch 1198, quorum 0,1,2 web23,web25,web26
fsmap e464: 1/1/1 up {0=web26=up:active}, 3 up:standby
osdmap e325: 4 osds: 4 up, 4 in
flags sortbitwise,require_jewel_osds
pgmap v42475: 128 pgs, 3 pools, 274 GB data, 1710 kobjects
854 GB used, 2939 GB / 3793 GB avail
128 active+clean
client io 181 kB/s wr, 0 op/s rd, 5 op/s wr

ceph df
GLOBAL:
SIZE AVAIL RAW USED %RAW USED
3793G 2941G 851G 22.46
POOLS:
NAME ID USED %USED MAX AVAIL OBJECTS
rbd 0 0 0 822G 0
cephfs_metadata 1 17968k 0 822G 101458
cephfs_data 2 273G 24.99 822G 1643059

kernel mount cephfs
grep design /etc/fstab
192.168.65.24:,192.168.65.55:,192.168.65.56:/ /design ceph rw,relatime,name=design,secret=.......,_netdev,noatime 0 0

I can not delete same files and dir.
rm -rf /design/4*
rm: cannot remove '/design/4695/8/6-50kb.jpg': No space left on device
rm: cannot remove '/design/4695/8/9-300kb.png': No space left on device
rm: cannot remove '/design/4695/8/0-300kb.png': No space left on device

What ideas?
help

 

 


This email message is for the sole use of the intended recipient(s) and may contain confidential information.  Any unauthorized review, use, disclosure or distribution is prohibited.  If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux