> Op 17 juli 2017 om 16:41 schreef gencer@xxxxxxxxxxxxx: > > > Hi Wido, > > Each disk is 3TB SATA (2.8TB seen) but what I got is this: > > First let me gave you df -h: > > /dev/sdb1 2.8T 754M 2.8T 1% /var/lib/ceph/osd/ceph-0 > /dev/sdc1 2.8T 753M 2.8T 1% /var/lib/ceph/osd/ceph-2 > /dev/sdd1 2.8T 752M 2.8T 1% /var/lib/ceph/osd/ceph-4 > /dev/sde1 2.8T 752M 2.8T 1% /var/lib/ceph/osd/ceph-6 > /dev/sdf1 2.8T 753M 2.8T 1% /var/lib/ceph/osd/ceph-8 > /dev/sdg1 2.8T 752M 2.8T 1% /var/lib/ceph/osd/ceph-10 > /dev/sdh1 2.8T 751M 2.8T 1% /var/lib/ceph/osd/ceph-12 > /dev/sdi1 2.8T 751M 2.8T 1% /var/lib/ceph/osd/ceph-14 > /dev/sdj1 2.8T 751M 2.8T 1% /var/lib/ceph/osd/ceph-16 > /dev/sdk1 2.8T 751M 2.8T 1% /var/lib/ceph/osd/ceph-18 > > > Then here is my results from ceph df commands: > > ceph df > > GLOBAL: > SIZE AVAIL RAW USED %RAW USED > 200G 179G 21381M 10.44 > POOLS: > NAME ID USED %USED MAX AVAIL OBJECTS > rbd 0 0 0 86579M 0 > cephfs_data 1 0 0 86579M 0 > cephfs_metadata 2 2488 0 86579M 21 > Ok, that's odd. But I think these disks are using BlueStore since that's what Luminous defaults to. The partitions seem to be mixed up, so can you check on how you created the OSDs? Was that with ceph-disk? If so, what additional arguments did you use? Wido > ceph osd df > ID WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS > 0 0.00980 1.00000 10240M 1070M 9170M 10.45 1.00 173 > 2 0.00980 1.00000 10240M 1069M 9170M 10.45 1.00 150 > 4 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 148 > 6 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 167 > 8 0.00980 1.00000 10240M 1069M 9171M 10.44 1.00 166 > 10 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 171 > 12 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 160 > 14 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 179 > 16 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 182 > 18 0.00980 1.00000 10240M 1069M 9170M 10.44 1.00 168 > 1 0.00980 1.00000 10240M 1069M 9170M 10.45 1.00 167 > 3 0.00980 1.00000 10240M 1069M 9170M 10.45 1.00 156 > 5 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 152 > 7 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 158 > 9 0.00980 1.00000 10240M 1069M 9170M 10.44 1.00 174 > 11 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 153 > 13 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 179 > 15 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 186 > 17 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 185 > 19 0.00980 1.00000 10240M 1067M 9172M 10.43 1.00 154 > TOTAL 200G 21381M 179G 10.44 > MIN/MAX VAR: 1.00/1.00 STDDEV: 0.00 > > > -Gencer. > > -----Original Message----- > From: Wido den Hollander [mailto:wido@xxxxxxxx] > Sent: Monday, July 17, 2017 4:57 PM > To: ceph-users@xxxxxxxxxxxxxx; gencer@xxxxxxxxxxxxx > Subject: Re: Ceph (Luminous) shows total_space wrong > > > > Op 17 juli 2017 om 15:49 schreef gencer@xxxxxxxxxxxxx: > > > > > > Hi, > > > > > > > > I successfully managed to work with ceph jewel. Want to try luminous. > > > > > > > > I also set experimental bluestore while creating osds. Problem is, I > > have 20x3TB hdd in two nodes and i would expect 55TB usable (as on > > jewel) on luminous but i see 200GB. Ceph thinks I have only 200GB > > space available in total. I see all osds are up and in. > > > > > > > > 20 osd up; 20 osd in. 0 down. > > > > > > > > Ceph -s shows HEALTH_OK. I have only one monitor and one mds. (1/1/1) > > and it is up:active. > > > > > > > > ceph osd tree gave me all OSDs in nodes are up and results are > > 1.0000... I checked via df -h but all disks ahows 2.7TB. Basically something is wrong. > > Same settings and followed schema on jewel is successful except luminous. > > > > What do these commands show: > > - ceph df > - ceph osd df > > Might be that you are looking at the wrong numbers. > > Wido > > > > > > > What might it be? > > > > > > > > What do you need to know to solve this problem? Why ceph thinks I have > > 200GB space only? > > > > > > > > Thanks, > > > > Gencer. > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |