Hi Wido,
Each disk is 3TB SATA (2.8TB seen) but what I got is this:
First let me gave you df -h:
/dev/sdb1 2.8T 754M 2.8T 1% /var/lib/ceph/osd/ceph-0
/dev/sdc1 2.8T 753M 2.8T 1% /var/lib/ceph/osd/ceph-2
/dev/sdd1 2.8T 752M 2.8T 1% /var/lib/ceph/osd/ceph-4
/dev/sde1 2.8T 752M 2.8T 1% /var/lib/ceph/osd/ceph-6
/dev/sdf1 2.8T 753M 2.8T 1% /var/lib/ceph/osd/ceph-8
/dev/sdg1 2.8T 752M 2.8T 1% /var/lib/ceph/osd/ceph-10
/dev/sdh1 2.8T 751M 2.8T 1% /var/lib/ceph/osd/ceph-12
/dev/sdi1 2.8T 751M 2.8T 1% /var/lib/ceph/osd/ceph-14
/dev/sdj1 2.8T 751M 2.8T 1% /var/lib/ceph/osd/ceph-16
/dev/sdk1 2.8T 751M 2.8T 1% /var/lib/ceph/osd/ceph-18
Then here is my results from ceph df commands:
ceph df
GLOBAL:
SIZE AVAIL RAW USED %RAW USED
200G 179G 21381M 10.44
POOLS:
NAME ID USED %USED MAX AVAIL OBJECTS
rbd 0 0 0 86579M 0
cephfs_data 1 0 0 86579M 0
cephfs_metadata 2 2488 0 86579M 21
ceph osd df
ID WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS
0 0.00980 1.00000 10240M 1070M 9170M 10.45 1.00 173
2 0.00980 1.00000 10240M 1069M 9170M 10.45 1.00 150
4 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 148
6 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 167
8 0.00980 1.00000 10240M 1069M 9171M 10.44 1.00 166
10 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 171
12 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 160
14 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 179
16 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 182
18 0.00980 1.00000 10240M 1069M 9170M 10.44 1.00 168
1 0.00980 1.00000 10240M 1069M 9170M 10.45 1.00 167
3 0.00980 1.00000 10240M 1069M 9170M 10.45 1.00 156
5 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 152
7 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 158
9 0.00980 1.00000 10240M 1069M 9170M 10.44 1.00 174
11 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 153
13 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 179
15 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 186
17 0.00980 1.00000 10240M 1068M 9171M 10.44 1.00 185
19 0.00980 1.00000 10240M 1067M 9172M 10.43 1.00 154
TOTAL 200G 21381M 179G 10.44
MIN/MAX VAR: 1.00/1.00 STDDEV: 0.00
-Gencer.
-----Original Message-----
From: Wido den Hollander [mailto:wido@xxxxxxxx]
Sent: Monday, July 17, 2017 4:57 PM
To: ceph-users@xxxxxxxxxxxxxx; gencer@xxxxxxxxxxxxx
Subject: Re: Ceph (Luminous) shows total_space wrong
> Op 17 juli 2017 om 15:49 schreef gencer@xxxxxxxxxxxxx:
>
>
> Hi,
>
>
>
> I successfully managed to work with ceph jewel. Want to try luminous.
>
>
>
> I also set experimental bluestore while creating osds. Problem is, I
> have 20x3TB hdd in two nodes and i would expect 55TB usable (as on
> jewel) on luminous but i see 200GB. Ceph thinks I have only 200GB
> space available in total. I see all osds are up and in.
>
>
>
> 20 osd up; 20 osd in. 0 down.
>
>
>
> Ceph -s shows HEALTH_OK. I have only one monitor and one mds. (1/1/1)
> and it is up:active.
>
>
>
> ceph osd tree gave me all OSDs in nodes are up and results are
> 1.0000... I checked via df -h but all disks ahows 2.7TB. Basically something is wrong.
> Same settings and followed schema on jewel is successful except luminous.
>
What do these commands show:
- ceph df
- ceph osd df
Might be that you are looking at the wrong numbers.
Wido
>
>
> What might it be?
>
>
>
> What do you need to know to solve this problem? Why ceph thinks I have
> 200GB space only?
>
>
>
> Thanks,
>
> Gencer.
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
