Re: PG numbers don't add up?

Ovidiu Poncea <ovidiu.poncea@xxxxxxxxxxxxx> · Fri, 16 Mar 2018 08:27:45 +0200



    Hi Nathan,

      
      Do you have replication enabled in any of your pools? E.g. a replication
      of 2 will allocate each PG to two OSDs so that PGs/OSD is twice
      the one you expect.

    
    On 03/14/2018 07:28 AM, Nathan Dehnel
      wrote:

    
      I try to add a data pool:
        

          OSD_STAT USED   AVAIL TOTAL HB_PEERS            PG_SUM
            PRIMARY_PG_SUM 
          9         1076M  930G  931G [0,1,2,3,4,5,6,7,8]    128   
                      5 
          8         1076M  930G  931G [0,1,2,3,4,5,6,7,9]    128   
                     14 
          7         1076M  930G  931G [0,1,2,3,4,5,6,8,9]    128   
                     14 
          6         1076M  930G  931G [0,1,2,3,4,5,7,8,9]    128   
                     19 
          5         1076M  930G  931G [0,1,2,3,4,6,7,8,9]    128   
                     15 
          4         1076M  930G  931G [0,1,2,3,5,6,7,8,9]    128   
                     17 
          0         1076M  930G  931G [1,2,3,4,5,6,7,8,9]    128   
                     16 
          1         1076M  930G  931G [0,2,3,4,5,6,7,8,9]    128   
                      8 
          2         1076M  930G  931G [0,1,3,4,5,6,7,8,9]    128   
                      8 
          3         1076M  930G  931G [0,1,2,4,5,6,7,8,9]    128   
                     12 
          sum      10765M 9304G 9315G     
          

          I try to add a metadata pool:
          

            sum 0 0 0 0 0 0 0 0 
            OSD_STAT USED   AVAIL TOTAL HB_PEERS            PG_SUM
              PRIMARY_PG_SUM 
            9         1076M  930G  931G [0,1,2,3,4,5,6,7,8]     73 
                         73 
            8         1076M  930G  931G [0,1,2,3,4,5,6,7,9]     40 
                         40 
            7         1076M  930G  931G [0,1,2,3,4,5,6,8,9]     56 
                         56 
            6         1076M  930G  931G [0,1,2,3,4,5,7,8,9]     42 
                         42 
            5         1076M  930G  931G [0,1,2,3,4,6,7,8,9]     54 
                         54 
            4         1076M  930G  931G [0,1,2,3,5,6,7,8,9]     59 
                         59 
            0         1076M  930G  931G [1,2,3,4,5,6,7,8,9]     38 
                         38 
            1         1076M  930G  931G [0,2,3,4,5,6,7,8,9]     57 
                         57 
            2         1076M  930G  931G [0,1,3,4,5,6,7,8,9]     45 
                         45 
            3         1076M  930G  931G [0,1,2,4,5,6,7,8,9]     48 
                         48 
            sum      10766M 9304G 9315G   
          
          
          I try to add both pools:
        
        
          Error ERANGE:  pg_num 128 size 10 would mean 2816 total
            pgs, which exceeds max 2000 (mon_max_pg_per_osd 200 *
            num_in_osds 10)
        
        
        That's over a thousand more PGs than both pools combined.
          Where are they coming from?
      
      
      _______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

    
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com