Adding new CEPH monitor keep SYNCHRONIZING

Ali Hussein <ali.alkhazraji@xxxxxxxxxxxxxxxxx> · Mon, 18 May 2015 11:53:32 +0300



    Hi all

    
    I have two ceph monitors working fine , i have added them a while
    ago, for now i have added a new Ceph Monitor and it does showing me
    the following log file

    
    2015-05-18 10:54:42.585123 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28280 MB, avail 22894 MB

    2015-05-18 10:55:44.418861 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28437 MB, avail 22737 MB

    2015-05-18 10:56:45.442884 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28287 MB, avail 22887 MB

    2015-05-18 10:57:47.710088 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28449 MB, avail 22725 MB

    2015-05-18 10:59:13.436988 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28266 MB, avail 22908 MB

    2015-05-18 11:00:15.069245 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28511 MB, avail 22663 MB

    2015-05-18 11:01:46.333054 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28285 MB, avail 22889 MB

    2015-05-18 11:02:48.268613 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28521 MB, avail 22653 MB

    2015-05-18 11:04:21.107442 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28287 MB, avail 22887 MB

    2015-05-18 11:05:24.336678 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28552 MB, avail 22622 MB

    2015-05-18 11:07:02.355146 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28266 MB, avail 22908 MB

    2015-05-18 11:08:04.168761 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28527 MB, avail 22647 MB

    2015-05-18 11:09:25.942629 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28296 MB, avail 22878 MB

    2015-05-18 11:10:28.410838 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28555 MB, avail 22619 MB

    2015-05-18 11:12:06.534287 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28284 MB, avail 22890 MB

    2015-05-18 11:13:09.433899 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28337 MB, avail 22837 MB

    2015-05-18 11:14:09.485415 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28297 MB, avail 22877 MB

    2015-05-18 11:15:13.061472 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28520 MB, avail 22654 MB

    2015-05-18 11:16:47.296862 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28288 MB, avail 22886 MB

    2015-05-18 11:17:48.454379 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28480 MB, avail 22694 MB

    2015-05-18 11:19:24.178109 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28288 MB, avail 22886 MB

    2015-05-18 11:20:25.370240 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 44%
    total 51175 MB, used 28536 MB, avail 22638 MB

    2015-05-18 11:21:25.424047 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 43%
    total 51175 MB, used 28712 MB, avail 22462 MB

    2015-05-18 11:22:27.585467 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 43%
    total 51175 MB, used 28978 MB, avail 22196 MB

    2015-05-18 11:23:27.666819 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 42%
    total 51175 MB, used 29197 MB, avail 21977 MB

    2015-05-18 11:24:30.853669 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 42%
    total 51175 MB, used 29429 MB, avail 21745 MB

    2015-05-18 11:25:32.661380 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 42%
    total 51175 MB, used 29672 MB, avail 21502 MB

    2015-05-18 11:26:32.979662 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 41%
    total 51175 MB, used 29716 MB, avail 21458 MB

    2015-05-18 11:27:33.285880 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 41%
    total 51175 MB, used 29871 MB, avail 21303 MB

    2015-05-18 11:28:36.325777 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 41%
    total 51175 MB, used 30034 MB, avail 21140 MB

    2015-05-18 11:29:36.385018 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30203 MB, avail 20971 MB

    2015-05-18 11:30:42.881925 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30205 MB, avail 20969 MB

    2015-05-18 11:31:44.213153 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30457 MB, avail 20717 MB

    2015-05-18 11:33:05.049889 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30301 MB, avail 20873 MB

    2015-05-18 11:34:06.791687 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30514 MB, avail 20660 MB

    2015-05-18 11:35:35.469873 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30293 MB, avail 20881 MB

    2015-05-18 11:36:37.022678 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30534 MB, avail 20640 MB

    2015-05-18 11:38:03.430840 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30293 MB, avail 20881 MB

    2015-05-18 11:39:04.691948 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30503 MB, avail 20671 MB

    2015-05-18 11:40:28.083978 7f4a9609d700  0
    mon.monitor03@2(synchronizing).data_health(0) update_stats avail 40%
    total 51175 MB, used 30293 MB, avail 20881 MB

    
    And in my Ceph Dashboard Showing me these warnings for

    
         Cluster
            Status:HEALTH_WARN
        noscrub,nodeep-scrub
          flag(s) set
        1
          mons down, quorum 0,1 monitor01,monitor02
        mon.monitor01
          store is getting too big! 17355 MB >= 15360 MB
        mon.monitor02
          store is getting too big! 23785 MB >= 15360 MB
      
    It tells me 1 mons down because when i added my new
    Ceph Monitor , it doesn't comes up in there

    any help i would appreciate it

    
    Thanx in advance

  
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com