Re: old osds take much longer to start than newer osd

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Does deleting/reformatting the old osds improve the performance?

On Fri, Feb 27, 2015 at 6:02 AM, Corin Langosch
<corin.langosch@xxxxxxxxxxx> wrote:
> Hi guys,
>
> I'm using ceph for a long time now, since bobtail. I always upgraded every few weeks/ months to the latest stable
> release. Of course I also removed some osds and added new ones. Now during the last few upgrades (I just upgraded from
> 80.6 to 80.8) I noticed that old osds take much longer to startup than equal newer osds (same amount of data/ disk
> usage, same kind of storage+journal backing device (ssd), same weight, same number of pgs, ...). I know I observed the
> same behavior earlier but just didn't really care about it. Here are the relevant log entries (host of osd.0 and osd.15
> has less cpu power than the others):
>
> old osds (average pgs load time: 1.5 minutes)
>
> 2015-02-27 13:44:23.134086 7ffbfdcbe780  0 osd.0 19323 load_pgs
> 2015-02-27 13:49:21.453186 7ffbfdcbe780  0 osd.0 19323 load_pgs opened 824 pgs
>
> 2015-02-27 13:41:32.219503 7f197b0dd780  0 osd.3 19317 load_pgs
> 2015-02-27 13:42:56.310874 7f197b0dd780  0 osd.3 19317 load_pgs opened 776 pgs
>
> 2015-02-27 13:38:43.909464 7f450ac90780  0 osd.6 19309 load_pgs
> 2015-02-27 13:40:40.080390 7f450ac90780  0 osd.6 19309 load_pgs opened 806 pgs
>
> 2015-02-27 13:36:14.451275 7f3c41d33780  0 osd.9 19301 load_pgs
> 2015-02-27 13:37:22.446285 7f3c41d33780  0 osd.9 19301 load_pgs opened 795 pgs
>
> new osds (average pgs load time: 3 seconds)
>
> 2015-02-27 13:44:25.529743 7f2004617780  0 osd.15 19325 load_pgs
> 2015-02-27 13:44:36.197221 7f2004617780  0 osd.15 19325 load_pgs opened 873 pgs
>
> 2015-02-27 13:41:29.176647 7fb147fb3780  0 osd.16 19315 load_pgs
> 2015-02-27 13:41:31.681722 7fb147fb3780  0 osd.16 19315 load_pgs opened 848 pgs
>
> 2015-02-27 13:38:41.470761 7f9c404be780  0 osd.17 19307 load_pgs
> 2015-02-27 13:38:43.737473 7f9c404be780  0 osd.17 19307 load_pgs opened 821 pgs
>
> 2015-02-27 13:36:10.997766 7f7315e99780  0 osd.18 19299 load_pgs
> 2015-02-27 13:36:13.511898 7f7315e99780  0 osd.18 19299 load_pgs opened 815 pgs
>
> The old osds also take more memory, here's an example:
>
> root     15700 22.8  0.7 1423816 485552 ?      Ssl  13:36   4:55 /usr/bin/ceph-osd -i 9 --pid-file
> /var/run/ceph/osd.9.pid -c /etc/ceph/ceph.conf --cluster ceph
> root     15270 15.4  0.4 1227140 297032 ?      Ssl  13:36   3:20 /usr/bin/ceph-osd -i 18 --pid-file
> /var/run/ceph/osd.18.pid -c /etc/ceph/ceph.conf --cluster ceph
>
>
> It seems to me there is still some old data around for the old osds which was not properly migrated/ cleaned up during
> the upgrades. The cluster is healthy, no problems at all the last few weeks. Is there any way to clean this up?
>
> Thanks
> Corin
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux