Re: Best upgrade strategy

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



There isn't a recommendation to use 3.16.  We have experiences that force us to use 3.16 due to our setup.

We upgraded from 3.13 to 3.19 on our storage nodes to avoid occasional XFS kernel panics, but we found that we would randomly have load spikes that would lock up the storage node and show as an xfs stack trace.  We found a commit that was merged into 3.19 that looked like it could explain it.  It existed in 4.2 (latest kernel at the time) so we went to 3.16.  It resolved our XFS kernel panics and we didn't have the load spikes of death.

If you aren't seeing these things, then it isn't an issue for you.  The point stands, though, that upgrading for the sake of upgrading has a chance to introduce regressions into your environment.
________________________________________
From: Uwe Mesecke [uwe@xxxxxxxxxxx]
Sent: Monday, June 06, 2016 8:01 AM
To: David Turner
Cc: ceph-users@xxxxxxxx
Subject: Re:  Best upgrade strategy

Hi David,


> Am 06.06.2016 um 15:26 schrieb David Turner <david.turner@xxxxxxxxxxxxxxxx>:
>
> Best practices in general say to do them separate. If something doesn't work... Is it the new kernel, some package that are different on 16.04, Jewel, etc. The less things in that list the easier it is to track down the issue and fix it.
>
> As far as order, hammer 0.94.5 wasn't built with 16.04 in mind. Jewel lists both as compatibilities. I would say upgrade to jewel, make sure things are stable, and then upgrade to 16.04. It might even be wise to add a third upgrade step of running the kernel you'll be using on 16.04 before you upgrade to 16.04 to separate the 2 out.
>
> The biggest question though is why you're upgrading Ubuntu. New software does not mean better. The 3.16 kernel was the last kernel before an xfs file system regression was added to the kernel and hadn't been fixed yet in 4.2. So Ceph storage nodes running osds on xfs would be better on the much older kernel.

do you have a link describing the regression in xfs? I was not aware of any recommendation of using 3.16 kernels with osds running on xfs.

Thanks!
Uwe

>
> Sent from my iPhone
>
>> On Jun 5, 2016, at 5:48 PM, Sebastian Köhler <sk@xxxxxxxxx> wrote:
>>
>> Hi,
>>
>> we are running a cluster with 6 storage nodes(72 osds) and 3 monitors.
>> The osds and and monitors are running on Ubuntu 14.04 and with ceph 0.94.5.
>> We want to upgrade the cluster to Jewel and at the same time the OS to
>> Ubuntu 16.04. What would be the best way to this? First to upgrade the
>> OS and then ceph to 0.94.7 followed by 10.2.1. Or should we first
>> upgrade Ceph and then Ubuntu? Or maybe doing it all at once?
>>
>> Regards
>> Sebastian
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux