On Wed, January 2, 2013 03:53, Robert Dinse wrote:
>
> Friday, I moved our servers to a new co-lo facility and ran into
> an interesting problem with virtual machines.
>
> I did an orderly shutdown of the CentOS 6.3 host, and it in
> turn suspends all the guests. It took about an hour and a half
> to move and fire up the host.
>
> The guests, being suspended, were then an hour and a half
> behind and it seems ntpd does not want to correct more than 1000
> seconds of error so it would not automatically adjust the clocks.
>
> I tried the -g argument which is supposed to override the
> 1000 second limit but it did not. I ended up having to manually
> set the clocks close enough for ntpd to correct.
>
> Since there is no hardware clock for the virtual machines
> to use when they boot, it seems that shutdown and reboot of the
> virtual machines probably would not have avoided this.
>
> Any suggestions for addressing this particular scenerio other
> than having to manually set a bunch of clocks?
>
I ran into this situation several times whilst testing KVM and the
lessons I learned from the experiences can be summarized as:
1. Never allow the kvm hypervisior to handle guests during a host
shutdown. Use 'virsh shutdown' on each of the guests first and then
shutdown the host. Use autostart to restart guests on a host's
reboot. Write a script to process 'virsh list' to feed active domains
to 'virsh shutdown' if automation is required and link that to
/etc/rc0.d/K10<whatever>.
2. In the situation where a kvm guest pause and restore sequence
leads to an excessive disconnect between guest time and wall time use
ntpd -q to hard set the time. From the guest's point of view you are
always going ahead in time in the case of a pause and resume so this
is not likely to ever cause a problem. But, having written that down,
it probably will at some point.
3. Run ntpd on the host system and have its guests configured to only
use that time server source.
4. On each guest have a cron job that checks for ntpd at regular
intervals which reports failures and restarts the time service as
necessary. We use:
JOBNAME="Check ntpd status and restart if required" ; \
ntpstat > /dev/null && \
if [[ $? -gt 0 ]]; then /sbin/service ntpd start; fi
--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:ByrneJB@xxxxxxxxxxxxx
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3
_______________________________________________
CentOS-virt mailing list
CentOS-virt@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos-virt
