On Tue, Sep 25, 2018 at 12:02:32AM +0200, Eric W. Biederman wrote: > Andrey Vagin <avagin@xxxxxxxxxxxxx> writes: > > > On Fri, Sep 21, 2018 at 02:27:29PM +0200, Eric W. Biederman wrote: > >> Dmitry Safonov <dima@xxxxxxxxxx> writes: > >> > >> > Discussions around time virtualization are there for a long time. > >> > The first attempt to implement time namespace was in 2006 by Jeff Dike. > >> > From that time, the topic appears on and off in various discussions. > >> > > >> > There are two main use cases for time namespaces: > >> > 1. change date and time inside a container; > >> > 2. adjust clocks for a container restored from a checkpoint. > >> > > >> > “It seems like this might be one of the last major obstacles keeping > >> > migration from being used in production systems, given that not all > >> > containers and connections can be migrated as long as a time dependency > >> > is capable of messing it up.” (by github.com/dav-ell) > >> > > >> > The kernel provides access to several clocks: CLOCK_REALTIME, > >> > CLOCK_MONOTONIC, CLOCK_BOOTTIME. Last two clocks are monotonous, but the > >> > start points for them are not defined and are different for each running > >> > system. When a container is migrated from one node to another, all > >> > clocks have to be restored into consistent states; in other words, they > >> > have to continue running from the same points where they have been > >> > dumped. > >> > > >> > The main idea behind this patch set is adding per-namespace offsets for > >> > system clocks. When a process in a non-root time namespace requests > >> > time of a clock, a namespace offset is added to the current value of > >> > this clock on a host and the sum is returned. > >> > > >> > All offsets are placed on a separate page, this allows up to map it as > >> > part of vvar into user processes and use offsets from vdso calls. > >> > > >> > Now offsets are implemented for CLOCK_MONOTONIC and CLOCK_BOOTTIME > >> > clocks. > >> > > >> > Questions to discuss: > >> > > >> > * Clone flags exhaustion. Currently there is only one unused clone flag > >> > bit left, and it may be worth to use it to extend arguments of the clone > >> > system call. > >> > > >> > * Realtime clock implementation details: > >> > Is having a simple offset enough? > >> > What to do when date and time is changed on the host? > >> > Is there a need to adjust vfs modification and creation times? > >> > Implementation for adjtime() syscall. > >> > >> Overall I support this effort. In my quick skim this code looked good. > > > > Hi Eric, > > > > Thank you for the feedback. > > > >> > >> My feeling is that we need to be able to support running ntpd and > >> support one namespace doing googles smoothing of leap seconds while > >> another namespace takes the leap second. > >> > >> What I was imagining when I was last thinking about this was one > >> instance of struct timekeeper aka tk_core per time namespace. That > >> structure already keeps offsets for all of the various clocks from > >> the kerne internal time sources. What would be needed would be to > >> pass in an appropriate time namespace pointer. > >> > >> I could be completely wrong as I have not take the time to completely > >> trace through the code. Have you looked at pushing the time namespace > >> down as far as tk_core? > >> > >> What I think would be the big advantage (besides ntp working) is that > >> the bulk of the code could be reused. Allowing testing of the kernel's > >> time code by setting up a new time namespace. So a person in production > >> could setup a time namespace with the time set ahead a little bit and > >> be able to verify that the kernel handles the upcoming leap second > >> properly. > >> > > > > It is an interesting idea, but I have a few questions: > > > > 1. Does it mean that timekeeping_update() will be called for each > > namespace? This functions is called periodically, it updates times on the > > timekeeper structure, updates vsyscall_gtod_data, etc. What will be an > > overhead of this? > > I don't know if periodically is a proper characterization. There may be > a code path that does that. But from what I can see timekeeping_update > is the guts of settimeofday (and a few related functions). > > So it appears to make sense for timekeeping_update to be per namespace. > > Hmm. Looking at what is updated in the vsyscall_gtod_data it does > look like you would have to periodically update things, but I don't know > big that period would be. As long as the period is reasonably large, > or the time namespaces were sufficiently deschronized it should not > be a problem. But that is the class of problem that could make > my ideal impractical if there is measuarable overhead. > > Where were you seeing timekeeping_update being called periodically? timekeeping_update() is called HZ times per-second: [ 67.912858] timekeeping_update.cold.26+0x5/0xa [ 67.913332] timekeeping_advance+0x361/0x5c0 [ 67.913857] ? tick_sched_do_timer+0x55/0x70 [ 67.914409] ? tick_sched_do_timer+0x70/0x70 [ 67.914947] tick_sched_do_timer+0x55/0x70 [ 67.915505] tick_sched_timer+0x27/0x70 [ 67.916042] __hrtimer_run_queues+0x10f/0x440 [ 67.916639] hrtimer_interrupt+0x100/0x220 [ 67.917305] smp_apic_timer_interrupt+0x79/0x220 [ 67.918030] apic_timer_interrupt+0xf/0x20 > > > 2. What will we do with vdso? It looks like we will have to have a > > separate vsyscall_gtod_data for each ns and update each of them > > separately. > > Yes. But you don't have to have introduce another variable just make > certain vsyscall_gtod_data is a page aligned thing per time namespace. > > If I read the summary of the existing patchset something very similiar > is already going on. I mean vsyscall_gtod_data has some data which are often updated. There are timestamps for monotonic and wall clocks. clock_gettime() reads a time stamp from vsyscall_gtod_data and then use tsc to approximate the current value of a clock. Actually, this is not the second question, it is a part of the first question. update_vsyscall() is called from timekeeping_update(). > > Each process would only map one. And unshare of the time namespace > would need to act like the pid namespace or be limited to only being > allowed when there is only a single task using the mm. > > Eric
