On Mon, May 08, 2023 at 11:30:43PM +0000, Wei Liu wrote:
> On Mon, May 08, 2023 at 11:44:19PM +0200, Peter Zijlstra wrote:
> > On Mon, May 08, 2023 at 11:19:58PM +0200, Peter Zijlstra wrote:
> >
> > > --- a/drivers/clocksource/hyperv_timer.c
> > > +++ b/drivers/clocksource/hyperv_timer.c
> > > @@ -408,9 +408,9 @@ static u64 notrace read_hv_clock_tsc_cs(
> > > return read_hv_clock_tsc();
> > > }
> > >
> > > -static u64 notrace read_hv_sched_clock_tsc(void)
> > > +static u64 noinstr read_hv_sched_clock_tsc(void)
> > > {
> > > - return (read_hv_clock_tsc() - hv_sched_clock_offset) *
> > > + return (hv_read_tsc_page(hv_get_tsc_page()) - hv_sched_clock_offset) *
> > > (NSEC_PER_SEC / HV_CLOCK_HZ);
> > > }
> > >
> > > --- a/include/clocksource/hyperv_timer.h
> > > +++ b/include/clocksource/hyperv_timer.h
> > > @@ -38,7 +38,7 @@ extern void hv_remap_tsc_clocksource(voi
> > > extern unsigned long hv_get_tsc_pfn(void);
> > > extern struct ms_hyperv_tsc_page *hv_get_tsc_page(void);
> > >
> > > -static inline notrace u64
> > > +static __always_inline notrace u64
> > > hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg, u64 *cur_tsc)
> > > {
> > > u64 scale, offset;
> > > @@ -85,7 +85,7 @@ hv_read_tsc_page_tsc(const struct ms_hyp
> > > return mul_u64_u64_shr(*cur_tsc, scale, 64) + offset;
> > > }
> > >
> > > -static inline notrace u64
> > > +static __always_inline notrace u64
> > > hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg)
> > > {
> > > u64 cur_tsc;
> >
> > Hyper-V folks!
> >
> > While reviewing all this I found the following 'gem':
> >
> > hv_init_clocksource()
> > hv_setup_sched_clock()
> > paravirt_set_sched_clock(read_hv_sched_clock_msr)
> >
> > read_hv_sched_clock_msr() [notrace]
> > read_hv_clock_msr() [notrace]
> > hv_get_register() *traced*
> > hv_get_non_nested_register() ...
> > hv_ghcb_msr_read()
> > WARN_ON(in_nmi())
> > ...
> > local_irq_save()
> >
> >
> > Note that:
> >
> > a) sched_clock() is used in NMI context a *LOT*
> > b) sched_clock() is notrace (or even noinstr with these patches)
> > and local_irq_save() implies tracing
> >
>
> Tianyu and Michael, what's your thought on this?
>
> Is the MSR-based GHCB usable at this point?
>
> What other clock source can be used?
You do have TSC support -- which is what I fixed for you. It's just the
whole MSR thing that is comically broken.
You could do a read_hv_clock_msr() implementation using
__rdmsr() and add some sanity checking that anything GHCB using (SEV?)
*will* use TSC.
Anyway, will you guys do that, or should I pull out the chainsaw and fix
it for you?
