On Tue, 30 Mar 2021 22:37:06 +0530 Vaibhav Jain <vaibhav@xxxxxxxxxxxxx> wrote: > > Thanks for looking into this patch Greg. My responses below inline. > > > Greg Kurz <groug@xxxxxxxx> writes: > > > Hi Vaibhav, > > > > Great to see you around :-) > > :-) > > > > > On Mon, 29 Mar 2021 21:52:59 +0530 > > Vaibhav Jain <vaibhav@xxxxxxxxxxxxx> wrote: > > > >> Add support for H_SCM_HEALTH hcall described at [1] for spapr > >> nvdimms. This enables guest to detect the 'unarmed' status of a > >> specific spapr nvdimm identified by its DRC and if its unarmed, mark > >> the region backed by the nvdimm as read-only. > >> > > > > Any chance that you can provide the documentation of this new hcall ? > > > H_SCM_HEALTH specifications is already documented in linux kernel > documentation at > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/powerpc/papr_hcalls.rst > > That documentation was added when kernel support for H_SCM_HEALTH hcall > support was implemented in 5.9 kernel. > Oops I skipped that indeed. Maybe even make it: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/powerpc/papr_hcalls.rst#n220 for faster access. > >> The patch adds h_scm_health() to handle the H_SCM_HEALTH hcall which > >> returns two 64-bit bitmaps (health bitmap, health bitmap mask) derived > >> from 'struct nvdimm->unarmed' member. > >> > >> Linux kernel side changes to enable handling of 'unarmed' nvdimms for > >> ppc64 are proposed at [2]. > >> > >> References: > >> [1] "Hypercall Op-codes (hcalls)" > >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/powerpc/papr_hcalls.rst > >> > >> [2] "powerpc/papr_scm: Mark nvdimm as unarmed if needed during probe" > >> https://lore.kernel.org/linux-nvdimm/20210329113103.476760-1-vaibhav@xxxxxxxxxxxxx/ > >> > >> Signed-off-by: Vaibhav Jain <vaibhav@xxxxxxxxxxxxx> > >> --- > >> hw/ppc/spapr_nvdimm.c | 30 ++++++++++++++++++++++++++++++ > >> include/hw/ppc/spapr.h | 4 ++-- > >> 2 files changed, 32 insertions(+), 2 deletions(-) > >> > >> diff --git a/hw/ppc/spapr_nvdimm.c b/hw/ppc/spapr_nvdimm.c > >> index b46c36917c..e38740036d 100644 > >> --- a/hw/ppc/spapr_nvdimm.c > >> +++ b/hw/ppc/spapr_nvdimm.c > >> @@ -31,6 +31,13 @@ > >> #include "qemu/range.h" > >> #include "hw/ppc/spapr_numa.h" > >> > >> +/* DIMM health bitmap bitmap indicators */ > >> +/* SCM device is unable to persist memory contents */ > >> +#define PAPR_PMEM_UNARMED (1ULL << (63 - 0)) > > > > This looks like PPC_BIT(0). > > > Yes, right. Will update the patch in v2 to use the PPC_BIT macro. > Well, since this was copied from the kernel sources, I guess you can leave it as is. Maybe just explain where this macro comes from if you do so. > >> + > >> +/* Bits status indicators for health bitmap indicating unarmed dimm */ > >> +#define PAPR_PMEM_UNARMED_MASK (PAPR_PMEM_UNARMED) > >> + > >> bool spapr_nvdimm_validate(HotplugHandler *hotplug_dev, NVDIMMDevice *nvdimm, > >> uint64_t size, Error **errp) > >> { > >> @@ -467,6 +474,28 @@ static target_ulong h_scm_unbind_all(PowerPCCPU *cpu, SpaprMachineState *spapr, > >> return H_SUCCESS; > >> } > >> > >> +static target_ulong h_scm_health(PowerPCCPU *cpu, SpaprMachineState *spapr, > >> + target_ulong opcode, target_ulong *args) > >> +{ > >> + uint32_t drc_index = args[0]; > >> + SpaprDrc *drc = spapr_drc_by_index(drc_index); > >> + NVDIMMDevice *nvdimm; > >> + > >> + if (drc && spapr_drc_type(drc) != SPAPR_DR_CONNECTOR_TYPE_PMEM) { > >> + return H_PARAMETER; > >> + } > >> + > >> + nvdimm = NVDIMM(drc->dev); > > > > Yeah as already suggested by Shiva, drc->dev should be checked like > > in h_scm_bind_mem(). > > > Yes, will send a v2 with this case handled. > > >> + > >> + /* Check if the nvdimm is unarmed and send its status via health bitmaps */ > >> + args[0] = nvdimm->unarmed ? PAPR_PMEM_UNARMED_MASK : 0; > >> + > > > > Shouldn't ^^ use PAPR_PMEM_UNARMED then ? > > > >> + /* health bitmap mask same as the health bitmap */ > >> + args[1] = args[0]; > >> + > > > > If so, it seems that PAPR_PMEM_UNARMED_MASK isn't even needed. > > Definition of these defines are similar to what kernel implementation > uses at > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/powerpc/platforms/pseries/papr_scm.c#n53 > > Since unarmed condition can also arise due to an unhealthy nvdimm hence > the kernel implementation uses a mask thats composed of two bits > PPC_BIT(0) and PPC_BIT(6) being set. Though we arent using PPC_BIT(6) > right now in qemu, it will change in future when better nvdimm health > reporting will be done. Hence kept the PPC_BIT(0) define as well as the > mask to mimic the kernel definitions. > After a deeper look into the kernel documentation and code, it seems that we really don't need PAPR_PMEM_UNARMED_MASK at all on the QEMU side : it is _just_ the mask of bits that linux does support (i.e. bit 0 and 6) and applies to the health bitmap returned by H_SCM_HEALTH. On the QEMU side, as you say, we just support setting bit 0 from the time being : args[0] should thus be set to PAPR_PMEM_UNARMED if the NVDIMM is unarmed, not PAPR_PMEM_UNARMED_MASK. Also, it seems that the purpose of the health bitmap mask is to tell the guest about the bits supported by the hypervisor : args[1] should thus unconditionally be set to PAPR_PMEM_UNARMED, otherwise we'd tell the guest none of the bits are valid when the NVDIMM isn't unarmed. > > > > Having access to the excerpts from the PAPR addendum that describes > > this hcall would _really_ help in reviewing. > > > The kernel documentation for H_SCM_HEALTH mentioned above captures most > if not all parts of the PAPR addendum for this hcall. I believe it > contains enough information to review the patch. If you still need more > info than please let me know. > Sorry again for having missed the link the first time. I guess I have all the information I need now. > > >> + return H_SUCCESS; > >> +} > >> + > >> static void spapr_scm_register_types(void) > >> { > >> /* qemu/scm specific hcalls */ > >> @@ -475,6 +504,7 @@ static void spapr_scm_register_types(void) > >> spapr_register_hypercall(H_SCM_BIND_MEM, h_scm_bind_mem); > >> spapr_register_hypercall(H_SCM_UNBIND_MEM, h_scm_unbind_mem); > >> spapr_register_hypercall(H_SCM_UNBIND_ALL, h_scm_unbind_all); > >> + spapr_register_hypercall(H_SCM_HEALTH, h_scm_health); > >> } > >> > >> type_init(spapr_scm_register_types) > >> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h > >> index 47cebaf3ac..18859b9ab2 100644 > >> --- a/include/hw/ppc/spapr.h > >> +++ b/include/hw/ppc/spapr.h > >> @@ -538,8 +538,8 @@ struct SpaprMachineState { > >> #define H_SCM_BIND_MEM 0x3EC > >> #define H_SCM_UNBIND_MEM 0x3F0 > >> #define H_SCM_UNBIND_ALL 0x3FC > >> - > >> -#define MAX_HCALL_OPCODE H_SCM_UNBIND_ALL > >> +#define H_SCM_HEALTH 0x400 > >> +#define MAX_HCALL_OPCODE H_SCM_HEALTH > >> > >> /* The hcalls above are standardized in PAPR and implemented by pHyp > >> * as well. > > >
