On Wed, Apr 18, 2018 at 10:00 PM, Samudrala, Sridhar <sridhar.samudrala@xxxxxxxxx> wrote: > On 4/18/2018 9:41 PM, Michael S. Tsirkin wrote: >> >> On Wed, Apr 18, 2018 at 04:33:34PM -0700, Samudrala, Sridhar wrote: >>> >>> On 4/17/2018 5:26 PM, Siwei Liu wrote: >>>> >>>> I ran this with a few folks offline and gathered some good feedbacks >>>> that I'd like to share thus revive the discussion. >>>> >>>> First of all, as illustrated in the reply below, cloud service >>>> providers require transparent live migration. Specifically, the main >>>> target of our case is to support SR-IOV live migration via kernel >>>> upgrade while keeping the userspace of old distros unmodified. If it's >>>> because this use case is not appealing enough for the mainline to >>>> adopt, I will shut up and not continue discussing, although >>>> technically it's entirely possible (and there's precedent in other >>>> implementation) to do so to benefit any cloud service providers. >>>> >>>> If it's just the implementation of hiding netdev itself needs to be >>>> improved, such as implementing it as attribute flag or adding linkdump >>>> API, that's completely fine and we can look into that. However, the >>>> specific issue needs to be undestood beforehand is to make transparent >>>> SR-IOV to be able to take over the name (so inherit all the configs) >>>> from the lower netdev, which needs some games with uevents and name >>>> space reservation. So far I don't think it's been well discussed. >>>> >>>> One thing in particular I'd like to point out is that the 3-netdev >>>> model currently missed to address the core problem of live migration: >>>> migration of hardware specific feature/state, for e.g. ethtool configs >>>> and hardware offloading states. Only general network state (IP >>>> address, gateway, for eg.) associated with the bypass interface can be >>>> migrated. As a follow-up work, bypass driver can/should be enhanced to >>>> save and apply those hardware specific configs before or after >>>> migration as needed. The transparent 1-netdev model being proposed as >>>> part of this patch series will be able to solve that problem naturally >>>> by making all hardware specific configurations go through the central >>>> bypass driver, such that hardware configurations can be replayed when >>>> new VF or passthrough gets plugged back in. Although that >>>> corresponding function hasn't been implemented today, I'd like to >>>> refresh everyone's mind that is the core problem any live migration >>>> proposal should have addressed. >>>> >>>> If it would make things more clear to defer netdev hiding until all >>>> functionalities regarding centralizing and replay are implemented, >>>> we'd take advices like that and move on to implementing those features >>>> as follow-up patches. Once all needed features get done, we'd resume >>>> the work for hiding lower netdev at that point. Think it would be the >>>> best to make everyone understand the big picture in advance before >>>> going too far. >>> >>> I think we should get the 3-netdev model integrated and add any >>> additional >>> ndo_ops/ethool ops that we would like to support/migrate before looking >>> into >>> hiding the lower netdevs. >> >> Once they are exposed, I don't think we'll be able to hide them - >> they will be a kernel ABI. >> >> Do you think everyone needs to hide the SRIOV device? >> Or that only some users need this? > > > Hyper-V is currently supporting live migration without hiding the SR-IOV > device. So i don't > think it is a hard requirement. And also, as we don't yet have a consensus This is a vague point as Hyper-V is mostly Windows oriented: the target users don't change adapter settings in device manager much as it's hidden too deep already. Actually it does not address the general case for SR-IOV live migration but just a subset, why are we making such comparison? Note it's always the hard requirement for live migration that *all states* should be migrated no matter what the implementation it is going to be. The current 3-netdev model is remote to be useful for real world scenario and it has no advantage compared to using userspace generic bonding. -Siwei > on how to hide > the lower netdevs, we could make it as another feature bit to hide lower > netdevs once > we have an acceptable solution. > _______________________________________________ Virtualization mailing list Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx https://lists.linuxfoundation.org/mailman/listinfo/virtualization
