On 31/03/17 03:38 PM, Sinan Kaya wrote: > On 3/31/2017 5:23 PM, Logan Gunthorpe wrote: >> What exactly would you white/black list? It can't be the NIC or the >> disk. If it's going to be a white/black list on the switch or root port >> then you'd need essentially the same code to ensure they are all behind >> the same switch or root port. > > What is so special about being connected to the same switch? > > Why don't we allow the feature by default and blacklist by the root ports > that don't work with a quirk. Well root ports have the same issue here. There may be more than one root port or other buses (ie QPI) between the devices in question. So you can't just say "this system has root port X therefore we can always use p2pmem". In the end, if you want to do any kind of restrictions you're going to have to walk the tree, as the code currently does, and figure out what's between the devices being used and black or white list accordingly. Then seeing there's just such a vast number of devices out there you'd almost certainly have to use some kind of white list and not a black list. Then the question becomes which devices will be white listed? The first to be listed would be switches seeing they will always work. This is pretty much what we have (though it doesn't currently cover multiple levels of switches). The next step, if someone wanted to test with specific hardware, might be to allow the case where all the devices are behind the same root port which Intel Ivy Bridge or newer. However, I don't think a comprehensive white list should be a requirement for this work to go forward and I don't think anything you've suggested will remove any of the "ugliness". What we discussed at LSF was that only allowing cases with a switch was the simplest way to be sure any given setup would actually work. > I'm looking at this from portability perspective to be honest. I'm looking at this from the fact that there's a vast number of topologies and devices involved, and figuring out which will work is very complicated and could require a lot of hardware testing. The LSF folks were primarily concerned with not having users enable the feature and see breakage or terrible performance. > I'd rather see the feature enabled by default without any assumptions. > Using it with a switch is just a use case that you happened to test. > It can allow new architectures to use your code tomorrow. That's why I was advocating for letting userspace decide such that if you're setting up a system with this you say to use a specific p2pmem device and then you are responsible to test and benchmark it and decide to use it in going forward. However, this has received a lot of push back. Logan
