On 12/25/21 2:26 AM, Leon Romanovsky wrote:
On Fri, Dec 24, 2021 at 03:07:57PM +0800, Cheng Xu wrote:
On 12/23/21 9:44 PM, Leon Romanovsky wrote:
On Thu, Dec 23, 2021 at 08:59:14PM +0800, Cheng Xu wrote:
On 12/23/21 6:23 PM, Leon Romanovsky wrote:
On Wed, Dec 22, 2021 at 11:35:44AM +0800, Cheng Xu wrote:
<...>
For the ECS instance with RDMA enabled, there are two kinds of devices
allocated, one for ERDMA, and one for the original netdev (virtio-net).
They are different PCI deivces. ERDMA driver can get the information about
which netdev attached to in its PCIe barspace (by MAC address matching).
This is very questionable. The netdev part should be kept in the
drivers/ethernet/... part of the kernel.
Thanks
The net device used in Alibaba ECS instance is virtio-net device, driven
by virtio-pci/virtio-net drivers. ERDMA device does not need its own net
device, and will be attached to an existed virtio-net device. The
relationship between ibdev and netdev in erdma is similar to siw/rxe.
siw/rxe binds through RDMA_NLDEV_CMD_NEWLINK netlink command and not
through MAC's matching.
Thanks
Both siw/rxe/erdma don't need to implement netdev part, this is what I
wanted to express when I said 'similar'.
What you mentioned (the bind mechanism) is one major difference between
erdma and siw/rxe. For siw/rxe, user can attach ibdev to every netdev if
he/she wants, but it is not true for erdma. When user buys the erdma
service, he/she must specify which ENI (elastic network interface) to be
binded, it means that the attached erdma device can only be binded to
the specific netdev. Due to the uniqueness of MAC address in our ECS
instance, we use the MAC address as the identification, then the driver
knows which netdev should be binded to.
Nothing prohibits from you to implement this MAC check in RDMA_NLDEV_CMD_NEWLINK.
I personally don't like the idea that bind logic is performed "magically".
OK, I agree with you that using RDMA_NLDEV_CMD_NEWLINK is better. But it
means that erdma can not be ready to use like other RDMA HCAs, until
user configure the link manually. This way may be not friendly to them.
I'm not sure that our current method is acceptable or not. If you
strongly recommend us to use RDMA_NLDEV_CMD_NEWLINK, we will change to
it.
Before you are rushing to change that logic, could you please explain
the security model of this binding?
I'm as an owner of VM can replace kernel code with any code I want and
remove your MAC matching (or replace to something different). How will
you protect from such flow?
(I'm sorry for wrong editing format in the two former responses, please
ignore them.)
I think this topic belongs to anti-attack. One principle of anti-attack
in our cloud is that the attacker MUST NOT have influence on users but
themselves.
Before I answer the question, I want to describe some more details of
our architecture.
In our MOC architecture, virtio-net device (e.g, virtio-net back-end) is
fully offloaded to MOC, not in host hypervisor. One virtio-net device
belongs to a vport, and if it has a peer erdma device, erdma device also
belongs to the vport. The protocol headers of the network flows in the
virtio-net and erdma devices must be consistent with the vport
configurations (mac address, ip, etc. ) by checking the OVS rules.
Back to the question, we can not prevent attackers from modifying the
code, making devices binding wrongly in the front-end, or in some worse
cases, making driver sending invalid commands to devices. If binding
wrongly, the erdma network will be unreachable, because the OVS module
in MOC hardware can distinguish this situation and drop all the invalid
network packets, and this has no influence to other users.
If you don't trust VM, you should perform binding in hypervisor and
this erdma driver will work out-of-the-box in the VM.
As mentioned above, we also have the binding configuration in the
back-end (e.g, MOC hardware), only when the configuration is correct of
the front-end, the erdma can work properly.
Thanks,
Cheng Xu
Thanks
