On 9/7/20 6:22 PM, Leon Romanovsky wrote:
On Mon, Sep 07, 2020 at 05:28:23PM +0800, Ka-Cheong Poon wrote:
On 9/7/20 5:04 PM, Leon Romanovsky wrote:
On Mon, Sep 07, 2020 at 04:24:26PM +0800, Ka-Cheong Poon wrote:
On 9/7/20 3:18 PM, Leon Romanovsky wrote:
On Mon, Sep 07, 2020 at 11:33:38AM +0800, Ka-Cheong Poon wrote:
On 9/6/20 3:44 PM, Leon Romanovsky wrote:
On Fri, Sep 04, 2020 at 10:02:10PM +0800, Ka-Cheong Poon wrote:
On 9/4/20 7:32 PM, Jason Gunthorpe wrote:
On Fri, Sep 04, 2020 at 12:01:12PM +0800, Ka-Cheong Poon wrote:
On 9/4/20 1:39 AM, Jason Gunthorpe wrote:
On Thu, Sep 03, 2020 at 10:02:01PM +0800, Ka-Cheong Poon wrote:
When a struct ib_client's add() function is called. is there a
supported method to find out the namespace of the passed in
struct ib_device? There is rdma_dev_access_netns() but it does
not return the namespace. It seems that it needs to have
something like the following.
struct net *rdma_dev_to_netns(struct ib_device *ib_dev)
{
return read_pnet(&ib_dev->coredev.rdma_net);
}
Comments?
I suppose, but why would something need this?
If the client needs to allocate stuff for the namespace
related to that device, it needs to know the namespace of
that device. Then when that namespace is deleted, the
client can clean up those related stuff as the client's
namespace exit function can be called before the remove()
function is triggered in rdma_dev_exit_net(). Without
knowing the namespace of that device, coordination cannot
be done.
Since each device can only be in one namespace, why would a client
ever need to allocate at a level more granular than a device?
A client wants to have namespace specific info. If the
device belongs to a namespace, it wants to associate those
info with that device. When a namespace is deleted, the
info will need to be deleted. You can consider the info
as associated with both a namespace and a device.
Can you be more specific about which info you are talking about?
Actually, a lot of info can be both namespace and device specific.
For example, a client wants to have a different PD allocation policy
with a device when used in different namespaces.
And what is the client that is net namespace-aware from one side,
but from another separate data between them "manually"?
Could you please elaborate what is meant by "namespace aware from
one side but from another separate data between them manually"?
I understand what namespace aware means. But it is not clear what
is meant by "separating data manually". Do you mean having different
behavior in different namespaces? If this is the case, there is
nothing special here. An admin may choose to have different behavior
in different namespaces. There is nothing manual going on in the
client code.
We are talking about net-namespaces, and as we wrote above, the ib_device
that supports such namespace can exist only in a single one
The client that implemented such support can check its namespace while
"client->add" is called. It should be equal to be seen by ib_device.
See:
rdma_dev_change_netns ->
enable_device_and_get ->
add_client_context ->
client->add(device)
This is the original question. How does the client's add() function
know the namespace of device? What is your suggestion in finding
the net namespace of device at add() time?
As I wrote above, "It should be equal to be seen by ib_device.", check net
namespace of your client.
Could you please be more specific? A client calls ib_register_client() to
register with the RDMA framework. Then when a device is added, the client's
add() function is called with the struct ib_device. How does the client
find out the namespace "seen by the ib_device"? Do you mean that there is
a variant of ib_register_client() which can take a net namespace as parameter?
Or is there a variant of struct ib_client which has a net namespace field?
Or? Thanks.
"Do you mean that there is a variant of ib_register_client()
which can take a net namespace as parameter?"
No, it doesn't exist but it is easy to extend and IMHO the right
thing to do.
This may require a number of changes and the way a client interacts with
the current RDMA framework. For example, currently a client registers
once using one struct ib_client and gets device notifications for all
namespaces and devices. Suppose there is rdma_[un]register_net_client(),
it may need to require a client to use a different struct ib_client to
register for each net namespace. And struct ib_client probably needs to
have a field to store the net namespace. Probably all those client
interaction functions will need to be modified. Since the clients xarray
is global, more clients may mean performance implication, such as it takes
longer to go through the whole clients xarray.
There are probably many other subtle changes required. It may turn out to
be not so straight forward. Is this community willing the take such changes?
I can take a stab at it if the community really thinks that this is preferred.
Thanks.
--
K. Poon
ka-cheong.poon@xxxxxxxxxx
