Hi Christoph:
On 8/30/2021 8:00 PM, Christoph Hellwig wrote:
Sorry for the delayed answer, but I look at the vmap_pfn usage in the
previous version and tried to come up with a better version. This
mostly untested branch:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/hyperv-vmap
No problem. Thank you very much for your suggestion patches and they are
very helpful.
get us there for swiotlb and the channel infrastructure I've started
looking at the network driver and didn't get anywhere due to other work.
As far as I can tell the network driver does gigantic multi-megabyte
vmalloc allocation for the send and receive buffers, which are then
passed to the hardware, but always copied to/from when interacting
with the networking stack. Did I see that right? Are these big
buffers actually required unlike the normal buffer management schemes
in other Linux network drivers?
For send packet, netvsc tries batching packet in send buffer if
possible. It passes the original skb pages directly to
hypervisor when send buffer is not enough or packet length is larger
than section size. These packets are sent via
vmbus_sendpacket_pagebuffer() finally. Please see netvsc_send() for
detail. The following code is to check whether the packet could be
copied into send buffer. If not, the packet will be sent with original
skb pages.
1239 /* batch packets in send buffer if possible */
1240 msdp = &nvchan->msd;
1241 if (msdp->pkt)
1242 msd_len = msdp->pkt->total_data_buflen;
1243
1244 try_batch = msd_len > 0 && msdp->count < net_device->max_pkt;
1245 if (try_batch && msd_len + pktlen + net_device->pkt_align <
1246 net_device->send_section_size) {
1247 section_index = msdp->pkt->send_buf_index;
1248
1249 } else if (try_batch && msd_len + packet->rmsg_size <
1250 net_device->send_section_size) {
1251 section_index = msdp->pkt->send_buf_index;
1252 packet->cp_partial = true;
1253
1254 } else if (pktlen + net_device->pkt_align <
1255 net_device->send_section_size) {
1256 section_index =
netvsc_get_next_send_section(net_device);
1257 if (unlikely(section_index == NETVSC_INVALID_INDEX)) {
1258 ++ndev_ctx->eth_stats.tx_send_full;
1259 } else {
1260 move_pkt_msd(&msd_send, &msd_skb, msdp);
1261 msd_len = 0;
1262 }
1263 }
1264
For receive packet, the data is always copied from recv buffer.
If so I suspect the best way to allocate them is by not using vmalloc
but just discontiguous pages, and then use kmap_local_pfn where the
PFN includes the share_gpa offset when actually copying from/to the
skbs.
When netvsc needs to copy packet data to send buffer, it needs to
caculate position with section_index and send_section_size.
Please seee netvsc_copy_to_send_buf() detail. So the contiguous virtual
address of send buffer is necessary to copy data and batch packets.
