maharishi bhargava <bhargavamaharishi@xxxxxxxxx> writes:
> On Wed, Jun 3, 2020 at 4:41 PM Maciej Fijalkowski
> <maciej.fijalkowski@xxxxxxxxx> wrote:
>>
>> On Wed, Jun 03, 2020 at 01:07:05PM +0200, Toke Høiland-Jørgensen wrote:
>> > Maciej Fijalkowski <maciej.fijalkowski@xxxxxxxxx> writes:
>> >
>> > > On Wed, Jun 03, 2020 at 12:49:25PM +0200, Toke Høiland-Jørgensen wrote:
>> > >> maharishi bhargava <bhargavamaharishi@xxxxxxxxx> writes:
>> > >>
>> > >> > On Tue, Jun 2, 2020 at 9:31 PM Toke Høiland-Jørgensen <toke@xxxxxxxxxx> wrote:
>> > >> >>
>> > >> >> maharishi bhargava <bhargavamaharishi@xxxxxxxxx> writes:
>> > >> >>
>> > >> >> > On Tue 2 Jun, 2020, 14:31 Toke Høiland-Jørgensen, <toke@xxxxxxxxxx> wrote:
>> > >> >> >>
>> > >> >> >> maharishi bhargava <bhargavamaharishi@xxxxxxxxx> writes:
>> > >> >> >>
>> > >> >> >> > Hi, in my XDP program, I want to redirect some packets using AF_XDP
>> > >> >> >> > and redirect other packets directly from driver space.
>> > >> >> >> > Redirection through AF_XDP works fine, but redirection through dev map
>> > >> >> >> > stops after some packets are processed.
>> > >> >> >>
>> > >> >> >> Do you mean it stops even if you are *only* redirecting to a devmap, or
>> > >> >> >> if you are first redirecting a few packets to AF_XDP, then to devmap?
>> > >> >> >>
>> > >> >> >> Also, which driver(s) are the physical NICs you're redirecting to/from
>> > >> >> >> using, and which kernel version are you on?
>> > >> >> >>
>> > >> >> >> -Toke
>> > >> >> >
>> > >> >> >
>> > >> >> >
>> > >> >> > Currently, I'm trying to redirect packets only using devmap. But also
>> > >> >> > have code for redirection using AF_XDP(only when a given condition is
>> > >> >> > satisfied). A DPDK program is running in userspace which will receive
>> > >> >> > packets from AF_XDP.
>> > >> >>
>> > >> >> Right, so it's just devmap redirect that breaks. What do you mean
>> > >> >> 'redirection stops', exactly? How are you seeing this? Does xdp_monitor
>> > >> >> (from samples/bpf) report any exceptions?
>> > >> >>
>> > >> >> -Toke
>> > >> >>
>> > >> > So, In my setup, there are three systems, Let's Assume A, B, C. System
>> > >> > B is acting as a forwarder between A and C. So I can see the number of
>> > >> > packets received at system C. To be specific, only 1024 packets are
>> > >> > received. If I remove the xsks_map part from the code and don't run
>> > >> > DPDK in userspace. This problem does not occur. Also if I forward all
>> > >> > the packets using AF_XDP, there is no such issue.
>> > >>
>> > >> I thought you said you were seeing the problem when only redirecting to
>> > >> a devmap? So why does the xsk_map code impact this? I think you may have
>> > >> to share some code...
>> > >
>> > > Isn't the case here that either xsk_map or dev_map consumes the frame and
>> > > therefore the latter doesn't see it? so cloning might be needed here?
>> >
>> > Yeah, certainly you can't redirect *the same packet* to both xsk_map and
>> > devmap - but that wasn't what I understood was the use case here?
>>
>> Maybe the best would be if Maharishi shared the code as you requested :)
>>
>> >
>> > -Toke
>> >
> CODE:
> BPF MAPS:
>
>
> struct bpf_map_def SEC("maps") xsks_map = {
> .type = BPF_MAP_TYPE_XSKMAP,
> .key_size = sizeof(int),
> .value_size = sizeof(int),
> .max_entries = 64, /* Assume netdev has no more than 64 queues */
> };
>
> struct bpf_map_def SEC("maps") tx_port = {
> .type = BPF_MAP_TYPE_DEVMAP,
> .key_size = sizeof(int),
> .value_size = sizeof(int),
> .max_entries = 1024,
> };
>
> struct Ingress_qos_lts_value{
> struct bpf_spin_lock lock;
> u64 timestamp;
> };
> struct bpf_map_def SEC("maps") Ingress_qos_lts = {
> .type = BPF_MAP_TYPE_ARRAY,
> .key_size = sizeof(u32),
> .value_size = sizeof(struct Ingress_qos_lts_value),
> .max_entries = 1025,
> };
> BPF_ANNOTATE_KV_PAIR(Ingress_qos_lts,u32,struct Ingress_qos_lts_value);
>
>
> SEC("prog")
> int ebpf_filter(struct xdp_md *ctx){
> struct xdp_output xout;
> xout.output_port = 1;
> void* ebpf_packetStart = ((void*)(long)ctx->data);
> void* ebpf_packetEnd = ((void*)(long)ctx->data_end);
> u64 rate = 100;//100 Kbps
> rate *= 1000*1000*100;//10 Gbps
> u32 key = 1;//some key
> u64 packet_length=(ebpf_packetEnd-ebpf_packetStart-42)*8;
> packet_length *= 1000000000; //packet length * 10^9, to convert
> rate from second to nanosecond
> struct Ingress_qos_lts_value* val;
> val = bpf_map_lookup_elem(&Ingress_qos_lts, &key);
> u64 now = bpf_ktime_get_ns();
> u64 lts;
> if (val) {
> bpf_spin_lock(&val->lock);
> lts = *(&val->timestamp)+(packet_length/rate);
> if(now>lts){
> lts = now;
> }
> *(&val->timestamp) = lts;
> bpf_spin_unlock(&val->lock);
> // printk("Time : %x %x\n",lts,now);
> if(lts>now){
> return bpf_redirect_map(&xsks_map, ctx->rx_queue_index, 0);
> }
> }
> return bpf_redirect_map(&tx_port,xout.output_port,0);
> }
>
> So, Basically this code redirects the packet to some other interface
> or sends the packet to userspace based on the incoming packet rate.
Well, if you say it goes away when you remove the xsk code, the obvious
explanation would be that the packets are being redirected to userspace
instead? What does xdp_monitor say?
-Toke
|