On 12/May/11 11:40, nowhere wrote:
>> It seems enough to avoid delaying the call to nfq_set_verdict for the
>> first packet of a burst. For a shot in the dark, packets seem to get
>> lost if they arrive between the first one and the corresponding call
>> to nfq_set_verdict. Indeed, setting a fixed real_delay of 0.2, with
>> ping -i 0.2 it looses no packets, with ping -i 0.19 it looses just the
>> second one, with ping -i 0.09 icmp_reqs #2 and #3.
>>
>> No error is returned, whether NETLINK_NO_ENOBUFS is set or not.
>
> Well, seems like this is the case. If nfqueue becomes empty, first
> enqueued packet must not be delayed.
I retract, possibly I've been too hasty blaming nfnetlink queue. I
made a simple variation of nfqnl_test.c --which I attach. It just
accepts the previous packet id. The "last" packet is obviously always
lost. Because of this bug(?), I also loose the second packet of a
sequence of pings, no matter the speed.
However, if I "ping -c 1" using two terminal windows, I correctly
receive all odd ids in one window and even ones in the other (except
last pkt). In this case, I delay every packet. Also, if I run a
sequence from a window, and, immediately after it starts, run a single
ping using the other window, then both the single ping and the
sequence (except last pkt) go correctly through.
I don't understand how come the kernel+filter system can distinguish
between a second packet coming as part of a sequence and a second
packet coming asynchronously, given that packets are not inspected.
Nice puzzle, isn't it?
NB, I used iptables -t mangle -A POSTROUTING -p icmp -d 172.25.197.158
-j NFQUEUE --queue-num 13, as in
http://www.spinics.net/lists/netfilter/msg50829.html
/*
$ ping -c 8 172.25.197.158
PING 172.25.197.158 (172.25.197.158) 56(84) bytes of data.
64 bytes from 172.25.197.158: icmp_req=1 ttl=128 time=1010 ms
64 bytes from 172.25.197.158: icmp_req=3 ttl=128 time=1000 ms
64 bytes from 172.25.197.158: icmp_req=4 ttl=128 time=1000 ms
64 bytes from 172.25.197.158: icmp_req=5 ttl=128 time=999 ms
64 bytes from 172.25.197.158: icmp_req=6 ttl=128 time=1000 ms
64 bytes from 172.25.197.158: icmp_req=7 ttl=128 time=1010 ms
--- 172.25.197.158 ping statistics ---
8 packets transmitted, 6 received, 25% packet loss, time 7020ms
rtt min/avg/max/mdev = 999.452/1003.574/1010.633/4.883 ms, pipe 2
# grep -E ^ *13 /proc/net/netfilter/nfnetlink_queue
13 13584 1 1 0 0 0 8 1
^queue_num,
^peer_pid,
^queue_total,
^copy_mode,
^copy_range,
^queue_dropped,
^queue_user_dropped,
^id_sequence,
^1
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <netinet/in.h>
#include <linux/types.h>
#include <linux/netfilter.h> /* for NF_ACCEPT */
#include <libnetfilter_queue/libnetfilter_queue.h>
#include <errno.h>
struct packet_data
{
u_int32_t id, good;
};
static int cb(struct nfq_q_handle *qh, struct nfgenmsg *nu,
struct nfq_data *nfa, void *data)
{
struct packet_data *pd = (struct packet_data*)data;
struct nfqnl_msg_packet_hdr *ph = nfq_get_msg_packet_hdr(nfa);
if (pd && ph)
{
if (pd->good) // since last time
nfq_set_verdict(qh, pd->id, NF_ACCEPT, 0, NULL);
pd->id = ntohl(ph->packet_id);
pd->good = 1;
printf("received packet %d\n", pd->id);
}
return 0;
(void)nu;
}
int main()
{
struct nfq_handle *h;
struct nfq_q_handle *qh;
struct packet_data pd;
int fd;
int rv;
char buf[4096] __attribute__ ((aligned));
printf("opening library handle\n");
h = nfq_open();
if (!h) {
fprintf(stderr, "error during nfq_open()\n");
exit(1);
}
printf("unbinding existing nf_queue handler for AF_INET (if any)\n");
if (nfq_unbind_pf(h, AF_INET) < 0) {
fprintf(stderr, "error during nfq_unbind_pf()\n");
exit(1);
}
printf("binding nfnetlink_queue as nf_queue handler for AF_INET\n");
if (nfq_bind_pf(h, AF_INET) < 0) {
fprintf(stderr, "error during nfq_bind_pf()\n");
exit(1);
}
pd.good = 0;
qh = nfq_create_queue(h, 13 /* <--queue number here */, &cb, &pd);
if (!qh) {
fprintf(stderr, "error during nfq_create_queue()\n");
exit(1);
}
if (nfq_set_mode(qh, NFQNL_COPY_META, 0) < 0) {
fprintf(stderr, "can't set packet_copy mode\n");
exit(1);
}
fd = nfq_fd(h);
for (;;) {
if ((rv = recv(fd, buf, sizeof(buf), 0)) >= 0) {
nfq_handle_packet(h, buf, rv);
continue;
}
/* if the computer is slower than the network the buffer
* may fill up. Depending on the application, this error
* may be ignored */
if (errno == ENOBUFS) {
printf("pkt lost!!\n");
continue;
}
printf("recv failed: errno=%d (%s)\n",
errno, strerror(errno));
}
printf("unbinding from queue 0\n");
nfq_destroy_queue(qh);
#ifdef INSANE
/* normally, applications SHOULD NOT issue this command, since
* it detaches other programs/sockets from AF_INET, too ! */
printf("unbinding from AF_INET\n");
nfq_unbind_pf(h, AF_INET);
#endif
printf("closing library handle\n");
nfq_close(h);
exit(0);
}
