On Wed, Sep 9, 2015 at 12:21 PM, Eric Leblond <eric@xxxxxxxxx> wrote: > Hello, > > Le 9 sept. 2015 8:27 AM, Akshat Kakkar <akshat.1984@xxxxxxxxx> a écrit : >> >> On Mon, Sep 7, 2015 at 5:45 PM, Akshat Kakkar <akshat.1984@xxxxxxxxx> wrote: >> > Hi! >> > >> > I am using JSON plugin of ULOG to log iptables traffic. When the input >> > rate is arond 4000 packets/sec, ulog is not logging all the packets. > > Please use nflog instead. Ulog has been removed from recent kernel. > >> > At 4000 pkts/sec, it drops around 0.1 % of pkts. >> > At 5000 pkts/sec, it drops around 2.4% of pkts. >> > At 6000 pkts/sec, it drops around 3.67% of pkts. >> > At 15000 pkts/sec, it drops around 16.7% of pkts. >> > >> > Is this expected? or am I missing something. > > This is expected, there is a performance cost in logging. > > Some ways to try to improve this: > - use nflog batch feature (iptables/nftables option) I am currently batching at maximum value of 50. But one thing, Why 50 is the limit? > - increase nfnetlink buffer size (ulogd conf) This increased the delay at times. > - write json file to a faster disk (I doubt this is the key point here) Presently I am having 15K sas. Putting json file on ramfs (ram: DDR3 1600MHz) makes that 4000 to 7000 but not more that. With o/p in PCAP on ramfs, this increases to 16000 pkts/sec. I want to use kibana (+logsash and elastic-search) for viewing and analyzing this log. Logstash dont understand pcap directly. So I cant use PCAP. Is there a format which is understood by logstash and is better (in performance) than JSON for ulog/nflog. -- To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
