On Tue, 29 Jun 2021 10:37:18 +0200 Pablo Neira Ayuso <pablo@xxxxxxxxxxxxx> wrote: > On Mon, Jun 28, 2021 at 10:02:41PM -0400, Neal P. Murphy wrote: > > On Mon, 28 Jun 2021 10:43:10 +0100 > > Kerin Millar <kfm@xxxxxxxxxxxxx> wrote: > > > > > Now you benefit from atomicity (the rules will either be committed at once, in full, or not at all) and proper error handling (the exit status value of iptables-restore is meaningful and acted upon). Further, should you prefer to indent the body of the heredoc, you may write <<-EOF, though only leading tab characters will be stripped out. > > > > > > > [minor digression] > > > > Is iptables-restore truly atomic in *all* cases? > > Packets either see the old table or the new table, no intermediate > ruleset state is exposed to packet path. > > > Some years ago, I found through experimentation that some rules were > > 'lost' when restoring more than 25 000 rules. > > Could you specify kernel and userspace versions? Rules are not 'lost' > when restoring large rulesets. This goes back to late '10, early '11: linux 2.6.35, iptables 1.4.10; possibly even earlier releases. I don't recall trying the experiment since. > > > If I placed a COMMIT every 20 000 rules or so, then all rules would > > be properly loaded. I think COMMITs break atomicity. > > Why are you placing COMMIT in every few rules 20 000 rules? If I recall correctly, when I attempted to restore/load more than 'N' rules or so back then, a few rules at that 'N' 'boundary' would not be in the final iptables ruleset. IIRC, it didn't matter how quickly or slowly I added the rules. By including a COMMIT every 20k rules or so, all of the rules I loaded would be added, be it 10k or 100k (or more). There was also a vmalloc failure up around 130k-150k rules. A couple references: - https://marc.info/?l=netfilter&m=133814281919741&w=2 - https://community.smoothwall.org/forum/viewtopic.php?p=292390#p292390 > > > I tested with 100k to 1M rules. > > iptables is handling very large rulesets already. > > > I was comparing the efficiency of iptables-restore with another tool > > that read from STDIN; the other tool was about 5% more efficient. > > Could you please specify what other tool are you refering to? The tool is ipbatch (part of Smoothwall Express). It was written because, at least back then, iptables was not able to read from STDIN. Smoothwall Express reads its config/settings files and generates the firewall rules on the fly and pipes them to ipbatch. Also, the rules are generated by function/application, not by table; iptables-restore would not be usable without major changes to the way Smoothwall Express generates its firewall rules. Neal
