Hi, On Fri, 17 Feb 2017, Shaun Crampton wrote: > My application (Project Calico) makes heavy use of IP sets for > firewall configuration and, to update the IP sets quickly, we use > "ipset restore" for bulk operations. After running a churn test on > Calico that adds and removes a lot of IPs from IP sets, it seems that > IP sets can become inconsistent. We add an IP in one call to ipset, > then try to remove it later but ipset says it isn't in the set (or > vice versa). The ipset call that adds the IP returns a good status > code. > > To remove some variables, I recorded a trace of every ipset call that > Calico ran as a shell script. Sometimes the script [1] runs to > completion (so I think the script is consistent) but it typically > fails at a random point with something like "ipset v6.29: Error in > line 66: Element cannot be deleted from the set: it's not added". That's a bug and was reported just a few days ago, see https://bugzilla.netfilter.org/show_bug.cgi?id=1119. The fix is out in the ipset git tree and I'll release a new version at the weekend. Best regards, Jozsef - E-mail : kadlec@xxxxxxxxxxxxxxxxx, kadlecsik.jozsef@xxxxxxxxxxxxx PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences H-1525 Budapest 114, POB. 49, Hungary -- To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html