Hi, My application (Project Calico) makes heavy use of IP sets for firewall configuration and, to update the IP sets quickly, we use "ipset restore" for bulk operations. After running a churn test on Calico that adds and removes a lot of IPs from IP sets, it seems that IP sets can become inconsistent. We add an IP in one call to ipset, then try to remove it later but ipset says it isn't in the set (or vice versa). The ipset call that adds the IP returns a good status code. To remove some variables, I recorded a trace of every ipset call that Calico ran as a shell script. Sometimes the script [1] runs to completion (so I think the script is consistent) but it typically fails at a random point with something like "ipset v6.29: Error in line 66: Element cannot be deleted from the set: it's not added". I've tried to boil it down to a smaller repro but I haven't come up with anything that hits the problem as reliably as the recorded script. (I tried a python script that adds ~1000 IPs to a set and then churns them at random; I think I hit the issue once but haven't been able to hit it again since.) Any help to diagnose or work around or a fix would be appreciated. Thanks, -Shaun [1] https://transfer.sh/13zUTe/repro.sh -- To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
