Why Cilium
A hash map beats a linked list when the list has three thousand entries. That is the entire Cilium argument, and nobody explains it that way. The marketing says eBPF is the future. The mechanism is a data structure, and the difference shows up on a busy node at fleet scale.
To be fair to the old way, it worked for a long time. kube-proxy in iptables mode shipped a lot of clusters, and for a hundred services it's fine. The problems show up at fleet scale, and they show up in ways that look like application bugs until you know what to look for.
The old way, a list you walk
Every Service in the cluster becomes a set of iptables rules. clusterIP to pod IPs, DNAT, conntrack bookkeeping. A packet arrives at a node, and the kernel walks the chains in order until it finds a match. With a hundred services, that's a hundred rule steps. With a thousand, a thousand. With three thousand, you're walking a very long list for every packet.
Two costs grow with scale, and they grow together.
The first is CPU. Every packet that arrives walks the chain, and the walk gets longer as the fleet grows. On a busy node, that's real burn, and it's invisible on every dashboard that graphs "pod CPU" because it's kernel time, not pod time.
The second is churn. In older kube-proxy versions, every Service and pod change meant a full iptables ruleset rewrite with iptables-restore, a slow operation that takes a lock, and on large clusters the update window was long enough to drop or delay new connections. That's the random one to two second latency spike, or the connection refused right after a deploy, that everyone blames on the application. Kubernetes 1.28 changed this. Iptables mode now makes minimal updates, touching only the Services and EndpointSlices that actually changed, so the churn is much smaller on modern clusters.
There's a third, quieter one. Conntrack is a table, and when it fills, packets get dropped. It fills faster when the node is doing more work, which is exactly when you can least afford it.
The eBPF way, a map you look up
Cilium does not replace CNI, it implements CNI, and it implements the data path differently. Instead of rule chains, BPF programs run at the kernel packet hooks and consult BPF maps. Hash lookups, average O(1) per packet. The number of services in the cluster does not change the cost of the lookup, which means the per packet cost stays flat as the fleet grows. One caveat worth knowing. The maps have capacity limits, and a node that outgrows them needs attention, but normal fleet scale is nowhere near that edge.
The other half is how state gets in. No iptables-restore, no lock, no drop window. When Kubernetes objects change, the events push state into the maps. Event driven, not rewrite driven. A deploy that creates two hundred Services at once is a map update, not a rule swap.
The packet path becomes a lookup. Look up the service, get the backend, DNAT, forward. Same work, constant cost.
What you notice first
On a busy iptables node, three symptoms appear that you will never trace to the CNI without knowing the mechanism. CPU creep that tracks service count, latency jitter right after churn, and dropped connections during rule rewrites. On Cilium, all three just don't show up. Not because the node is faster, because the work is different.
The rest of Cilium is real and worth having, the L7 policy without sidecars, the cluster mesh for multi cluster, Hubble for traffic flow visibility, an ingress controller, IPAM. But those are features. The data structure is the reason the features hold up at scale.
One correction most people make
It's common to say eBPF wins because everything lives in the kernel. iptables also lives in the kernel. The location was never the difference. The difference is the lookup, a walk versus a map, and the churn model, a rewrite versus an event. Say it that way and the whole thing makes sense.
The boring conclusion
You don't adopt Cilium because eBPF is trendy. You adopt it because a hash map beats a linked list when the list has three thousand entries, and because map updates beat rule swaps when a deploy touches everything at once. The features are the headline. The data structure is the product.
What's the network bug you chased for hours that turned out to be iptables all along?