Service mesh
A service mesh is a dedicated infrastructure layer that manages how microservices talk to each other: encrypting traffic, retrying failed calls, balancing load, and recording what happened.
Istio and Linkerd are the best-known meshes. While a mesh promises to take networking headaches away from developers, it adds its own: another complex system to run, upgrade, and debug when things go wrong.
Service mesh, explained at three levels
At home
Imagine a large office where every department sends memos to the others. A service mesh is like a mailroom that seals every envelope, resends lost memos, and keeps a log of who sent what. Each department no longer has to handle that itself.
At work
In an app made of many small services, a service mesh secures and monitors the traffic between them. Teams get encryption, retries, and traffic dashboards without writing that code into every service.
At home
Traditionally, a mesh puts a small proxy, called a 'sidecar,' next to each service. All traffic goes through these proxies, which apply rules set by a central 'control plane.' Newer designs share proxies across a whole machine to save resources.
At work
Common mesh features include mutual TLS between services, fine-grained traffic routing for canary releases, automatic retries and timeouts, circuit breaking, and consistent metrics and traces. These features overlap with what API gateways, cloud load balancers, and Kubernetes itself increasingly offer, so check what you already have.
At home
The mesh splits into a data plane (the proxies handling every request) and a control plane (which pushes configuration to them). In my opinion, the data plane is where costs hide: each proxy adds memory use and a little latency to every call, which adds up across thousands of pods.
At work
Adopt a mesh for a specific need, usually zero-trust service-to-service encryption or advanced traffic management, not as a default. Evaluate sidecar versus sidecarless (ambient or eBPF-based) designs, upgrade cadence, multi-cluster support, and how failures in the mesh itself will be detected. A mesh outage can take down every service at once, so it needs the same operational rigour as your core network.
Real-world examples
At home
Carlos was studying for a cloud certification and set up Linkerd on a test cluster. Within minutes he could see live success rates and response times for every service, which made the course material on microservices finally feel concrete.
At work
Lena runs infrastructure at a Toronto bank that needed encrypted traffic between all internal services to satisfy auditors. Rather than change 150 services, her team rolled out a service mesh with mutual TLS, then spent the next quarter tuning its resource use.
How it compares
| feature | Service mesh | API gateway |
|---|---|---|
| Main job | Traffic between internal services | Traffic entering from outside |
| Typical location | Next to every service | At the edge of the system |
| Key features | Mutual TLS, retries, internal routing | Authentication, rate limiting, public API management |
| Who uses it | Platform and infrastructure teams | API and product teams |
Gateways guard the front door. Meshes manage the hallways inside.
For analysts
A service mesh solves real problems, but only at a certain scale. I'd ask for the specific requirement, such as encryption mandated by an auditor, before approving one, and budget for the people needed to run it.
History & origin
As companies split applications into dozens or hundreds of microservices in the 2010s, every service needed the same networking logic. Twitter and Netflix built shared libraries for it, but libraries tied you to one programming language. Buoyant released Linkerd in 2016, and its CEO, William Morgan, is generally credited with popularizing the term 'service mesh.' Lyft open-sourced the Envoy proxy that same year, and Google, IBM, and Lyft announced Istio, built on Envoy, in May 2017. Istio graduated within the Cloud Native Computing Foundation in 2023, and its 'ambient' mode, which drops the per-pod sidecar proxy, reached general availability in late 2024. In my view, the move away from sidecars is an admission that early meshes cost too much for what most teams needed.