Service mesh
A service mesh is a dedicated infrastructure layer that manages how microservices talk to each other: encrypting traffic, retrying failed calls, balancing load, and recording what happened.
Istio and Linkerd are the best-known meshes. While a mesh promises to take networking headaches away from developers, it adds its own: another complex system to run, upgrade, and debug when things go wrong.
Service mesh, explained at three levels
At home
Think of a large office where departments constantly send memos. A service mesh is like a mailroom that sits between them: it seals every envelope, resends memos that get lost, routes them to the right desk, and keeps a log of everything. In software, the departments are small services that make up an app, and the mesh handles their messages so each service doesn't have to.
At work
A service mesh is infrastructure that manages communication between the many small services, or microservices, inside an application. It encrypts traffic, retries failed requests, balances load, and records metrics, all without developers adding that code to each service. Businesses care because it delivers consistent security and visibility across hundreds of services, though it adds another complex system that someone has to run.
At home
A mesh has two parts. The data plane is a set of small proxies, usually Envoy or Linkerd's own proxy, that every request passes through. Traditionally each service gets its own proxy, called a sidecar, which intercepts its traffic. The control plane is a central component that tells all those proxies what to do, such as which traffic to encrypt or how many times to retry. Newer designs share one proxy per machine instead of one per service.
At work
In practice, a mesh gives each service a cryptographic identity, then uses mutual TLS so services prove who they are to each other and encrypt everything in transit. The control plane distributes routing rules, so you can send 5% of traffic to a new version for a canary release, set timeouts and retries, and break circuits to failing services. Proxies emit uniform metrics and traces, giving consistent visibility without changing application code.
At home
Every request in a sidecar mesh makes two extra hops, out through one proxy and in through another, and every pod carries a proxy using its own memory. Across thousands of pods, that overhead adds up. While sidecars give each service fine-grained control, in my opinion that cost is why newer designs such as Istio's ambient mode, which uses a shared proxy per node plus optional proxies for advanced routing, are the better default for most teams.
At work
Meshes differ mainly in data-plane design: Envoy sidecars in classic Istio, a lightweight Rust proxy in Linkerd, a per-node ztunnel plus optional waypoint proxies in Istio's ambient mode, and eBPF-assisted datapaths in Cilium. Workload identity typically follows the SPIFFE standard. Key limits are latency, resource overhead, and the mesh as a shared failure point: a bad control-plane push or an expired certificate can break every service at once. Adopt one for a specific requirement, and run it like core network infrastructure.
Real-world examples
At home
Carlos was studying for a cloud certification and set up Linkerd on a test cluster. Within minutes he could see live success rates and response times for every service, which made the course material on microservices finally feel concrete.
At work
Lena runs infrastructure at a Toronto bank that needed encrypted traffic between all internal services to satisfy auditors. Rather than change 150 services, her team rolled out a service mesh with mutual TLS, then spent the next quarter tuning its resource use.
How it compares
| feature | Service mesh | API gateway |
|---|---|---|
| Main job | Traffic between internal services | Traffic entering from outside |
| Typical location | Next to every service | At the edge of the system |
| Key features | Mutual TLS, retries, internal routing | Authentication, rate limiting, public API management |
| Who uses it | Platform and infrastructure teams | API and product teams |
Gateways guard the front door. Meshes manage the hallways inside.
For analysts
A service mesh solves real problems, but only at a certain scale. I'd ask for the specific requirement, such as encryption mandated by an auditor, before approving one, and budget for the people needed to run it.
History & origin
As companies split applications into dozens or hundreds of microservices in the 2010s, every service needed the same networking logic. Twitter and Netflix built shared libraries for it, but libraries tied you to one programming language. Buoyant released Linkerd in 2016, and its CEO, William Morgan, is generally credited with popularizing the term 'service mesh.' Lyft open-sourced the Envoy proxy that same year, and Google, IBM, and Lyft announced Istio, built on Envoy, in May 2017. Istio graduated within the Cloud Native Computing Foundation in 2023, and its 'ambient' mode, which drops the per-pod sidecar proxy, reached general availability in late 2024. In my view, the move away from sidecars is an admission that early meshes cost too much for what most teams needed.