Gateways, proxies and service meshes
What belongs at the edge, what belongs beside each service, and what a mesh actually buys.
What an API gateway is for
intermediateA gateway is the single front door to a set of services, and its job is the work that would otherwise be repeated in each of them: terminating TLS, authenticating the caller, applying rate limits, routing by path or host, and emitting consistent logs and metrics. Doing those once, correctly, is worth more than doing them eleven times with three subtle differences.
The clearest argument for it is security. Authentication implemented per service is authentication implemented inconsistently, and the service that gets it wrong is the one nobody remembers exists. Terminating at the gateway means one implementation to audit, one place to rotate keys, and a uniform answer to who is calling, which the services behind it can then trust rather than reimplement.
The clearest argument against a large one is that gateways attract logic. Request transformation, response shaping, per-client special cases, a small amount of business rule that had nowhere else to go, and eventually the gateway is a shared component that every team must change and nobody owns. That is the enterprise service bus reborn with a modern name, and the reason to state the boundary explicitly: routing, authentication, limits and observability, and no domain logic at all.
Aggregation is the case that genuinely justifies more. A mobile client on a slow network making eleven calls to render one screen is paying eleven round trips, and a backend for frontends, a gateway specific to that client, can make those calls internally and return one response. The important part is that it is owned by the client team and is one per client type, rather than a shared layer with every client's needs in it.
The failure to plan for is that the gateway is now on the path of everything. It must be stateless and horizontally scaled, its configuration must be deployable without a restart, and its own failure has to be understood, because a gateway that is down is an outage of everything behind it regardless of how healthy those services are.
The gateway exists to make cross-cutting concerns consistent, and it earns its place exactly as long as it holds that line. Every piece of domain logic added to it converts a component nobody has to think about into one that every team must coordinate through.
The backend for frontends pattern, one gateway per client type owned by that client's team, is the version that survives, because it puts the aggregation where its requirements come from rather than in a shared layer with every client's needs mixed together.
Why is per-service authentication worse than terminating at the gateway?
Service meshes, and whether you need one
advancedA mesh moves the concerns of service-to-service communication out of the application and into a proxy deployed beside each instance. Retries, timeouts, circuit breaking, mutual TLS, traffic splitting and per-call telemetry become configuration rather than library code, and crucially they become uniform across services written in different languages by different teams.
That last point is the honest argument. In a single-language estate, a shared client library does most of this, and a library is simpler to operate than a fleet of proxies. In a polyglot estate, a library means implementing and maintaining the same behaviour four times, which is where the sidecar's per-instance cost starts looking cheap.
The features that are genuinely hard to get any other way are mutual TLS everywhere with automatic certificate rotation, consistent traffic shifting for canaries at the network level, and telemetry that is identical across services because it is emitted by the same proxy. Each of those is possible without a mesh and each is fiddly enough that most teams do not do it consistently.
The cost is a second network to understand. A proxy per instance is CPU, memory and a latency addition on every hop, the control plane is a new dependency in the request path's configuration, and debugging now involves asking whether the problem is the application, its sidecar, the other sidecar or the control plane. That is a genuine increase in the number of places a failure can live.
So the honest test is the number of services and the number of languages. A handful of services in one language does not need a mesh and will feel the cost immediately. Fifty services in four languages, with a platform team to run it, is where the uniformity pays for the complexity. Everything in between is a judgement, and the sidecar-free variants that push the same functions into the kernel are a live attempt to lower the price.
A mesh buys uniformity across services you do not control and cannot rewrite, which is worth a great deal at fifty services and nothing at five. The question is not whether the features are useful but whether a library would give you them for less.
Istio and Linkerd are the common implementations, and the recent direction of travel is toward reducing the per-pod cost, with ambient and sidecar-free modes moving functions out of a proxy per instance. That effort exists because the sidecar tax was the main objection.
What does a mesh add to the debugging surface?
The edge and the interior are different problems
intermediateTraffic entering the estate and traffic moving inside it have almost nothing in common, and conflating them produces designs that are wrong at one end. External traffic comes from clients you do not control, over networks you cannot measure, with credentials that may be stolen and volumes that may be hostile. Internal traffic comes from services you deployed, in a network you configured, and every caller has an identity you issued.
So the concerns differ. At the edge: authentication of untrusted callers, per-tenant rate limiting, request size limits, bot mitigation, TLS termination and caching for anonymous traffic. Inside: identity between services, timeouts and retries with budgets, circuit breaking, load balancing that understands long-lived connections, and tracing that spans the whole call tree.
Zero trust is the idea that the interior should not be treated as safe simply because it is the interior. A flat internal network where any service can call any other with no identity means one compromised component reaches everything, which is how a modest breach becomes a total one. Mutual TLS and per-service authorisation make lateral movement a series of separate obstacles rather than one.
The mistake in the other direction is applying edge machinery inside. A heavyweight gateway between every pair of internal services adds a hop, a failure domain and a bottleneck to calls that were fine. Internal calls want the lightest thing that provides identity, timeouts and telemetry, which is the argument for a sidecar or a library rather than for a second gateway.
The practical layering that most estates converge on is a CDN and a public gateway at the edge for untrusted traffic, and a mesh or a shared library inside for identity and resilience between services. Naming which layer a requirement belongs to is usually enough to settle an argument about where a feature should live.
The two directions differ in who the caller is and what can be trusted about them, and every feature decision follows from that. The most common design error is a single layer expected to serve both, which ends up too heavy for internal calls and too naive for external ones.
Zero trust guidance from NIST and others makes the same argument for the interior: authenticate every call regardless of network position, because a flat trusted network turns one compromised component into access to everything on it.
Which concern belongs at the edge rather than inside?