API gateway

Once a system has several services, clients should not need to know where each one lives. An API gateway is the one entry point that takes the request, does the common checks, and sends it to the right place.

Crux: put cross-cutting concerns (auth, rate limits, routing, TLS) in one front door, keep business logic out of it, and do not spend much interview time here.

What it is and why it exists

An API gateway is a specialised reverse proxy: a server that sits in front of your backend and accepts requests on its behalf. It gives clients one address and one set of rules. Without it, every service must implement authentication, throttling and logging again.

How it works, at the level you must explain

For each request the gateway typically:

• Terminates TLS. The secure connection ends here and traffic goes on inside your network.
• Authenticates. It validates the token (a JWT or an API key) and passes the caller's identity on to services in a header. Services should trust identity from the gateway, never from the request body.
• Authorises coarse access, such as "this caller may use this API".
• Rate limits by user, key or IP, commonly with a token bucket kept in a shared store such as Redis.
• Routes by path, host or header to the right service, and can do versioned routes.
• Transforms the request or response where needed, for example between protocols.
• Logs, traces and measures every call, which gives one place to see traffic.
• Caches some responses, and can short-circuit with a clear error when a service is down (a circuit breaker).

A related idea is the backend for frontend (BFF): a gateway-like layer per client type (mobile, web) that shapes data for each. Use it when clients need quite different responses.

How it relates to a load balancer

A gateway understands the API (paths, auth, limits). A load balancer spreads traffic over identical servers. They are often both present: the gateway as a layer 7 front door, load balancers in front of it and in front of each service group. Managed gateways, such as AWS API Gateway, bundle some load balancing. Self-run options include Kong, NGINX and Envoy.

The request lifecycle, in order

• 1. Validate and authenticate. Reject malformed or unauthenticated requests early, cheaply.
• 2. Apply middleware. Rate limits, quotas, header and logging rules, and any request rewriting.
• 3. Route. Match the path, host or method against a routing table and pick the target service. This is the core job.
• 4. Call the backend. Forward with a timeout, and retry only safe (idempotent) requests. Open a circuit breaker if the service keeps failing.
• 5. Transform the response. Strip internal headers, change formats, or translate between protocols (for example, public JSON over HTTP outside, a binary RPC protocol such as gRPC between the gateway and services).
• 6. Cache where the answer is identical for everyone. Never cache a response that depends on the caller.

A routing rule looks like this: requests whose path starts with /orders go to the order service pool, and /search goes to the search pool. A change in the table is a deploy of the gateway, not of every service.

Scaling and placement

• The gateway keeps no per-request state, so you run several instances behind a load balancer and add more as traffic grows.
• Deploy it per region and send each user to the nearest, with the region failing over to another.
• A managed gateway (from your cloud) saves operations work but costs more per request and limits you to its features. A self-run gateway (Kong, NGINX, Envoy) gives control and costs operations effort. I leave out prices.
• Whether the gateway or a separate load balancer does the balancing between service instances is a detail. Say it is handled, and move on.

What to say in the interview

• Lead with routing: "clients call one address, and the gateway sends each request to the right service". Then add auth and rate limits in a clause.
• Propose it when the design has several services. For a single service or a simple client-server app, leave it out.
• Spend a few sentences and return to the part of the design the question is about.

When to use it, trade-offs, numbers

• What: the single entry point that applies shared rules and routes to services.
• Use when: more than one service, public APIs, shared auth and rate limits. In most product designs it is the first box after the client.
• Know: TLS termination, token validation, rate limiting, routing, and the single point of failure it creates unless it is run as several instances.
• Careful: do not put business logic in it, and remember it adds a hop of latency. A simple one-service app may not need a separate gateway.

Gateway or something else?

• Just a load balancer: pick it for a single service with simple needs.
• Service mesh (for example Istio): handles service-to-service traffic inside the system (mutual TLS, retries). A gateway handles traffic coming in from outside.
• A framework's own middleware: fine for a monolith.

Numbers: I am leaving out managed-gateway timeouts, payload limits and prices. They differ by product and change, so check current documentation if one decides your design. A safe statement is that the gateway adds a small, usually single-digit milliseconds, hop.

Common mistakes

• Putting business rules in the gateway, so it becomes a hidden monolith.
• Running one instance, so the front door is a single point of failure.
• Trusting the user id from the request body instead of the validated token.
• Doing heavy work such as large body transformations in it.
• Spending ten minutes on it. It is rarely probed in depth.

• Describing only middleware and never saying that it routes requests.
• Putting it into a design with one service.
• Caching a response that is specific to the user.
• Retrying non-idempotent requests at the gateway.

Check yourself

• Name five things a gateway does for every request.
• How does a service learn who the caller is?
• How does an API gateway differ from a load balancer?
• Where would rate-limit counters be stored, and why?
• What is a backend for frontend?
• What are the risks of putting logic in the gateway?
• List the steps of a request through a gateway in order.
• Why can you run many gateway instances at once?
• When would you leave the gateway out of a design?