Common patterns: an overview

Most new problems are old problems in disguise. A chat app, a ticket site and a video service look different, but each leans on the same few shapes. Learn the shapes, and a fresh prompt stops being fresh.

Crux: recognise the situation first, then reach for the pattern. Start with the simplest version and add complexity only when a requirement forces it.

Why patterns help

A pattern gives you three things:

• Recognition. You can say "this is a contention problem" and know where to look.
• A fallback. When you are stuck, you have a known good approach to start from.
• A list of failure modes. Every pattern comes with its usual ways of going wrong, so you can raise them before you are asked.

Patterns also combine. Real systems use several at once, and each breakdown later in this course names the ones it uses.

The eight patterns

1. Pushing real-time updates

• The problem: show live data to users, as in chat, notifications or dashboards.
• Options: polling is simplest and least efficient. Server-sent events and WebSockets are made for this, and harder to run. On the server side, pub/sub decouples publishers from subscribers (a chat app), while stateful servers on a consistent hash ring handle heavier work (a shared document editor).
• Start with: polling. Move to server-sent events or WebSockets only when you need to.

2. Managing long-running tasks

• The problem: work that is too slow to finish inside one request, such as video encoding or a big report.
• The shape: validate the request, put a job on a queue, return a job ID straight away, and let a pool of workers do the work. Track status, retry failures and send poison messages to a dead-letter queue.
• Careful: do not reach for a queue too quickly. If the job takes only a few seconds, doing it in the request is simpler and gives the user a better experience.

3. Dealing with contention

• The problem: many users compete for one resource, such as the last ticket or the top auction bid.
• Options: pessimistic locking, optimistic concurrency, distributed locks, two-phase commit, or serialising writes through a queue.
• Trade-off: speed against correctness, and one database against distributed coordination.
• Start with: one database. Splitting data early means taking on problems that a database already solves.

4. Scaling reads

• The problem: reads are usually the first thing to outgrow the system.
• The ladder: better indexes and a little denormalisation, then read replicas, then caches and a CDN.
• Watch for: stale cache entries, replication lag and hot keys.
• A rule of thumb: reads often outnumber writes by about 10 to 1, and often 100 to 1 or more. That ratio is why this pattern comes up so often.

5. Scaling writes

• The problem: one database eventually cannot take more writes.
• Options: shard the data, partition it vertically, absorb bursts with a write queue, shed load to protect the important writes, and batch small writes.
• The key decision: a partition key that spreads load evenly while keeping related data together.

6. Handling large blobs

• The problem: large files (video, images, documents) should not pass through your app servers.
• The shape: give the client a presigned URL and let it upload straight to blob storage. Serve downloads through a CDN.
• Watch for: keeping the database record in step with the stored file, uploads that fail halfway, and cleaning up old files.

7. Multi-step processes

• The problem: a workflow across several services (an order, an onboarding, a payment) that must survive crashes and retries.
• Options: orchestrate from one server, use a workflow engine such as Temporal or AWS Step Functions, or use event sourcing, where each step emits an event that triggers the next.
• Careful: a step can run more than once after a retry, so make each step idempotent.

8. Proximity-based services

• The problem: find entities by location, as in a ride-hailing or delivery app.
• The shape: a geospatial index (for example PostGIS, Redis geo types or Elasticsearch geo queries), which lets you skip most of the map.
• Careful: an index pays off at hundreds of thousands to millions of items. For around a thousand, scanning them all is simpler.

Patterns combine

Take a video platform. One product, four patterns:

• Large blobs: the client uploads the raw video straight to blob storage.
• Long-running tasks: the upload puts a transcode job on a queue, and workers process it.
• Real-time updates: progress events are pushed back to the client.
• Multi-step process: one workflow coordinates the steps and retries failures.

When you meet a new problem, ask which of the eight apply, and in what order the request meets them.

Choosing: start simple

• Polling before WebSockets.
• A single database before sharding.
• A synchronous call before a queue.
• One server orchestrating before a workflow engine.
• A scan before a geospatial index.

Add the next rung only when a requirement or a number says you must. That is the same habit as the delivery framework: name the complexity, defer it, and keep the basic path working.

Common mistakes

• Naming a pattern before understanding the problem.
• Reaching for the heaviest option first.
• Forgetting the failure modes: stale caches, hot keys, lost uploads, duplicate steps.
• Treating patterns as separate, when a real design combines several.

Check yourself

• What is the simplest way to push updates to a client, and when would you upgrade it?
• When is a queue the wrong answer for slow work?
• Name three ways to handle contention on one resource.
• What is the usual order for scaling reads?
• Why must steps in a multi-step process be idempotent?
• Which patterns does a video platform combine?