Numbers to know · System Design Course

Rough capacities of a modern server and of the common building blocks, a few documented hard limits, and the rules of thumb that turn a number into a box on the diagram. Ranges to size a design, not benchmarks.

The estimation lesson gave you the arithmetic: powers of 1,000, seconds in a day, the nines. This lesson adds the other half: how much one machine and one component can take. Without it, you can compute 12,000 requests per second and still not know whether that needs one server or fifty.

Crux: know the rough capacity of one server and of each building block, to the nearest factor of ten, so you can tell when one box is enough and when a number forces a second.

Read this first: how reliable these numbers are

Be plain about this in a design discussion, too.

Every figure here is a range for sizing, not a benchmark. Real throughput changes by ten times or more with hardware, data size, query shape, network and configuration.
None of them was measured for this lesson. They are order-of-magnitude figures from common engineering experience, and the figures say so.
The hard limits in section 03 come from product documentation as it stood when this was written. Limits change, so check the current docs before you rely on one.
Prices and cloud instance sizes are left out on purpose: they change often and cannot be confirmed here.

The goal is a feel for the first digit and the number of zeros, not for the second digit.

One server, one component

Ranges for sizing. If your design needs ten times the top of a range, add machines. Below it, one is enough.

Two lessons hide in the figure.

A single modern server is large. Hundreds of gigabytes of memory and terabytes of fast local storage are routine. A dataset of a few hundred gigabytes fits in the memory of one machine. Many designs are less distributed than people expect, so say "one node is enough for X years" when the arithmetic backs it.
Components differ by orders of magnitude. An in-memory store does simple operations about ten times faster than a relational database does indexed reads, and a log such as Kafka is measured in bytes per second, not operations. Match the component to the load.

Time, refreshed

The estimation lesson used a 2012-era latency list. Today's hardware is faster in some places. Use these as a feel, not exact values:

Memory access: around 100 nanoseconds. Barely changed.
Fast SSD random read (4 KB): tens to about a hundred microseconds on modern NVMe drives. Older SATA SSDs are slower.
Round trip inside one datacenter: roughly half a millisecond to a millisecond.
Round trip across a continent: tens of milliseconds. Across an ocean: about 100 to 150 ms, and the speed of light in fibre sets a floor (see the networking lesson).

The ratios still hold: memory is far faster than SSD, SSD is far faster than a network hop, and a hop across an ocean is slower than all of them.

Hard limits and rules of thumb

Documented limits change the design directly: a 10 MB value will not fit in a Kafka message with default settings. The rules below are judgement calls.

Limits like these shape a design:

A video or image does not go through your database or queue. It goes to blob storage, with a pointer (a URL or key) in the record. Limits on message and item sizes are one reason.
A large item in a document store is a smell. A 400 KB item limit means a growing list stored inside one record will hit a wall. Split it into many items.
Settings can raise a default (a Kafka max message size can be raised), but a limit that exists for a reason is a sign to change the design instead.

A worked example: does the link shortener need more than one box?

Take the inputs from the estimation lesson (assumptions): 40 writes per second, 12,000 reads per second at peak, 240 GB of links per year, clicks at 4,000 per second.

Links storage:    240 GB a year      → one node holds it for years  (well under 1 TB)
Link writes:      40 / s             → one primary, trivial
Link reads:       12,000 / s peak    → a cache absorbs it (one Redis node: ~100,000 ops/s)
                                       90% hit rate: database sees 1,200 / s, fine
Click writes:     4,000 / s × 100 B  = 400 KB / s   → a log (Kafka) takes that easily
Click storage:    12.6 TB a year     → this is the only number that forces more than one node

Two tests tell you where the design must grow:

Throughput test: divide the peak rate by what one component handles. 12,000 ÷ 100,000 = 0.12 of one cache node, so one node (plus a replica for safety, not for load).
Size test: divide the data by what one node holds. 12.6 TB ÷ 2 TB ≈ 7 nodes for the first year (the 2 TB is an assumption).

The answer to "does it need sharding?" is: links, no; clicks, yes (or archive them). The numbers pointed at the same part of the design as the estimation lesson, which is the point.

Rules of thumb

Use these as a starting place and say they are judgement calls.

Data well under 1 TB: one database node. Replicas for safety and reads.
Writes in the low thousands per second: one primary can usually keep up. Tens of thousands means batching, a queue or sharding.
Reads far above writes: cache first, then read replicas.
Peak above average: design for the peak (assume 2 to 3 times the average, then ask).
Data or writes past one node: shard, and pick the key from the queries.
Large objects: blob storage plus a pointer.
A user waiting: keep the path to a few round trips and under a few hundred milliseconds in total.

Where the numbers change a decision, component by component

Cache. One node holds tens to hundreds of GB in memory and serves on the order of 10^5 simple operations a second. Check memory first, then throughput.
Database. One node holds terabytes. Indexed reads in the thousands to tens of thousands per second, writes in the low thousands. Past that, replicas, queues or shards.
App server. Stateless and cheap to add. The usual limit is connections or CPU per request, so ask how much work one request does.
Message queue. A log broker like Kafka is built for sequential writes and is commonly quoted in the tens to hundreds of MB a second per node (a rough range; check the figure above and your hardware). A queue is the answer to bursts of writes, not to slow reads.

Mistakes with numbers that cost interviews

Premature sharding. Splitting data that one node holds easily (a few hundred GB). Do the size and write-rate division first.
Overestimating latency. Treating a memory read or a same-datacenter hop as slow, and adding a cache or an extra layer that the numbers did not ask for. Equally, treating an ocean crossing as free.
Over-engineering for write throughput. Reaching for sharding or a queue at a few hundred writes per second. One primary handles that.
Ignoring cost. More boxes, more copies and more regions all have a price. Say the cheaper design first and what would make you leave it. (Prices are left out of this lesson on purpose.)

Mistakes in quoting numbers

Quoting a number as fact. Say "roughly" and give a range.
Treating one benchmark as universal, or a vendor's best case as typical.
Adding machines before checking whether one is enough.
Ignoring a hard limit until the design hits it.
Mixing bits and bytes: network speeds are in bits per second (1 Gbit/s is about 125 MB/s).
Memorising digits instead of ratios and powers of ten.

Check yourself

Roughly how many simple operations per second can one Redis node serve, and how sure should you sound?
A service needs 12,000 reads per second at peak. Does that alone force more than one database node? What would you try first?
6 TB of data and nodes that comfortably hold 2 TB. How many nodes, and what assumption did you make?
Why is a 5 MB document a poor fit for a store with a 400 KB item limit, and what do you do instead?
Name two ratios from the latency list that you would keep even if the digits changed.
Why were prices and instance sizes left out of this lesson?