I want to tell you about something I can’t stop thinking about.

Last year I watched a Software Engineer bomb a behavioural interview. He was smart, had 12 years of experience, and did really well in the previous rounds.

He presented a good story, too. He led a platform migration, handled stakeholder pushback, and managed to deliver on time.

Then my peer interviewer asked: “What was the measurable impact of this project?”

The candidate froze, rambled for 90 seconds, and then said something like: “We delivered it on time.”

Ultimately, he didn’t get the job... and that job paid £120k.

I’ve seen this happen over and over. Experienced people who know their stuff but can’t show it during an interview. They have enough stories but they just haven’t practised saying them out loud in a way that lands to interviewers.

Coaching with a human helps, but at £150-200/hour, you cover maybe 4-5 questions per session. You need 5-10 reps on each question before you actually get good at it. Do the maths.. It gets expensive fast.

So I built something.

It’s a voice AI interviewer. You pick a question, talk out loud, and it’s trained to come back at you the way a real interviewer would. “How did you measure success?” “What would you do differently?”

After each answer, it scores you and tells you where you dropped marks.

130+ questions pulled from real interviews at Google, Amazon, Meta, and Stripe, or you can create your own. It’s only £25/month and you can practise anytime - even at 11pm the night before your interview if you need to.

Your first question is free. No credit card required.

→ Try it here: https://www.em-tools.io/interview-prep

If you’re not job hunting right now, send this to someone who is. Nobody should lose a job in this market over a question they could have practised.

Stephane

P.S. Everyone who’s tried it so far scores 3-4 out of 10 on their first go, then 7-8 by their fifth. The gap between rehearsing in your head and practising out loud is bigger than you’d think.

Metrics

Metrics are numerical measurements collected at regular intervals. CPU usage, memory consumption, request count, error rate, response time.

They answer one question: “How is the system doing right now?”

Metrics are compact. A single data point is just a timestamp and a number, which means you can collect and store millions of them efficiently. This makes metrics ideal for dashboards, alerting, and trend analysis.

The “Four Golden Signals” are all metrics:

• Latency - how long requests take

• Traffic - request volume

• Errors - failure rate

• Saturation - resource utilization

When an alert fires, it’s almost always because a metric crossed a threshold.

Limitation: Metrics tell you that something is wrong, not what is wrong. A spike in error rate doesn’t tell you which errors, which users, or which code path.

Logs

Logs are records of discrete events. A user logged in. A request was received. An error was thrown. A configuration changed.

They answer a different question: “What happened?”

Logs contain rich detail: timestamps, user IDs, request parameters, stack traces, error messages. This detail is essential for debugging. When you need to know exactly what went wrong for a specific request, you look at logs.

Limitation: Logs are verbose. A busy system generates enormous volumes of log data. Storing, indexing, and searching logs at scale is expensive. Unstructured logs make analysis harder. High log generation rates can even impact application performance.

The typical workflow: use metrics to detect a problem, then query logs to understand what happened.

Traces

Traces track a single request as it flows through a distributed system. When one user action triggers calls across multiple services, a trace shows the complete path.

They answer: “How did this request move through the system?”

A trace is a collection of spans. Each span represents a unit of work: an HTTP request, a database query, a message processed from a queue. Spans include timing information, so you can see exactly where time was spent.

Traces are essential for debugging latency in microservices. When a request is slow, a trace shows you which service or operation is the bottleneck.

Limitation: Traces require instrumentation. You need to propagate trace IDs through your code and across service boundaries. Tracing also generates significant data volume, particularly in high-throughput systems. Sampling is often necessary, which means you won’t capture every request.

How They Work Together

Each pillar has a role:

Metrics detect problems. Your error rate spiked. Response times are climbing. Something is wrong.

Logs explain problems. The database connection failed. A null pointer exception occurred. The config was invalid.

Traces locate problems in distributed systems. The request was fine until it reached the payment service, which called inventory, which timed out.

In practice, you move between all three. Metrics for detection and alerting. Logs for detailed investigation. Traces for understanding request flow across services.

In short

Metrics are efficient and great for alerting but lack detail. Logs are detailed but expensive at scale. Traces show request flow but require instrumentation.

Use metrics to know something is wrong. Use logs to understand what went wrong. Use traces to see where it went wrong.

The three pillars complement each other. Relying on just one leaves gaps in your ability to understand and debug your systems.

Like posts like this?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

Monolithic Architecture

A monolith packages everything into a single deployable unit. Your UI, business logic, and data access all run in the same process, communicating through function calls rather than network requests.

This works brilliantly for early-stage products, small teams, and tightly coupled domains. You can move fast, validate ideas quickly, and avoid the operational overhead of managing multiple services. The trade-off is that scaling means replicating everything, deployments become riskier as the codebase grows, and a bug anywhere can bring down the whole system.

Many successful systems start here and extract services later once domain boundaries become clearer. A modular monolith with well-structured internal boundaries makes this transition much easier.

Distributed Architecture

Distributed systems split your application into multiple independent services communicating over the network. This includes both service-oriented architecture and its modern evolution, microservices.

Each service owns a specific capability, can have its own database, and can be developed, deployed, and scaled independently. This suits large organisations with multiple teams, systems with components that have very different scaling requirements, and mature products with well-understood boundaries.

The challenges are significant though. Network communication introduces latency and partial failures. Distributed transactions require patterns like sagas or eventual consistency. Operational overhead increases dramatically, and debugging issues that span services requires sophisticated tooling.

Serverless Architecture

Serverless abstracts away infrastructure entirely. You deploy individual functions that run on-demand, with your cloud provider handling scaling, availability, and capacity planning.

This is great for unpredictable or spiky workloads, event-driven workflows like file processing or webhooks, and MVPs where speed matters more than optimisation. You pay per execution, meaning near-zero cost when idle.

The constraints include cold start latency, execution time limits (typically 15 minutes maximum), and potential vendor lock-in. Debugging and local development can also be more difficult.

A note on categorisation: Serverless is technically a form of distributed architecture, but the operational model is different enough that it’s useful to consider separately. You don’t manage the distribution - the platform does.

Making the Choice

There’s no universally correct answer. Small teams often thrive with monoliths due to reduced operational complexity. Large organisations with multiple teams benefit from distributed architectures that enable independent work. Serverless suits unpredictable workloads and teams wanting to minimise operational overhead.

Consider your team size, acceptable operational complexity, scaling requirements, and how quickly you need to reach production. The right architecture is the one that matches your current constraints while leaving room to evolve.

Like posts like this?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

You give HTTP context through headers, cookies, and metadata. This statelessness is actually a feature: it makes HTTP simple, human-readable, and supported by every browser.

But stateless request/response isn’t always what you need.

HTTP

HTTP is the protocol of the internet. It’s request/response by nature - client asks, server answers, connection closes. This simplicity is why it’s everywhere.

The trade-off is overhead. Every request requires a new handshake. For most web traffic, that’s fine. For real-time applications, it becomes a bottleneck.

WebSockets

WebSockets provide bidirectional communication over a persistent connection. Once established, the connection stays open. Both client and server can send messages whenever they want, eliminating the handshake overhead on every message.

This makes WebSockets stateful - they can remember what happened previously. That’s a significant shift from HTTP’s stateless model.

Use WebSockets when you need true real-time bidirectional communication: chat applications, collaborative editing tools, multiplayer games, or any scenario where both sides need to initiate messages.

The cost is complexity. You’re maintaining persistent connections, handling reconnection logic, and managing state.

Server-Sent Events (SSE)

Server-Sent Events are actually older than WebSockets, yet far less commonly used. They provide one-way communication from server to client over standard HTTP.

Many “real-time” features only need server-to-client communication. News feeds, live scores, notifications, status updates - the client doesn’t need to send data back, it just needs to receive updates.

For these use cases, SSE is simpler than WebSockets. You don’t need bidirectional connections, you don’t need special libraries, and it works over standard HTTP infrastructure.

Why aren’t more people using SSE? Browser support was limited in the past. But it’s worth adding to your toolkit.

gRPC

gRPC is fast and uses a strict contract between services. It runs exclusively over HTTP/2 and uses Protocol Buffers for serialisation - not human-readable, but extremely efficient.

Browsers don’t support gRPC natively. You need special libraries and proxies to make it work in a browser context.

This makes gRPC ideal for service-to-service communication where you control both ends. Microservices architectures, IoT devices, low-latency internal APIs - anywhere you need speed and don’t need browser compatibility.

Generally, don’t use gRPC for frontend-to-backend communication. It’s not what it’s designed for.

REST and GraphQL (API Styles Over HTTP)

REST and GraphQL aren’t protocols - they’re architectural styles for building APIs on top of HTTP. Both use standard HTTP requests under the hood; they just differ in how you structure those requests and responses.

You can read more about thei differences here:

Making the Decision

When choosing a protocol, start with these questions.

Is it service-to-service communication where you control both ends? Use gRPC. The performance benefits are worth the setup complexity when browsers aren’t involved.

Does it need to be real-time? If you need bidirectional communication, WebSockets are your answer. But if you only need to push updates from server to client, try SSE first.

For frontend-to-backend APIs, the REST vs GraphQL decision comes down to data complexity. Do you have simple, uniform data needs? REST. Do you have complex data from multiple sources with varied client requirements? GraphQL.

Like posts like this?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

It comes down to one core trade-off: REST prioritizes simplicity and standardization. GraphQL prioritizes flexibility and efficiency.

REST: One Endpoint Per Resource

REST (Representational State Transfer) organizes your API around resources. Each resource gets its own URL, and you use HTTP methods to interact with it.

The server decides what data each endpoint returns. If you call GET /users/123, you get whatever fields the backend team decided to include: name, email, address, preferences, etc.

The Problem: Over-fetching and Under-fetching

Over-fetching: You only need the user’s name, but the endpoint returns 50 fields. You’re transferring data you’ll never use.

Under-fetching: You need the user AND their orders. That’s two separate API calls: GET /users/123 then GET /users/123/orders. More round-trips, higher latency.

GraphQL: One Endpoint, Client-Specified Data

GraphQL flips the model. Instead of multiple endpoints, you have a single endpoint (typically /graphql). The client sends a query specifying exactly what data it needs.

One request. You get exactly name and orders, nothing more.

The Trade-off: Complexity Moves to the Server

GraphQL requires a strongly-typed schema that defines every type and field. The server must resolve arbitrary queries, which can introduce performance challenges.

Caching also becomes harder. REST’s GET /users/123 is easily cached at the HTTP level. GraphQL POST requests with dynamic queries aren’t.

Key Differences

Structure: REST uses multiple endpoints (one per resource). GraphQL uses a single endpoint for everything.

Data fetching: In REST, the server decides what’s returned. In GraphQL, the client specifies exact fields.

Schema: REST schemas are optional (OpenAPI). GraphQL schemas are mandatory and strongly typed.

Versioning: REST typically uses URL versioning (/v1, /v2). GraphQL uses schema evolution - you add or deprecate fields without breaking clients.

Caching: REST has built-in HTTP caching. GraphQL requires custom caching strategies.

Error handling: REST uses HTTP status codes (404, 500). GraphQL always returns 200; errors appear in the response body.

Learning curve: REST is lower (uses familiar HTTP patterns). GraphQL is higher (new query syntax to learn).

When to Use REST

Your API maps cleanly to resources. Users, orders, products - if your domain is CRUD-shaped, REST fits naturally.

Caching matters. REST’s HTTP caching works out of the box. CDNs, browser caches, and reverse proxies all understand it.

You’re building a public API. REST is widely understood. External developers can integrate faster without learning GraphQL.

Your data model is stable. If endpoints rarely change, REST’s simplicity wins over GraphQL’s flexibility.

When to Use GraphQL

You have multiple clients with different data needs. A mobile app needs minimal data; a web dashboard needs everything. GraphQL lets each client request exactly what it needs.

You’re aggregating data from multiple sources. GraphQL can sit in front of multiple services and present a unified API.

Your frontend team iterates faster than backend. Frontend devs can add fields to their queries without waiting for backend changes.

Network efficiency matters. On slow mobile connections, fewer round-trips and smaller payloads make a real difference.

The Hybrid Approach

Many companies use both. A common pattern: REST for public APIs (wide compatibility, easy caching) and GraphQL internally (flexibility for frontend teams, data aggregation).

GraphQL can even sit on top of existing REST APIs as an aggregation layer, translating multiple endpoints into a single schema. You don’t have to choose one forever.

Like posts like this?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

Here are the ones that can set you apart:

2xx - Success

• 200 OK → Request succeeded

• 201 Created → New resource created (POST)

• 204 No Content → Success, nothing to return (DELETE)

3xx - Redirection

• 301 Moved Permanently → URL changed forever

• 302 Found → Temporary redirect

4xx - Client Error

• 400 Bad Request → Malformed request

• 401 Unauthorized → Not authenticated

• 403 Forbidden → Authenticated, but not allowed

• 404 Not Found → Resource doesn’t exist

5xx - Server Error

• 500 Internal Server Error → Something broke

• 502 Bad Gateway → Upstream server failed

• 503 Service Unavailable → Server overloaded

• 504 Gateway Timeout → Upstream didn’t respond

Like posts like this?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

Here are the nine types you should know.

1. Smoke Testing

The first check after deployment. Does the API even work?

You’re testing basic connectivity, key endpoints returning 200, and authentication not failing immediately. A simple GET /health request is usually enough.

2. Functional Testing

Does the API do what it’s supposed to do?

You’re validating inputs and outputs against requirements. Send a POST /users with valid data - did it create a user? Is the response structure correct? Do all CRUD operations behave as documented?

3. Integration Testing

Does the API work with everything it connects to?

APIs don’t exist in isolation. They talk to databases, other services, message queues. Integration testing verifies data flows correctly across these boundaries.

Create a user via API, then check if the downstream service received it.

4. Regression Testing

Did the new changes break old functionality?

Every time you deploy, you risk breaking something that worked before. Regression testing runs your existing test suite to catch unintended side effects.

Automate this in CI/CD.

5. Load Testing

How does the API perform under expected traffic?

You’re measuring response times, throughput, and resource usage when many users hit the API simultaneously. Simulate 500 concurrent users and watch what happens to latency.

6. Stress Testing

What happens when traffic exceeds expectations?

Load testing checks normal conditions. Stress testing pushes past the limit. You want to know the breaking point and whether the API fails gracefully or crashes entirely.

7. Security Testing

Is the API protected from attacks?

Check authentication and authorization. Try accessing admin resources with user tokens. Test for SQL injection, XSS, and CSRF. Verify rate limiting works.

8. UI Testing (API-Driven)

Does the frontend display API data correctly?

The API might return perfect data, but if the UI mishandles it, users see bugs. This tests the contract between backend responses and frontend rendering.

9. Fuzz Testing

How does the API handle garbage input?

Send malformed JSON, random strings, oversized payloads, unexpected data types. You’re looking for unhandled exceptions and system crashes that attackers could exploit.

Like posts like this?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

Here are my top picks for creating clear, professional system design diagrams.

1. Excalidraw

This is my go-to tool for system design work, and it’s what I recommend to everyone I coach.

Excalidraw gives you a hand-drawn aesthetic that feels approachable. It’s free, runs in your browser, and has virtually no learning curve. You can start sketching architecture in seconds.

What makes it perfect for system design practice is the flexibility. You’re not locked into rigid shapes or connectors. You can quickly iterate on designs, which is exactly what you need when you’re working through a problem in an interview setting.

For paid subscribers: I’ve created two resources to accelerate your practice:

• My Excalidraw System Design Template – copy it and adjust when you practice, so you’re not starting from scratch every time

  

• My System Design Component Library – essential Excalidraw components for databases, load balancers, caches, queues, and more

  

2. Draw.io

The Swiss Army knife of diagramming tools. It’s free, feature-rich, and integrates with Google Drive, Dropbox, GitHub, and Confluence.

The template library is massive, covering flowcharts, network diagrams, process maps, and more. Best for polished, professional diagrams you need to share with stakeholders or embed in documentation.

3. Lucidchart

The enterprise standard. If your company already uses it, lean into the template library and real-time collaboration features.

Works well for team environments where multiple people need to contribute simultaneously, with live updates and commenting built in.

4. DiagramGPT

Describe what you want in plain text, and it generates the diagram. Useful for quick iterations or when you’re exploring ideas and don’t want to manually place boxes yet.

You can tweak the output afterward, adjusting layout and colors to fit your needs.

5. Diagramming AI

Smart generation from simple descriptions and an interactive chat interface. You can make modifications using natural language commands and get suggestions as you build.

Integrates with other tools to streamline your workflow.

Want to learn more about System Design?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ System Design Walkthroughs of Popular Interview Questions
→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

But caching strategies matter. Pick the wrong one and you’ll spend weekends debugging stale data issues or wondering why your writes are suddenly slow.

There are two fundamental patterns you should be aware of: Read-Through and Write-Through caching.

Read-Through Cache

In a Read-Through setup, your application only talks to the cache. It never queries the database directly.

When your application requests data:

1. The service checks if the cache has the data

2. If yes (cache hit), it returns immediately

3. If no (cache miss), the cache fetches from the database, stores a copy, then returns it

Your application code stays simple. It doesn’t handle cache misses - the cache layer manages that automatically.

When it works well:

• Read-heavy workloads (product catalogues, user profiles)

• When you want simpler application code

• Systems where the same data gets requested repeatedly

The tradeoff: First-time reads are slow since every cache miss means a database trip. If your cache restarts, expect a flood of requests hitting your database while the cache warms up.

Write-Through Cache

In a Write-Through setup, every write operation goes to both the cache and database simultaneously.

The write only succeeds when both complete. Your cache and database stay perfectly in sync.

When it works well:

• When data consistency is non-negotiable (financial systems, inventory)

• Read-heavy systems that also need reliable writes

• When stale data causes real business problems

The tradeoff: Writes are slower because every operation has two steps. You might also cache data that never gets read, wasting memory.

How to decide between the two

Ask yourself two questions:

How painful is stale data? If outdated information causes customer complaints or financial errors, Write-Through gives you that consistency guarantee.

What’s your read-to-write ratio? Read-Through excels when reads vastly outnumber writes. Write-Through makes more sense when you write frequently and read that same data soon after.

Caveats

You rarely use just one strategy.

Many systems combine both. Write-Through for critical data that must stay consistent (user account balances). Read-Through for less critical, read-heavy data (product recommendations).

Some teams add Write-Behind (writing to cache immediately, database asynchronously) for high-throughput scenarios where eventual consistency is acceptable.

Neither is universally better. Understanding when to use each is what separates engineers who “add caching” from engineers who design systems that actually scale.

Like posts like this?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

Behind that seamless experience is a carefully orchestrated setup of distributed systems, each performing its role with precision. Understanding this process is fundamental to system design, and it’s why this question shows up in engineering interviews.

Let’s break down exactly what happens, step by step.

URLs

Behind that seamless experience is a carefully orchestrated dance of distributed systems, each performing its role with precision. Understanding this process is fundamental to system design, and it’s why this question keeps showing up in senior engineering interviews.

Let’s break down exactly what happens, step by step.

The Journey Begins: Understanding URLs

When you type a URL like https://www.bbc.co.uk/news/technology into your browser, you’re actually providing four pieces of information:

1. Scheme (https://) - Tells the browser which protocol to use. HTTPS means the connection will be encrypted.

2. Domain (www.bbc.co.uk) - The human-readable name of the server.

3. Path (/news) - Think of it as a directory in a file system.

4. Resource (/technology) - The specific file or content you want.

Computers don’t speak in domain names though. They speak in IP addresses - unique numerical identifiers for every device on the internet. So our browser has a problem to solve.

Step 1: DNS Lookup - Finding the Server’s Address

The browser needs to translate “www.bbc.co.uk“ into an IP address. This is where DNS (Domain Name System) comes in - essentially the internet’s phone book.

But DNS lookups can be slow, so the system is designed with multiple layers of caching:

First stop: Browser cache
The browser checks if it recently looked up this domain. If yes, it uses the cached IP address.

Second stop: Operating system cache
If the browser doesn’t have it, it asks the OS, which maintains its own short-term cache.

Third stop: DNS resolver
If neither has the answer, the OS queries a DNS resolver (usually provided by your ISP). This kicks off a chain of requests through the DNS infrastructure, from root servers to authoritative name servers, until the IP address is found.

The answer gets cached at every step along the way, making subsequent requests fast.

Step 2: Establishing a TCP Connection

Now that the browser has the IP address, it needs to establish a connection with the server. This happens through a TCP handshake, a multi-step process that takes several round trips across the network.

Modern browsers use something called “keep-alive connections”. Instead of opening a new connection for every request, they reuse existing connections whenever possible. This dramatically reduces latency.

If you’re using HTTPS (which you should be), there’s an additional step: the TLS/SSL handshake. This establishes an encrypted tunnel between your browser and the server, but it’s computationally expensive. Browsers mitigate this cost using SSL session resumption, allowing them to skip parts of the handshake for repeated connections.

Step 3: Sending the HTTP Request

Once the connection is established, the browser sends an HTTP request over it. HTTP itself is simple, just a text-based protocol asking the server for specific resources.

The server processes the request, fetches the necessary files (HTML, CSS, JavaScript), and sends back a response.

Step 4: Rendering the Page

The browser receives the HTML and starts parsing it.

As it reads through the HTML, it discovers additional resources it needs: CSS stylesheets, JavaScript files, images, fonts. For each of these, the browser repeats the entire process:

• DNS lookup (likely hitting cache this time)

• TCP connection (likely reusing an existing one)

• HTTP request

This is why web performance optimization matters so much. Each additional resource means more round trips, more latency, more waiting.

Why This Matters for System Design

This seemingly simple flow touches nearly every fundamental concept in distributed systems:

• Caching at multiple levels (browser, OS, DNS)

• Load balancing (DNS can return different IPs based on geography)

• Latency optimization (connection reuse, SSL session resumption)

• Fault tolerance (redundant DNS servers, CDNs)

• Scalability (handling millions of concurrent requests)

When you’re designing systems at scale, you’re essentially orchestrating variations of these same patterns. Understanding how they work together in something as “simple” as loading a webpage gives you the mental models you need to design robust, performant systems.

Like posts like this?

You may also like these:

By subscribing, you get a breakdown like this every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

Never store passwords as plaintext.

Recommendations from OWASP

According to the Open Web Application Security Project (OWASP), secure password storage requires two key ingredients:

1. Use a Modern Hashing Algorithm

A hash function is a one-way mathematical operation. You can turn a password into a hash, but you cannot reverse a hash back into the password.

But not all hashing functions are equal.

Common hash functions like MD5 and SHA-1 are fast - which sounds good, but is actually terrible for passwords. Fast hashing means attackers can try billions of password guesses per second using brute-force attacks.

Instead, use password-specific hashing algorithms that are intentionally slow and resource-intensive (such as bcrypt).

These algorithms make brute-force attacks computationally expensive and unattractive.

2. Salt the Passwords

A salt is a unique, randomly generated string added to each password before hashing. Even if two users have the same password (”password123”), their hashes will be completely different.

Why is this important?

Without salts, attackers can use precomputed attack methods:

• Rainbow tables: Massive databases of precomputed hashes for common passwords

• Database lookups: Quick searches to crack passwords in seconds

By adding a unique salt to each password, you ensure every hash is unique, making these precomputed attacks useless.

How It Works: Registration Flow

When a user creates an account:

1. User submits their password

2. System generates a random salt (e.g., “k8Px2mN9”)

3. System combines password + salt

4. System computes hash using a hashing algorithm like bcrypt

5. System stores both the hash AND the salt in the database

The salt isn’t a secret. It can be stored as plaintext right next to the hash. Its job is to make each hash unique, not to hide information.

Example:

Password: "Welcome123!"
Salt: "S4!t"
Combined: "Welcome123!S4!t"
Hash: "$2y$10$Qbpn8Bq...b6MVioit/bG" (this gets stored)

How It Works: Login Validation

When a user logs in:

1. User submits username and password

2. System fetches the stored hash and salt from the database

3. System appends the stored salt to the submitted password

4. System computes a new hash

5. System compares: if computed hash == stored hash, password is valid ✓

Additional Security Layers

Consider these complementary defenses:

• Rate limiting: Limit login attempts per IP address or user account

• Account lockout: Temporarily disable accounts after multiple failed attempts

• Adaptive hash parameters: Increase bcrypt “cost factor” over time as hardware improves

• Pepper (optional): Add a secret key stored outside the database for an extra layer

Like posts like this?

By subscribing, you get a breakdown like this one every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you’re into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

• SQL: Structured data in tables with predefined schemas. Relationships between tables via JOINs.

• NoSQL: Flexible data storage (documents, key-value, graph, wide-column). No fixed schema required.

SQL Databases

Key characteristic: Structured, relational, ACID-compliant

Structure:

Examples: PostgreSQL, MySQL, Oracle

Properties:

• ACID Transactions - All-or-nothing operations (critical for payments, banking)

• Complex Queries - JOINs across multiple tables, aggregations, filtering

• Data Integrity - Schema enforces structure, reduces errors

• Mature Tooling - Decades of optimisation, monitoring, backup tools

• Scaling

  • Vertical: Adding CPU/RAM to one server is expensive

  • Horizontal: Difficult to shard writes across multiple servers

• Rigid Schema - Changes require migrations, can lock tables

NoSQL Databases

Key characteristic: Flexible, distributed, eventually consistent

Types:

Properties:

• Horizontal Scaling - Add more servers to handle load

• Flexible Schema - Store different structures without migrations

• High Performance - Optimised for specific access patterns

• Distributed by Design - Natural fit for global, multi-region systems

• Eventual Consistency - Writes take time to propagate across nodes

• Limited Querying - Complex queries handled in application code

• Less Data Integrity - Schema validation happens in your application

A decision framework for interviews

Consistency Trade-off

Strong Consistency (SQL):

Eventual Consistency (NoSQL):

Scaling Patterns

SQL Scaling:

• Vertical: Add CPU/RAM to main server (Read replicas for read-heavy loads)

• Horizontal: Sharding (complex) - Split data across servers by key

NoSQL Scaling:

• Horizontal: Add nodes to cluster (Automatic data distribution)

Common Use Cases

Like posts like this?

By subscribing, you get a breakdown like this one every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you're into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

Authentication and Authorization solve two fundamental security questions in any system:

• Authentication: “Who are you?” - Proving and verifying user identity

• Authorization: “What are you allowed to do?” - Determining permissions and access rights

After a user logs in once, the system needs a way to “remember” them across multiple requests without asking for credentials repeatedly. There are two fundamentally different approaches to maintaining logged-in state

• session-based authentication

• JWT-based authentication

Session-Based Authentication

Key characteristic: The server stores session state, making it stateful. Session ID is just a reference key.

Components:

• Cookie: Contains session ID (small, opaque string)

• Session Store: Centralized storage (Redis, database, Memcached)

• Session Data: User ID, expiration, metadata stored server-side

JWT-Based Authentication

Key characteristic: No server-side storage, making it stateless. All data is in the token itself.

JWT Structure (three parts separated by dots):

header.payload.signature

Example:
eyJhbGc...  .  eyJ1c2Vy...  .  SflKxwRJ...
[algorithm]    [user data]     [signature]

Signing Algorithms:

• HMAC (HS256): Symmetric - same key for sign & verify (simpler, faster)

• RSA/ECDSA: Asymmetric - private key signs, public key verifies (more secure for distributed systems)

Token Refresh Pattern (Best Practice for JWT)

Access Token (short-lived: 15 min) ────┐
                                       ├──▶ Regular API requests
Refresh Token (long-lived: 7 days) ────┘
         │
         └──▶ Only sent to /refresh endpoint when access token expires

Trade-offs

Like posts like this?

By subscribing, you get a breakdown like this one every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you're into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

REST

REST (Representational State Transfer) is the most widely adopted API architecture style. It uses standard HTTP methods to perform operations on resources identified by URLs.

How it works:

• Client sends HTTP request to a URL

• Server processes request and returns data (usually JSON)

• Each request is independent - server doesn’t remember previous requests

Key characteristics:

• Stateless: Each request contains all information needed to process it

• Resource-based: Everything is treated as a resource with a unique URI

• Standard HTTP methods: GET (read), POST (create), PUT (update), DELETE (remove)

• Multiple formats: Typically JSON, but can support XML, plain text, etc.

When to use: CRUD operations, public APIs, and when you need broad compatibility across different platforms and languages.

gRPC

gRPC is Google’s high-performance RPC framework that uses Protocol Buffers for serialization and HTTP/2 for transport.

How it works:

• Define your API in a .proto file

• gRPC generates code for client and server

• Data transmitted as compressed binary

• Supports streaming in both directions

Key characteristics:

• Binary protocol: Uses Protocol Buffers instead of JSON

• HTTP/2 based: Multiplexing, server push, header compression

• Four communication patterns:

  • Unary (request-response)

  • Server streaming

  • Client streaming

  • Bidirectional streaming

• Strongly typed: Contract-first development with .proto files

When to use: Building microservices, need high performance, or require real-time streaming.

GraphQL

GraphQL is a query language and runtime that allows clients to request exactly what data they need, solving the over-fetching and under-fetching problems of REST.

How it works:

• Client sends a query specifying exact fields needed

• Server returns only those fields

• One endpoint handles all requests

• Clients control the response shape

Key characteristics:

• Single endpoint: All requests go to one URL

• Flexible queries: Clients specify exact data requirements

• Type system: Strongly typed schema

• Real-time subscriptions: Built-in support for live updates

• Self-documenting: Introspection capabilities

When to use: Different clients need different data, you have complex relationships, or want to prevent over/under-fetching.

Webhooks

Webhooks implement an event-driven architecture where the server pushes data to clients when specific events occur, eliminating the need for constant polling. Instead of clients asking for updates, the server tells them when something happens.

How it works:

• Client registers a URL with the server

• When an event occurs, server sends HTTP POST to that URL

• Client processes the event data

Key characteristics:

• Push-based: Server initiates communication

• Event-driven: Triggers on specific actions

• HTTP POST: Typically sends JSON payload to registered URLs

• Asynchronous: Fire-and-forget pattern

When to use: Payment notifications, CI/CD pipelines, third-party integrations, or any scenario where you need real-time event notifications.

WebSockets

WebSockets keep a connection open between client and server, allowing instant two-way communication.

How it works:

• Client and server establish connection with special handshake

• Connection stays open

• Either side can send messages anytime

• No request-response pattern needed

Key characteristics:

• Persistent connection: Stays open after initial handshake

• Low latency: No HTTP overhead for each message

• Bidirectional: Both client and server can initiate messages

• Text or binary: Supports multiple data formats

When to use: Chat applications, live sports scores, collaborative editing, real-time dashboards, or multiplayer games.

WebRTC

WebRTC enables peer-to-peer communication directly between browsers and applications without data passing through your servers.

How it works:

• Browsers exchange connection info through your server

• Once connected, they communicate directly peer-to-peer

• Audio, video, and data flow directly between users

• Your server doesn’t handle the media stream

Key characteristics:

• Peer-to-peer: Direct connection between clients

• Media streaming: Audio, video, and data channels

• NAT traversal: Works across different networks using STUN/TURN

• Encryption: Mandatory SRTP for media, DTLS for data

Example use case: In a video call application, after initial signaling through your server, participants exchange audio/video directly between their devices, reducing server load and latency.

When to use: Video conferencing, screen sharing, file sharing between users, or real-time collaboration tools.

Choosing the Right API

In system design interviews, selecting the appropriate API depends on several factors:

• Latency requirements: gRPC or WebSockets for low latency

• Data flexibility: GraphQL when clients need varied data shapes

• Real-time needs: WebSockets for bidirectional, Webhooks for server-push

• Bandwidth constraints: gRPC for efficient binary protocol

• Peer-to-peer: WebRTC for direct client communication

Modern systems often combine multiple API types. A video conferencing system might use REST for user management, WebSockets for chat, WebRTC for video streams, and Webhooks for notifications.

Like posts like this?

By subscribing, you get a breakdown like this one every week.

Free subscribers also get a little bonus:

🎁 The System Design Interview Preparation Cheat Sheet

If you're into visuals, paid subscribers unlock:

→ My Excalidraw system design template – so you have somewhere to start
→ My Excalidraw component library – used in the diagram of this issue

No pressure though. Your support helps me keep writing, and I appreciate it more than you know ❤️

What you’ll learn in ~15 minutes

• How I would scope the problem without missing important requirements (custom aliases, expirations, availability).

• Why choosing containers over serverless for code execution is better

• The pattern that makes leaderboards lightning-fast instead of crushing your database with expensive GROUP BY queries

• How to implement container security that prevents malicious code from taking down your entire platform

How this issue is structured
I split the write-up into the same sections I’d narrate at a whiteboard. Free readers get the full walkthrough up to the deep-dive parts. Paid members get the 🔒 sections.

• Initial Thoughts & Clarifying Questions

• Functional Requirements

• Non-Functional Requirements

• Back-of-the-envelope Estimations

• 🔒 System Design (the architecture I’d draw and the excalidraw link for it!)

• 🔒 Component Breakdown (why each piece exists + alternatives)

• 🔒 Trade-offs Made

• 🔒 Security & Privacy

• 🔒 Monitoring, Logging, and Alerting

• 🔒 Final Thoughts

Quick note: If you’ve been getting value from these and want the full deep dives, becoming a paid member helps me keep writing—and you’ll immediately unlock the 🔒 sections above, plus a few extras I lean on when I practice.

Members also get

• 12 Back-of-the-Envelope Calculations Every Engineer Should Know

• My Excalidraw System Design Template — drop-in canvas you can copy and tweak.

• My System Design Component Library

Subscribe now

Let’s get to it!

Initial Thoughts & Clarifying Questions

To begin, I’d want to understand the scope and constraints we’re working with here. LeetCode is actually a more manageable system than many interview questions since it’s not dealing with billions of users, but it has some unique challenges around code execution that make it interesting.

Let me ask a few clarifying questions:

What’s our user scale? I’d assume we’re looking at roughly 500,000 registered users with maybe 50,000 daily active users during normal periods, but potentially 100,000 concurrent users during major competitions. This is based on LeetCode’s actual scale being relatively modest compared to social media platforms.

What types of problems are we supporting? I’m thinking we need to handle algorithmic problems with various input types - arrays, trees, graphs, etc. We probably have around 4,000 problems total, which is manageable from a data storage perspective.

What’s our competition model? I’d assume competitions run for about 90 minutes, feature 10 problems, and need real-time leaderboards. Scoring would be based on number of problems solved, with time as a tiebreaker.

What programming languages do we need to support? I’d expect at least Python, Java, C++, JavaScript, and maybe Go, Rust, and a few others. Each language needs proper execution environments.

What’s our performance expectation for code execution? I’d target sub-5 second response times for code submissions, including compilation, execution against test cases, and result generation.

How real-time does the leaderboard need to be? I’m thinking 5-10 second latency is acceptable - users don’t need instant updates, but they want to see their ranking relatively quickly during competitions.

Functional Requirements

From what I understand, the core requirements are:

1. Problem browsing and viewing - Users need to see a paginated list of problems and view individual problem statements with code stubs in their preferred language

2. Code submission and execution - Users submit solutions that get executed against hidden test cases with pass/fail feedback

3. Competition leaderboards - Real-time ranking during timed competitions based on problems solved and completion time

4. Multi-language support - Same logical problem should work across different programming languages

I’m deliberately keeping user authentication, profiles, and payment processing out of scope since those are standard components that don’t add much to the interesting system design aspects.

Non-Functional Requirements

I’d expect this system to handle:

Availability over consistency - It’s better to show slightly stale leaderboard data than to have the system go down. Users can tolerate eventual consistency for rankings.

Secure code execution - This is critical since we’re running arbitrary user code. We need complete isolation to prevent malicious code from affecting our infrastructure.

Sub-5 second submission feedback - Users expect quick feedback on their solutions.

Competition scale of 100,000 concurrent users - During major contests, we need to handle significant traffic spikes without degrading performance.

High availability during competitions - A system outage during a major competition would be catastrophic.

Back-of-the-envelope Estimations

Let’s walk through the numbers to understand our scale:

Daily active users: 50,000 normal days, 100,000 during competitions

Submission patterns: Let’s say each active user makes 20 submissions per day on average. During competitions, this could spike to 50 submissions per user over 90 minutes.

• Normal day: 50,000 × 20 = 1M submissions/day = ~12 submissions/second

• Competition spike: 100,000 × 50 = 5M submissions over 90 minutes = ~925 submissions/second

Read traffic for problems: Users browse problems frequently. Maybe 10 problem views per active user per day.

• Normal: 50,000 × 10 = 500K problem views/day = ~6 views/second

• Competition: Could be 2-3x higher as users read competition problems

Leaderboard requests: During competitions, all active users check leaderboard every 30 seconds.

• Competition: 100,000 users × 2 requests/minute = ~3,300 requests/second for leaderboard

Storage requirements:

• 4,000 problems × 50KB average (including all language stubs, test cases) = 200MB for problems

• 1M submissions/day × 5KB average = 5GB/day submission data

• Over 5 years: ~9TB total storage (very manageable)

Bandwidth estimates:

• Code submissions: 925 peak submissions/second × 5KB = ~4.6MB/second upload

• Leaderboard responses: 3,300 requests/second × 2KB = ~6.6MB/second download

• Problem content: 6 views/second × 50KB = 300KB/second

The numbers show this is actually a quite manageable system from a scale perspective, but the complexity comes from the secure code execution requirements.

🔒 System Design

Here’s how I’d think about the system design:

Read more

For most of the past decade, I’ve been focused on something just as hard — how great teams are built.

That’s why I write a second newsletter.

Blog for Engineering Managers

Get 50+ EM Notion templates ($79), The Engineering Manager's Field Guide ($29), and all the posts when you subscribe.

By Stephane Moreau

It’s for senior engineers, tech leads, and EMs who want to grow as leaders.
Each week, I share honest lessons on communication, trust, feedback, and team design, based on my experience, successes (and mistakes).

And to help you get started, you’ll get 10 free Notion templates I use for:

• 1:1s that actually help people grow

• Clear performance reviews

• Tracking team health and delivery

If you’re ready to go from engineer to confident leader, you can join here

Join Blog for Engineering Managers

You’ll get 10 templates right away for free + two issues every week.

See you there,
Stephane

What you’ll learn in ~15 minutes

• How I would scope the problem without missing important requirements (custom aliases, expirations, availability).

• Why the “obvious” feed generation approach fails at scale and how Instagram’s hybrid fanout strategy solves the celebrity problem while keeping 99% of users happy.

• The media upload architecture that handles 4GB videos by never touching your application servers.

How this issue is structured
I split the write-up into the same sections I’d narrate at a whiteboard. Free readers get the full walkthrough up to the deep-dive parts. Paid members get the 🔒 sections.

• Initial Thoughts & Clarifying Questions

• Functional Requirements

• Non-Functional Requirements

• Back-of-the-envelope Estimations

• 🔒 System Design (the architecture I’d draw and the excalidraw link for it!)

• 🔒 Component Breakdown (why each piece exists + alternatives)

• 🔒 Trade-offs Made

• 🔒 Security & Privacy

• 🔒 Monitoring, Logging, and Alerting

• 🔒 Final Thoughts

Quick note: If you’ve been getting value from these and want the full deep dives, becoming a paid member helps me keep writing—and you’ll immediately unlock the 🔒 sections above, plus a few extras I lean on when I practice.

Members also get

• 12 Back-of-the-Envelope Calculations Every Engineer Should Know

• My Excalidraw System Design Template — drop-in canvas you can copy and tweak.

• My System Design Component Library

Subscribe now

Let’s get to it!

Initial Thoughts & Clarifying Questions

To begin, I’d want to understand the scope and scale we’re targeting before diving into the technical design. Let me ask some key questions:

What’s our scale target? I’m assuming we’re looking at around 500M daily active users with roughly 100M posts created per day. This gives us a sense of the read-heavy nature of the system.

What types of media are we supporting? I’ll assume we need to handle both photos (up to 8MB) and videos (up to 4GB).

What’s our latency expectation? For a social media platform, I’d expect feed loading to be under 500ms end-to-end, and media should render almost instantly once requested.

Are we building a global service? I’ll assume yes, which means we need to think about geographic distribution and CDN placement.

What’s our consistency model? For social media, I’m assuming eventual consistency is acceptable - it’s fine if a new post takes up to 2 minutes to appear in all followers’ feeds.

How many people does an average user follow? This is crucial for feed generation strategy. I’ll assume most users follow 100-500 accounts, but we need to handle edge cases of users following 10,000+ accounts.

What’s our read-to-write ratio? Social platforms are heavily read-skewed. I’d estimate each user views 100+ posts for every 1 post they create, which will heavily influence our caching and data distribution strategies.

Functional Requirements

From what I understand, the core requirements are:

Post Creation: Users must be able to upload photos and videos with captions. This needs to handle large files efficiently and provide immediate feedback to users about upload progress.

Social Graph Management: Users need to follow and unfollow other users. This creates a unidirectional relationship that drives our entire feed generation strategy.

Feed Generation: Users should see a chronological timeline of posts from accounts they follow. This is the most technically challenging requirement given our scale.

These three requirements form the foundation - everything else we might add (likes, comments, stories) would build on this core architecture.

Non-Functional Requirements

I’d expect this system to handle several critical non-functional requirements:

High Availability: We need 99.9% uptime minimum. Users expect Instagram to always be available, and downtime directly impacts user engagement and revenue.

Low Latency: Feed requests should complete under 500ms, and media should start rendering immediately. Users will abandon the app if it feels slow.

Massive Scale: Supporting 500M daily active users means handling potentially 100,000+ requests per second during peak hours. Our architecture needs to scale horizontally.

Global Distribution: Users worldwide should have similar performance, requiring strategic CDN placement and potentially regional data centers.

Eventual Consistency: It’s acceptable for new posts to take 1-2 minutes to appear in all followers’ feeds, allowing us to optimize for performance over strict consistency.

Back-of-the-envelope Estimations

Let’s work through the numbers to understand what we’re dealing with:

User Activity: With 500M daily active users, if each user checks their feed 10 times per day, that’s 5 billion feed requests daily. During peak hours (let’s say 8 hours), that’s roughly 175,000 feed requests per second.

Content Creation: 100M posts per day means about 1,200 posts per second on average, with peaks potentially 3-5x higher during major events or peak social hours.

Storage Requirements:

• If average media size is 2MB per post: 100M posts × 2MB = 200TB of new media daily

• Metadata per post (user ID, timestamp, caption, media reference): roughly 1KB × 100M = 100GB daily

• Over 10 years: ~750PB of media storage needed

Read/Write Patterns:

• Read QPS: ~175,000 feed requests/second + media requests (potentially 10x higher)

• Write QPS: ~1,200 posts/second + follow operations

• This confirms we’re dealing with a heavily read-skewed system

Memory Requirements: For feed caching, if we cache the last 100 posts for each active user’s feed (assuming 1KB per post summary), that’s 500M users × 100 posts × 1KB = 50TB of feed cache. This is manageable with distributed caching.

Bandwidth: During peak viewing, if 100M users are simultaneously viewing media averaging 2MB each, that’s 200TB/hour of bandwidth. This absolutely requires CDN distribution.

🔒 System Design

This is the design I am thinking of:

Read more

What you’ll learn in ~15 minutes

• How I would scope the problem without missing important requirements (custom aliases, expirations, availability).

• Why Server-Sent Events are better than WebSockets for one-way real-time updates and how to implement SSE at Facebook scale

• The pub-sub architecture pattern that enables millions of concurrent viewers without overwhelming your database

• How cursor-based pagination outperforms offset pagination for infinite scroll in high-volume comment streams

How this issue is structured
I split the write-up into the same sections I’d narrate at a whiteboard. Free readers get the full walkthrough up to the deep-dive parts. Paid members get the 🔒 sections.

• Initial Thoughts & Clarifying Questions

• Functional Requirements

• Non-Functional Requirements

• Back-of-the-envelope Estimations

• 🔒 System Design (the architecture I’d draw and the excalidraw link for it!)

• 🔒 Component Breakdown (why each piece exists + alternatives)

• 🔒 Trade-offs Made

• 🔒 Security & Privacy

• 🔒 Monitoring, Logging, and Alerting

• 🔒 Final Thoughts

Quick note: If you’ve been getting value from these and want the full deep dives, becoming a paid member helps me keep writing—and you’ll immediately unlock the 🔒 sections above, plus a few extras I lean on when I practice.

Members also get

• 12 Back-of-the-Envelope Calculations Every Engineer Should Know

• My Excalidraw System Design Template — drop-in canvas you can copy and tweak.

• My System Design Component Library

Subscribe now

Let’s get to it!

Initial Thoughts & Clarifying Questions

To begin, I’d want to understand the scope and constraints better before jumping into the design. Here are the key questions I’d ask the interviewer:

“What’s the expected scale we’re designing for?” - I’m assuming we need to handle millions of concurrent videos with thousands of comments per second per video. This tells me we’re designing for massive scale.

“How real-time do the comments need to be?” - I assume sub-200ms latency, which means we’re talking about truly real-time communication, not just “fast enough”.

“Do we need to handle comment moderation or spam filtering?” - I’ll assume this is out of scope for now, but it’s good to acknowledge that production systems would need robust content filtering.

“What about comment persistence - how long should we store comments?” - I’m assuming we need to store comments indefinitely for historical viewing, which impacts our storage design significantly.

“Do we need to support features like reactions, replies, or threading?” - I’ll assume these are out of scope to keep the core problem manageable, though I’d mention them as potential extensions.

“What platforms need to be supported?” - I’m assuming web and mobile clients, both of which can handle persistent connections reasonably well.

Functional Requirements

From what I understand, the core requirements are prioritized as follows:

Primary Requirements:

• Users can post comments on live video feeds with immediate persistence

• Viewers see new comments in real-time as they’re posted by others

• Users can view historical comments when joining a live stream

• Comments must be displayed in chronological order with proper pagination

Secondary Requirements:

• Users can scroll through comment history (infinite scroll pattern)

• The system gracefully handles user connections and disconnections

• Comments remain associated with the correct live video stream

We’re building both a real-time messaging system and a persistent chat history system. The real-time aspect is what makes this challenging at scale.

Non-Functional Requirements

I’d expect this system to handle some serious performance and scale requirements:

Scale Expectations:

• Support millions of concurrent live videos simultaneously

• Handle thousands of comments per second per popular live video

• Accommodate millions of concurrent viewers across all streams

Performance Requirements:

• Sub-200ms end-to-end latency for comment delivery

• High availability with minimal downtime (99.9%+ uptime)

• Eventual consistency is acceptable - comments don’t need to appear in exactly the same order for all users immediately

User Experience:

• Seamless real-time experience that feels instantaneous

• Reliable comment delivery without missing messages

• Smooth infinite scrolling for historical comments

• Graceful degradation when network conditions are poor

The 200ms latency requirement is particularly important because that’s the threshold where humans perceive interactions as real-time versus delayed.

Back-of-the-envelope Estimations

Let’s work through the numbers to understand what we’re dealing with:

User Activity Assumptions:

• 10 million daily active users watching live videos

• Peak concurrent users: ~2 million (20% of DAU)

• Average of 1,000 concurrent live videos during peak

• Popular videos might have 100,000+ concurrent viewers

Comment Volume Calculations:

• Average user posts 5 comments per session

• Active commenting users: ~20% of viewers

• For a popular video with 100K viewers: 20K active commenters

• If each commenter posts once every 2 minutes: ~167 comments/second per popular video

• Across 1,000 concurrent videos: ~167,000 comments/second system-wide

Storage Requirements:

• Average comment size: ~100 bytes (including metadata)

• Daily comments: 10M users × 5 comments = 50M comments

• Daily storage: 50M × 100 bytes = 5GB per day

• Annual storage: ~1.8TB (manageable, but we’ll want compression and archiving)

Bandwidth Estimates:

• Each comment needs to be delivered to all viewers of that video

• Popular video with 100K viewers receiving 167 comments/second

• Outbound traffic per video: 167 × 100 bytes × 100K viewers = 1.67GB/second

• This shows why we need efficient distribution mechanisms

Connection Requirements:

• 2 million concurrent SSE connections during peak

• If each server handles 50K connections: need ~40 servers minimum

• With redundancy and load distribution: ~100+ servers for connection handling

These numbers confirm we need a distributed architecture with efficient real-time communication.

🔒 System Design

This is the simple design I would draw:

Read more

What you’ll learn in ~15 minutes

• How I would scope the problem without missing important requirements

• Why Layer 4 load balancers are critical for WebSocket scaling

• The inbox table pattern for guaranteed message delivery (how to handle offline users without complex queuing systems)

• When Redis Pub/Sub outperforms Kafka for messaging

How this issue is structured
I split the write-up into the same sections I’d narrate at a whiteboard. Free readers get the full walkthrough up to the deep-dive parts. Paid members get the 🔒 sections.

• Initial Thoughts & Clarifying Questions

• Functional Requirements

• Non-Functional Requirements

• Back-of-the-envelope Estimations (QPS, storage, bandwidth, cardinality math)

• 🔒 System Design (the architecture I’d draw and the excalidraw link for it!)

• 🔒 Component Breakdown (why each piece exists + alternatives)

• 🔒 Trade-offs Made

• 🔒 Security & Privacy

• 🔒 Monitoring, Logging, and Alerting

• 🔒 Extensions & Advanced Considerations

Quick note: If you’ve been getting value from these and want the full deep dives, becoming a paid member helps me keep writing—and you’ll immediately unlock the 🔒 sections above, plus a few extras I lean on when I practice.

Members also get

• 12 Back-of-the-Envelope Calculations Every Engineer Should Know

• My Excalidraw System Design Template — drop-in canvas you can copy and tweak.

• My System Design Component Library

Subscribe now

Let’s get to it!

Initial Thoughts & Clarifying Questions

To begin, I'd want to understand the specific scope and constraints of this messaging system. Let me ask some clarifying questions to nail down the requirements:

Q: Are we focusing on one-to-one messaging, group chats, or both? I'm assuming we need to support both, with group chats being a generalization where a "group of two" represents one-to-one messaging. This simplifies our data model.

Q: What types of media should we support beyond text messages? I'll assume we need to handle images, videos, and file attachments - essentially any media type that users commonly share. This will significantly impact our storage and bandwidth considerations.

Q: Should we support real-time features like typing indicators, read receipts, or online presence? For the core design, I'm focusing on message delivery, but I'll mention how we could extend this for presence indicators later since it's a common follow-up question.

Q: What's our expected scale in terms of users and messages? Given that WhatsApp serves billions of users, I'm targeting a system that can handle billions of users sending hundreds of billions of messages daily.

Q: Do we need to support multiple devices per user? Initially, I'll design for one device per user to keep things simple, then show how to extend for multiple devices since that's realistic for modern messaging apps.

Q: What are our consistency requirements - do messages need to be delivered in order? I'm assuming eventual consistency is acceptable, but messages within a chat should maintain order.

Q: Any specific privacy or data retention requirements? Following WhatsApp's model, I'll design for minimal data retention - messages should be deleted from servers once delivered to maintain privacy.

Functional Requirements

From what I understand, the core requirements are:

1. Create and manage group chats - Users can start conversations with individuals or groups

2. Send and receive text messages - Real-time bidirectional messaging between participants

3. Send and receive media attachments - Support for images, videos, and files

4. Access messages after being offline - Message persistence and delivery to disconnected clients

5. Message delivery guarantees - Ensure messages reach their intended recipients

These requirements form the foundation of our system. Each one builds on the previous, creating natural stepping stones for our design evolution.

Non-Functional Requirements

I'd expect this system to handle several critical performance and operational characteristics:

Latency: We'd likely need sub-500ms message delivery for users who are online. This feels natural - anything slower would make conversations feel sluggish, but the difference between 100ms and 500ms isn't particularly noticeable to users.

Reliability: Message delivery must be guaranteed. It would be unacceptable for messages to simply disappear.

Scale: Supporting billions of users sending hundreds of messages per day. The math here gets interesting when we calculate the actual throughput requirements.

Privacy: Following WhatsApp's philosophy, we should minimize data retention. Messages should be deleted from servers after successful delivery.

Availability: This is a global system that needs to tolerate individual component failures without taking down the entire service. We can't have single points of failure.

Back-of-the-envelope Estimations

Let's say we have 2 billion daily active users... If each user sends 50 messages per day on average, that gives us 100 billion messages daily.

QPS Calculations:

• Daily messages: 2B users × 50 messages = 100B messages/day

• Messages per second: 100B / (24 × 3600) ≈ 1.16M messages/sec

• Read QPS: Assuming each message is read by 2.5 recipients on average = 2.9M reads/sec

• Peak load (3x average): 3.5M writes/sec, 8.7M reads/sec

Storage Requirements:

• Average message size: 1KB (including metadata)

• Daily storage: 100B × 1KB = 100TB/day

• With 30-day retention: 100TB × 30 = 3PB total

• But most messages are delivered immediately and deleted, so steady state is probably 10-20% of this = 300-600TB

Bandwidth Estimates:

• Message traffic: 1.16M/sec × 1KB = 1.16 GB/sec

• Media uploads (10% of messages, 100KB average): 116K/sec × 100KB = 11.6 GB/sec

• Total ingress: ~13 GB/sec

• With replication and delivery overhead: ~40 GB/sec total bandwidth

Memory Requirements:

• Active connections: Assuming 20% of users online simultaneously = 400M connections

• Connection overhead: ~10KB per connection = 4TB memory for connection state

• Message caching: Hot data cache of ~1 hour = 4TB

• Total memory across all servers: ~8TB

These numbers tell me we're definitely in distributed systems territory - no single machine can handle this load.

🔒 System Design

Here’s the system design I am thinking of:

Read more

What you’ll learn in ~15 minutes

• How I would scope the problem without missing important requirements (custom aliases, expirations, availability).

• Geohashing and quadtrees comparison for high-frequency location updates (crucial for any proximity-based system design)

• The distributed locking pattern that prevents double-booking drivers

• Where to be strict about ACID properties and where eventual consistency is acceptable

How this issue is structured
I split the write-up into the same sections I’d narrate at a whiteboard. Free readers get the full walkthrough up to the deep-dive parts. Paid members get the 🔒 sections.

• Initial Thoughts & Clarifying Questions

• Functional Requirements

• Non-Functional Requirements

• Back-of-the-envelope Estimations (QPS, storage, bandwidth, cardinality math)

• 🔒 System Design (the architecture I’d draw and the excalidraw link for it!)

• 🔒 Component Breakdown (why each piece exists + alternatives)

• 🔒 Trade-offs Made

• 🔒 Deep Dives (Location Service Architecture, Consistency in Matching)

• 🔒 Security & Privacy

• 🔒 Monitoring, Logging, and Alerting

Quick note: If you’ve been getting value from these and want the full deep dives, becoming a paid member helps me keep writing—and you’ll immediately unlock the 🔒 sections above, plus a few extras I lean on when I practice.

Members also get

• 12 Back-of-the-Envelope Calculations Every Engineer Should Know

• My Excalidraw System Design Template — drop-in canvas you can copy and tweak.

• My System Design Component Library

Subscribe now

Let’s get to it!

Initial Thoughts & Clarifying Questions

To begin, I'd want to understand the scope and constraints of what we're building here. Let me ask a few clarifying questions to make sure I'm designing the right system:

What's the core functionality we need to support? I'm assuming we need ride matching between drivers and passengers, fare estimation, and real-time location tracking. Based on typical Uber usage, I'll focus on these core features.

What's our scale? I'm thinking we're looking at millions of daily active users globally, with hundreds of thousands of concurrent rides during peak hours. This suggests we need a highly scalable, distributed system.

What about different vehicle types? For this interview, I'll scope this down to just one vehicle type - let's say UberX - to keep things focused. We can always discuss how to extend this later.

Real-time requirements? I'm assuming we need sub-minute matching times and real-time location updates. Users expect to be matched with a driver quickly, and drivers need accurate navigation.

Geographic scope? I'll design for a global system with regional deployments, but focus on one region for the core design.

From what I understand, we're building a ride-sharing platform where users can request rides, get fare estimates, and be matched with nearby drivers in real-time.

Functional Requirements

From what I understand, the core requirements are:

1. Fare estimation: Users should be able to input a start location and destination and receive an estimated fare and ETA

2. Ride requesting: Users should be able to request a ride based on an estimate and be matched with a nearby available driver in real-time

3. Driver operations: Drivers should be able to accept/deny ride requests and navigate to pickup and drop-off locations

I'm putting several features out of scope to stay focused: multiple car types, driver/rider ratings, scheduling rides in advance, and payment processing. These are important but secondary to the core matching functionality.

Non-Functional Requirements

I'd expect this system to handle several critical non-functional requirements:

Low latency matching: We need to match riders with drivers in under one minute, or return a "no drivers available" message. Users won't wait around forever, and quick matching is core to the user experience.

Consistency of matching: This is crucial - we need to ensure any given ride is only ever matched to one driver, and no driver gets multiple ride requests simultaneously. This is a classic consistency requirement where we can't afford duplicate bookings.

High availability outside of matching: The system should be available 24/7 for fare estimates, driver location updates, and general operations. Only the actual matching process might occasionally fail due to consistency requirements.

High throughput during surges: We need to handle massive spikes during big events - potentially hundreds of thousands of requests within a region in a short time window.

Back-of-the-envelope Estimations

Let me work through some numbers to understand the scale we're dealing with. I know that back-of-envelope calculations should only be done when they directly influence design decisions, and here they will - particularly for our location service.

Active drivers and location updates: Let's say Uber has about 6 million drivers globally, with roughly 3 million active at any given time. If we update driver locations every 5 seconds (which we need for accurate matching), that's:

3 million drivers ÷ 5 seconds = 600,000 location updates per second

This is a massive write load that will definitely influence our database choice.

Daily ride requests: With millions of daily active users, let's estimate 50 million ride requests globally per day. During peak hours (say 4 hours total), this could be 5x higher:

• Peak: ~175,000 ride requests per hour = ~50 requests per second

• During major events or surges: Could spike to 10x normal = 500+ requests per second

Storage for ride data: Each ride record might be ~2KB (with all metadata). At 50M rides/day:

• Daily: 100 GB of new ride data

• Annual: ~35 TB of ride data

Location data storage: 600K updates/second × 100 bytes per update = 60 MB/second of location data. However, we only need current locations, so storage is manageable - maybe 300 MB total for 3M driver locations.

Our location service needs to handle 600K writes per second while providing sub-second proximity queries. This definitely rules out traditional relational databases and points us toward specialized solutions.

🔒 System Design

Here’s the system design I am thinking of:

Read more