Skip to content
KR7Tech

Learning guide preview

System design from first principles

Follow one marketplace case study from requirements and core concepts to reliability, security, and senior-level design decisions.

Sign in and explore Pro Back to KR7Tech Student Plan

Enter the PIN separately for each Pro question or lesson. Opening one item does not unlock other items or topics.

System design from first principles

10 free of 314 questions / lessons

1. Workload is more than “users”

Translate users into operations: browse, search, checkout, upload, or background work. Estimate average and peak arrival rate, concurrency, payload size, read/write mix, hot keys, and growth. A thousand mostly-idle users can be easier than one popular product hit by a sudden burst.

in-flight work ≈ arrival rate × average response time

Example: 200 requests/second × 0.25 seconds ≈ 50 requests in flight on average. This is a starting estimate, not the capacity target; peak bursts and headroom still matter. Continue in sizing and traffic shifts.

2. An invariant is a rule that must never be violated

Write the business rule before picking a database. MarketSquare’s invariant: the same unit of inventory cannot be reserved for two active orders. The authoritative inventory owner must enforce that rule atomically; checking a cached count and then writing later is a race.

available = on hand − active reservations − sold

Use a conditional update, constraint, or transaction at the owner of the stock. Search and cache may show an approximate hint, but checkout must consult the authority. See transactions and LLD.

3. A consistency boundary is not always a service boundary

Facts that must change together belong inside a short atomic transaction when possible. A modular monolith can have clear domain ownership while sharing one database transaction. Notifications and search indexing can happen later because they do not decide whether an order is valid.

Rule of thumb: split a deployment boundary only when independent ownership, scaling, release, or isolation is worth the added network failures, duplicated data, retries, and operations. Read advanced HLD and architecture patterns.

4. End-to-end latency is a budget

A request’s response time includes network hops, queueing, application work, and dependencies. Sequential calls add latency; parallel calls finish when the slowest required branch finishes. Percentiles expose the slow tail: a good average can hide a poor p95 or p99 experience.

request deadline = total budget shared across all required hops

Set a deadline, propagate its remaining time, and ensure retries fit inside it. Measure each hop and queue wait before optimizing. See network path and traffic control.

5. Async means “later and recoverable,” not “guaranteed once”

A queue decouples work and absorbs bursts, but consumers may receive a message more than once, out of order, or after a delay. Persist business state and an outbox event together; publish from the outbox; make each consumer idempotent. Show users pending states honestly while projections catch up.

Example: an order is confirmed by its owner first; search, email, and analytics update asynchronously. A delayed email must never undo or redefine the order. Continue in cache and messaging.

6. Overload can create a retry feedback loop

When a dependency slows, callers wait longer, concurrency grows, timeouts trigger retries, and retries add more load to the already-slow dependency. Adding app instances may worsen this if the database is the bottleneck.

Break the loop: deadlines, bounded retries with jitter, concurrency limits, backpressure, admission control, and graceful degradation of optional work. A circuit breaker stops calls temporarily; it does not itself queue work or set a fair per-user quota. See reliability and incident lab.

Topic 1 · Queueing theory: why latency explodes before a service is “full”

Why it matters: requests do not arrive evenly. If work arrives faster than a constrained dependency completes it, requests wait in a queue. Waiting consumes connections, memory, and deadlines. Timeouts trigger retries, which add more arrivals and can create a positive feedback loop.

Arrival rate λRequests / second
Bounded workersOnly N operations can run
Dependency μDB/provider service rate
Queue + deadlineWait, reject, or shed safely

How to reason: Little’s Law says average in-flight work L = λW. At 200 requests/second and 250 ms average time, expect about 50 requests in flight. If a database pool can serve 40 checkout operations/second but 60 arrive/second, the backlog grows by about 20/second until arrivals fall or capacity changes. This simplified example ignores burstiness and service-time variation; real systems need load tests.

As utilization approaches 100%, small bursts and slow requests create disproportionately long waits. Do not solve this by raising every limit: that may move the queue into the database and exhaust its connections. Bound concurrency at the scarce resource, shed or defer work with a clear response, and reserve capacity for critical operations.

ASP.NET Core example · cap per-process checkout concurrency
var checkoutSlots = new SemaphoreSlim(80, 80);

app.MapPost("/checkout", async (
    CheckoutRequest request,
    ICheckoutService checkout,
    CancellationToken cancellationToken) =>
{
    if (!await checkoutSlots.WaitAsync(0, cancellationToken))
        return Results.StatusCode(StatusCodes.Status503ServiceUnavailable);

    try
    {
        return Results.Ok(await checkout.CreateAsync(request, cancellationToken));
    }
    finally
    {
        checkoutSlots.Release();
    }
});

The limit of 80 is an example, not a recommendation. A process-local semaphore multiplies by the number of app instances; it is not a global quota. Select the limit from database-pool budgets and load tests, return a retry hint where safe, and ensure the client uses bounded retries with jitter.

Measure: arrival/completion rate, active and waiting requests, queue age, p95/p99 latency, timeout/retry rate, DB pool wait, lock wait, and downstream saturation. See traffic control and traffic shifts.

Topic 2 · Sagas and the transactional outbox: recover across independent systems

Why it matters: a SQL transaction cannot atomically commit both a database row and a remote payment-provider action. Holding a database transaction open during a network call ties up locks and connections, yet a crash can still occur after the provider acts and before your app records the result.

1 · Order DBCommit Pending order + PaymentRequested outbox row
2 · Outbox relayPublish durable message; mark delivered
3 · Payment workerCall provider with stable idempotency key
4 · Result ownerRecord outcome + next outbox event
Failure path: unknown provider result → keep order/payment Pending → query provider or consume signed callback → reconcile → confirm or release reservation according to policy. Never guess “failed” from a timeout.

How to build it: each step is a local transaction with a durable state transition. The saga is a workflow, not a magic distributed rollback. A compensation (such as releasing a reservation) is a new business action; it may itself fail and must be retryable. Define which states are terminal, how late payment is handled, and how an operator can repair a stuck workflow.

SQL shape · save business intent and event together
BEGIN;

INSERT INTO orders (order_id, status, idempotency_key)
VALUES (:order_id, 'PENDING_PAYMENT', :request_key);

INSERT INTO outbox (event_id, aggregate_id, event_type, payload)
VALUES (:event_id, :order_id, 'PaymentRequested', :payload);

COMMIT;

The relay may publish and crash before marking the row delivered, so it can publish again. Consumers must deduplicate with a durable event/business key and commit their deduplication record with their own side effect. “Exactly once” is achieved as an idempotent business effect, not assumed from transport delivery.

Observed outcomePersisted stateSafe next action
Provider confirms successPayment = Succeeded; order transition is conditionalEmit OrderConfirmed once; fulfill idempotently
Provider confirms declinePayment = Declined; order remains unconfirmedRelease reservation through an idempotent compensation
Request times out / response unknownPayment = Unknown or PendingQuery by provider key/reference or reconcile callback; do not send a new logical charge
Duplicate/out-of-order callbackKeep valid monotonic state transition and event IDDeduplicate and ignore stale transition; alert on impossible state

Test: crash after DB commit, after publish, after provider success, and before local result commit. Re-run each operation and prove there is still one order, one logical charge, and a recoverable state. See reliability and distributed primitives.

Topic 3 · Replicas and read-your-writes: explain what “stale” means

Why it matters: a primary can acknowledge a write before an asynchronous replica has replayed it. A shopper updates an address, then immediately reads from a lagging replica and sees the old value. Replication improves read capacity and recovery options; it does not automatically provide read-after-write consistency.

Client writesUpdate address
Primary commits v42Returns commit version / token
Replica is v40It has not replayed the write
Read policyRoute to primary or wait boundedly for v42

How to choose: use the primary for checkout, authorization, and immediate confirmation when stale data could violate an invariant. For a profile or catalog read, a short stale window may be acceptable. For read-your-writes, return a version/commit token and route the caller to the primary (or wait briefly for a replica watermark); if the bound expires, return an explicit pending/stale result rather than silently claiming the update disappeared.

Optimistic concurrency · reject a stale edit instead of overwriting newer data
UPDATE products
SET title = :new_title,
    version = version + 1
WHERE product_id = :product_id
  AND version = :version_the_client_read;

-- 1 row changed: update accepted.
-- 0 rows changed: tell the client the record changed; reload/merge.

This compare-and-swap protects concurrent edits at the writer. It is not a substitute for authorization, validation, or a transaction when multiple rows must change together.

Interview check: name the authority, the allowed stale window, and the behavior when a replica is behind. “We have replicas” is not a consistency promise. Continue in replication and partitioning.

Topic 4 · SLOs and error budgets: translate reliability into product decisions

Why it matters: “high availability” is not measurable. An SLO names the user-visible journey, the indicator, the target, and the measurement window. It lets product and engineering make an explicit trade-off between shipping change and protecting reliability.

User journeyPlace a valid order
SLISuccessful eligible checkouts / eligible attempts
SLOFor example, 99.9% over 30 days
Error budgetAllowed unsuccessful fraction drives action

Worked example: with a 99.9% success target over a 30-day window, the allowed failure fraction is 0.1%. If measured as time-based availability, that is about 43.2 minutes in a 30-day month. For a request-based checkout SLI, calculate the budget from eligible attempts instead; do not convert it to downtime.

How to use it: define eligibility and exclusions before measuring (for example, separate invalid user input from server failures). Track latency as a distribution, not just an average; pair success with correctness, because a fast oversell is not a successful checkout. Alert on rapid budget burn and investigate slower trends. If the budget is being consumed too quickly, reduce risky releases, fix the dominant failure mode, and communicate the customer impact.

Useful signals: successful eligible journey rate · p95/p99 latency · saturation/queue age · duplicate-charge and oversell count · dependency timeout rate · recovery time. Each signal needs an owner, dashboard, alert threshold, and runbook action.

Do not do this: set an ambitious availability number without a measurement, treat every endpoint as equally important, or page on a metric that has no clear action. See operations and the full checklist.

Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only
Pro only