Skip to content

Glossary

The vocabulary of system design, A to Z: distributed systems, data, caching, messaging, reliability, observability, networking, security, cloud-native platforms and AI systems. Each entry is a one- or two-sentence definition with, when there is one, a link to the class page or the component catalog entry where the idea is taught in depth.

How to use

Use your browser's search (Ctrl/Cmd + F) or the site search to jump to a term. When you read an unfamiliar word in a class, a paper or an interview question, look it up here, then follow the link. Try defining the terms out loud in one sentence before reading the definition: that is exactly what an interviewer will ask you to do. The numbers page has the values behind the formulas mentioned here.

204 terms.

A

  • ACID β€” Atomicity, Consistency, Isolation, Durability: the guarantees of a database transaction β€” all or nothing, invariants kept, concurrent transactions isolated, committed data survives crashes. β†’ Databases β€Ί Transactions
  • Active-active / active-passive β€” Two redundancy styles: in active-active every site serves traffic at the same time; in active-passive a standby takes over only when the primary fails. β†’ Reliability β€Ί Disaster recovery
  • Admission control β€” Limiting how much work enters a system so that what is admitted completes well; a virtual waiting room is admission control for people. β†’ Case study: ticketing
  • Agent (AI) β€” An LLM running in a loop that decides which tools to call, observes the results and continues until the task is done. β†’ AI systems β€Ί Agents
  • Alerting β€” Rules over signals that notify a human when users are affected; good alerts are about symptoms and are actionable. β†’ Catalog β€Ί Alerting
  • Amdahl's Law β€” The speedup from parallelism is limited by the serial fraction of the work: speedup = 1 / ((1 βˆ’ p) + p/n). β†’ Estimation
  • ANN (approximate nearest neighbor) β€” Search that finds vectors close to a query vector quickly by accepting that it may miss a few true nearest neighbors; HNSW is a common index. β†’ Catalog β€Ί Vector DB
  • Anycast β€” Announcing the same IP address from many locations so the network routes each user to the nearest one; used by DNS providers and CDNs. β†’ Edge β€Ί DNS & CDN
  • API gateway β€” The single entry point of an API: routing, authentication, rate limiting, request shaping and observability. β†’ Catalog β€Ί API Gateway
  • At-least-once / at-most-once delivery β€” Messaging guarantees: at-least-once never loses a message but may deliver duplicates; at-most-once never duplicates but may lose messages. β†’ Messaging β€Ί Delivery semantics
  • Autoscaling β€” Adding or removing capacity automatically based on a metric (CPU, queue depth, requests per second) or a schedule. β†’ Compute β€Ί Serverless & autoscaling
  • Availability β€” The fraction of time (or of requests) in which a system works correctly, often expressed in nines: 99.9% allows about 8.8 hours of downtime a year. β†’ Estimation β€Ί Availability math
  • Availability zone (AZ) β€” One or more isolated data centers within a cloud region, with independent power and networking; spreading replicas over AZs survives a data-center failure. β†’ Catalog β€Ί Subnet

B

  • Backoff (exponential) β€” Waiting longer between successive retries (1 s, 2 s, 4 s…) so that a struggling dependency can recover. β†’ Reliability β€Ί Patterns
  • Back-of-the-envelope estimate β€” A quick calculation of load, storage, bandwidth or cost with rounded numbers, to get the order of magnitude that drives the design. β†’ Estimation β€Ί Numbers
  • Backpressure β€” A signal from a slow consumer that makes producers slow down or reject work instead of letting queues grow without limit. β†’ Messaging β€Ί Delivery semantics
  • Bandwidth β€” The maximum data rate of a link (e.g. 1 Gbps = 125 MB/s); throughput is what you actually achieve. β†’ Estimation β€Ί Numbers
  • BFF (backend for frontend) β€” A gateway or thin service tailored to one client type (web, mobile) that aggregates and shapes backend calls for it. β†’ Edge β€Ί API gateway
  • Bloom filter β€” A compact probabilistic set that answers "definitely not present" or "probably present"; used to skip disk reads and to deduplicate. β†’ Databases β€Ί Data models
  • Blue/green deployment β€” Running the new version in a full second environment and switching all traffic to it at once, with instant rollback by switching back. β†’ Cloud-native β€Ί Delivery
  • B-tree β€” The balanced tree index used by most relational databases: updates pages in place and gives fast point and range reads. β†’ Databases β€Ί Data models
  • Bulkhead β€” Isolating resources (thread pools, connections, instances) per dependency or tenant so one failure cannot consume everything. β†’ Reliability β€Ί Patterns
  • Burn rate β€” How fast an SLO's error budget is being consumed relative to the sustainable rate; a burn rate of 14.4 spends 2% of a 28-day budget in one hour. β†’ Observability β€Ί Alerting

C

  • Cache β€” A fast store (usually in memory) holding copies of data that is expensive to fetch or compute. β†’ Catalog β€Ί Cache
  • Cache-aside β€” The application reads the cache first and, on a miss, reads the database and fills the cache; on writes it updates the database and deletes the key. β†’ Caching β€Ί Patterns
  • Cache stampede β€” Many concurrent misses for the same hot key hitting the backend at once, typically when the key expires; also called thundering herd. β†’ Caching β€Ί Failure modes
  • Canary release β€” Sending a small share of traffic to a new version and comparing its metrics with the old one before rolling out further. β†’ Cloud-native β€Ί Delivery
  • Capacity planning β€” Estimating the resources needed to meet expected load with headroom, and when to add more, from measurements and growth forecasts. β†’ Performance β€Ί Capacity planning
  • CAP theorem β€” During a network partition, a replicated system must choose between consistency (refuse or delay) and availability (answer, possibly stale). β†’ Distributed systems β€Ί CAP & PACELC
  • Cardinality (metrics) β€” The number of distinct time series produced by a metric's label combinations; high-cardinality labels such as user ids explode cost. β†’ Catalog β€Ί Metrics
  • CDC (change data capture) β€” Streaming the changes committed to a database (from its log) to other systems such as Kafka, a cache or a data warehouse. β†’ Messaging β€Ί Patterns
  • CDN (content delivery network) β€” A global network of edge caches that serves content from locations close to users. β†’ Catalog β€Ί CDN
  • Chaos engineering β€” Running controlled experiments that inject failures (latency, crashes, zone outages) to find weaknesses before real incidents do. β†’ Reliability β€Ί Chaos engineering
  • Circuit breaker β€” A wrapper around calls to a dependency that stops calling it (fails fast) after too many errors, and probes it again after a pause. β†’ Reliability β€Ί Patterns
  • Cold start β€” The extra latency of the first request to a new instance (function, container, JVM) while it initializes. β†’ Catalog β€Ί Serverless
  • Consensus β€” Getting a group of nodes to agree on a value or an ordered log despite failures, as long as a majority is up (Raft, Paxos). β†’ Distributed systems β€Ί Consensus
  • Consistent hashing β€” Placing nodes and keys on a hash ring so that adding or removing a node moves only about 1/N of the keys. β†’ Edge β€Ί Load balancing
  • Consumer group β€” A set of consumers sharing a Kafka topic: each partition is read by exactly one member, so parallelism is at most the partition count. β†’ Catalog β€Ί Kafka
  • Container β€” A process packaged with its dependencies in an image and isolated by the operating system (namespaces, cgroups). β†’ Cloud-native β€Ί Containers
  • Coordinated omission β€” A load-testing error where a slow system makes the client send fewer requests, hiding the latency real users would see. β†’ Performance β€Ί Load testing
  • CQRS β€” Command Query Responsibility Segregation: separate models (and often stores) for writes and for reads, kept in sync by events. β†’ Messaging β€Ί Patterns
  • Cron β€” A schedule expression (and the classic Unix tool) for running jobs at fixed times or intervals. β†’ Catalog β€Ί Scheduler

D

  • Data warehouse β€” A columnar database for analytics over large history (OLAP), fed from operational systems by ETL/ELT or CDC. β†’ Catalog β€Ί Data Warehouse
  • Dead-letter queue (DLQ) β€” A queue where messages that keep failing are parked after N attempts so they do not block the rest. β†’ Catalog β€Ί Message Queue
  • Denormalization β€” Storing redundant copies of data to make reads cheaper, at the cost of more complex writes and possible inconsistency. β†’ Databases β€Ί Data models
  • Disaster recovery (DR) β€” The plan and mechanisms to restore service after a large failure (region loss, data corruption), measured by RPO and RTO. β†’ Reliability β€Ί Disaster recovery
  • Distributed lock β€” A lock shared by several processes or machines, usually with a lease (expiry) and a fencing token to stay safe when holders pause. β†’ Compute β€Ί Async work
  • DNS (Domain Name System) β€” The internet's directory that translates names into IP addresses, with cached answers controlled by TTLs. β†’ Catalog β€Ί DNS
  • Durability β€” The guarantee that committed data is not lost, typically by writing to disk and replicating to other machines or zones. β†’ Intro β€Ί Requirements

E

  • Edge computing β€” Running code at CDN locations close to users, for redirects, authentication checks, personalization or A/B tests. β†’ Edge β€Ί DNS & CDN
  • Embedding β€” A vector of numbers representing the meaning of a text, image or item, so that similar things have nearby vectors. β†’ Catalog β€Ί Vector DB
  • Error budget β€” The amount of unreliability an SLO allows (100% βˆ’ SLO), spent on releases and experiments; when it runs out, reliability work comes first. β†’ Observability β€Ί SLOs
  • ETL / ELT β€” Extract-Transform-Load (transform before loading) or Extract-Load-Transform (load raw data, transform inside the warehouse). β†’ Databases β€Ί Specialized stores
  • Event-driven architecture β€” Services communicate by publishing and reacting to events instead of calling each other directly, which decouples them in time. β†’ Messaging
  • Event sourcing β€” Storing the sequence of events that changed an entity as the source of truth, and deriving the current state by replaying them. β†’ Messaging β€Ί Patterns
  • Eventual consistency β€” If writes stop, all replicas eventually converge to the same value; until then, reads may return stale data. β†’ Distributed systems β€Ί Consistency
  • Exactly-once β€” Each message affects the result exactly once; in practice achieved as at-least-once delivery plus idempotent processing or transactions. β†’ Messaging β€Ί Delivery semantics

F

  • Failover β€” Switching traffic or the leader role to a standby when the primary fails, automatically or manually. β†’ Reliability β€Ί Disaster recovery
  • Fallback β€” An alternative answer when a dependency fails: cached data, a default, a degraded feature or a "try again later". β†’ Reliability β€Ί Patterns
  • Fan-out (on write / on read) β€” Delivering one item to many recipients: precompute each follower's feed when posting (on write) or assemble it when reading (on read). β†’ Case study: news feed
  • Feature flag β€” A runtime switch that turns code paths on or off for some users without a deploy; decouples release from deployment. β†’ Cloud-native β€Ί Delivery
  • FinOps β€” The practice of making cloud cost visible and owned by engineering teams, and optimizing it alongside performance and reliability. β†’ Cloud-native β€Ί FinOps
  • Flame graph β€” A visualization of sampled stack traces where width shows time spent; the fastest way to see where CPU goes. β†’ Performance β€Ί Bottlenecks
  • Functional requirement β€” What the system must do (buy a ticket, search events), as opposed to how well it must do it. β†’ Intro β€Ί Requirements

G

  • Geohash β€” An encoding of latitude and longitude into a string whose shared prefixes mean nearby areas; used to index locations. β†’ Case study: ride sharing
  • GitOps β€” Declaring the desired state of infrastructure and applications in Git and letting an agent (e.g. Argo CD) reconcile the cluster to it. β†’ Cloud-native β€Ί Delivery
  • Golden signals β€” Google SRE's four signals for a service: latency, traffic, errors and saturation. β†’ Observability β€Ί Signals
  • Gossip protocol β€” Nodes periodically exchange state with random peers so information (membership, failures) spreads through the cluster without a coordinator. β†’ Distributed systems
  • GraphQL β€” An API query language where the client asks for exactly the fields it needs from a typed schema, in one request. β†’ Compute β€Ί API styles
  • gRPC β€” A high-performance RPC framework over HTTP/2 with Protocol Buffers schemas and streaming; common between services. β†’ Compute β€Ί API styles
  • Guardrails (LLM) β€” Checks around a model on inputs, outputs and tool calls: prompt injection, personal data, policy, schema validation. β†’ Catalog β€Ί Safety Mesh

H

  • Head-of-line blocking β€” When one slow item at the front of a queue or connection delays everything behind it; HTTP/2 fixed it per request, HTTP/3 per packet. β†’ Edge β€Ί Protocols
  • Health check β€” A probe (liveness, readiness, startup) that tells load balancers and orchestrators whether an instance can serve or should be restarted. β†’ Catalog β€Ί Health Check
  • Hit ratio β€” The fraction of cache lookups served from the cache; the backend sees requests Γ— (1 βˆ’ hit ratio). β†’ Caching β€Ί Patterns
  • HNSW β€” Hierarchical Navigable Small World: a graph-based index for approximate nearest-neighbor search over vectors. β†’ Catalog β€Ί Vector DB
  • Horizontal scaling β€” Adding more machines behind a load balancer (scale out), which requires stateless services and shared state elsewhere. β†’ Intro β€Ί Scaling journey
  • Hot key / hot partition β€” A key or partition receiving a disproportionate share of traffic (a celebrity, a flash-sale event), saturating one node while others idle. β†’ Databases β€Ί Partitioning
  • HPA (Horizontal Pod Autoscaler) β€” The Kubernetes controller that changes a Deployment's replica count to keep a metric (e.g. CPU at 60%) near its target. β†’ Cloud-native β€Ί Kubernetes
  • HTTP/2 and HTTP/3 (QUIC) β€” HTTP/2 multiplexes many requests over one TCP connection; HTTP/3 runs over QUIC (UDP) to avoid TCP head-of-line blocking and speed up handshakes. β†’ Edge β€Ί Protocols

I

  • IaC (infrastructure as code) β€” Defining infrastructure in versioned files (Terraform, OpenTofu) applied by tools, instead of manual clicks. β†’ Cloud-native β€Ί Delivery
  • Idempotency β€” Doing an operation twice has the same effect as doing it once; it makes retries safe. β†’ Distributed systems
  • Idempotency key β€” A unique id sent by the client with a request; the server stores the first result and returns it for repeats instead of acting again. β†’ Compute β€Ί Integrations
  • Index β€” An auxiliary data structure that speeds up reads on some columns at the cost of slower writes and more storage. β†’ Databases β€Ί Data models
  • Ingress β€” The Kubernetes resource (and controller) that routes external HTTP traffic to services; the Gateway API is its successor. β†’ Catalog β€Ί Ingress
  • Inverted index β€” A map from each term to the documents containing it; the core of full-text search engines. β†’ Catalog β€Ί Search
  • Isolation level β€” How much concurrent transactions can see of each other: read committed, repeatable read / snapshot, serializable. β†’ Databases β€Ί Transactions

J

  • Jitter β€” Randomness added to retry delays or timers so that many clients do not act in sync and overload a service together. β†’ Reliability β€Ί Patterns
  • JWT (JSON Web Token) β€” A signed token carrying claims (user, scopes, expiry) that services can verify locally without calling the issuer. β†’ Network & security β€Ί Identity

K

  • Kafka β€” A distributed, partitioned, replicated event log where consumers read at their own offsets and can replay history. β†’ Catalog β€Ί Kafka
  • Key-value store β€” A database that maps keys to opaque values with very fast get/put by key and little else (Redis, DynamoDB). β†’ Catalog β€Ί NoSQL DB
  • Knee (of the latency curve) β€” The utilization (often around 70–80%) beyond which queueing makes latency grow sharply. β†’ Estimation β€Ί Queueing
  • Kubernetes β€” A container orchestrator that schedules pods on nodes, keeps the desired number running, and provides services, probes and autoscaling. β†’ Cloud-native β€Ί Kubernetes

L

  • L4 / L7 load balancing β€” L4 balances TCP/UDP connections by address and port; L7 understands HTTP and routes by path, header or cookie. β†’ Catalog β€Ί Load Balancer
  • Latency β€” The time to serve one request, best described by percentiles (p50, p99) rather than the average. β†’ Estimation β€Ί Numbers
  • Leader election β€” Choosing one node among many to coordinate (accept writes, run a job), with a new election when it fails. β†’ Distributed systems β€Ί Consensus
  • Leader-follower replication β€” One leader accepts writes and streams them to followers that serve reads and can take over on failure. β†’ Databases β€Ί Replication
  • Leaky bucket β€” A rate-limiting algorithm that queues requests and lets them out at a constant rate, smoothing bursts. β†’ Edge β€Ί API gateway
  • Linearizability β€” The strongest single-object consistency: every operation appears to take effect atomically at one instant, as if there were one copy. β†’ Distributed systems β€Ί Consistency
  • Little's Law β€” L = Ξ» Γ— W: the average number of items in a system equals the arrival rate times the average time each spends in it. β†’ Estimation β€Ί Queueing
  • LLM gateway β€” A proxy in front of model providers for routing, fallbacks, quotas, caching and token/cost metrics. β†’ Catalog β€Ί LLM Gateway
  • Load balancer β€” A component that spreads requests across healthy replicas using an algorithm such as round robin or least connections. β†’ Catalog β€Ί Load Balancer
  • Load shedding β€” Deliberately rejecting some requests (with 429/503) when overloaded so the rest are served within their deadlines. β†’ Reliability β€Ί Patterns
  • Load test β€” Generating realistic traffic against a system to measure throughput and latency; types include smoke, load, stress, spike, soak and breakpoint. β†’ Performance β€Ί Load testing
  • Logical clock β€” A counter-based clock (Lamport, vector) that orders events by causality instead of wall time. β†’ Distributed systems β€Ί Time & clocks
  • Logs β€” Timestamped records of discrete events, ideally structured (JSON) and correlated by trace id. β†’ Catalog β€Ί Logs
  • LRU / LFU β€” Cache eviction policies: evict the least recently used item, or the least frequently used one. β†’ Caching β€Ί Redis
  • LSM-tree β€” Log-Structured Merge tree: writes go to memory and are flushed to sorted files that are later compacted; very fast writes. β†’ Databases β€Ί Data models

M

  • MCP (Model Context Protocol) β€” An open protocol through which applications expose tools, resources and prompts to LLM clients in a standard way. β†’ Catalog β€Ί Tool Registry
  • Message queue β€” A buffer where producers put messages and competing consumers take them, each message processed by one consumer. β†’ Catalog β€Ί Message Queue
  • Metrics β€” Numeric time series aggregated over time (counters, gauges, histograms); cheap to store, the basis of dashboards and alerts. β†’ Catalog β€Ί Metrics
  • Microservices β€” An architecture of small services owned by teams and deployed independently, communicating over the network. β†’ Compute β€Ί Services
  • Modular monolith β€” One deployable application split internally into well-separated modules with clear boundaries; often the best starting point. β†’ Compute β€Ί Services
  • mTLS (mutual TLS) β€” TLS in which both client and server present certificates, so each side authenticates the other; standard inside service meshes. β†’ Catalog β€Ί Service Mesh
  • MTTR / MTBF β€” Mean time to repair (recover) and mean time between failures; availability β‰ˆ MTBF / (MTBF + MTTR). β†’ Reliability
  • Multi-leader replication β€” Several nodes accept writes and replicate to each other, which requires resolving conflicting writes. β†’ Databases β€Ί Replication
  • Multi-region β€” Running a system in more than one geographic region for lower latency to distant users and survival of a region outage. β†’ Reliability β€Ί Disaster recovery

N

  • N+1 query β€” Code that runs one query for a list and then one more per item, turning one page into hundreds of database round trips. β†’ Performance β€Ί Bottlenecks
  • NAT gateway β€” Lets machines in private subnets open outbound connections to the internet without being reachable from it. β†’ Catalog β€Ί NAT Gateway
  • Network partition β€” A failure where parts of a system cannot communicate with each other although each part is still running. β†’ Distributed systems β€Ί CAP & PACELC
  • NewSQL β€” Distributed databases that keep SQL and transactions while scaling horizontally (Spanner, CockroachDB, YugabyteDB). β†’ Databases β€Ί Data models
  • Non-functional requirement β€” How well the system must work: latency, availability, consistency, durability, scale, cost, security, operability. β†’ Intro β€Ί Requirements
  • NoSQL β€” Non-relational databases (key-value, document, wide-column, graph) that trade some relational features for scale or flexibility. β†’ Catalog β€Ί NoSQL DB

O

  • OAuth 2 / OIDC β€” OAuth 2 delegates authorization with access tokens; OpenID Connect adds authentication (who the user is) on top of it. β†’ Network & security β€Ί Identity
  • Object storage β€” Storage of immutable blobs by key over HTTP (S3 and similar), cheap and extremely durable; the origin for media. β†’ Catalog β€Ί Object Store
  • Observability β€” The ability to understand a system's internal state from the signals it emits: metrics, logs, traces and profiles. β†’ Observability
  • OLTP / OLAP β€” Online transaction processing (many small reads and writes on current data) vs online analytical processing (large scans and aggregates over history). β†’ Databases β€Ί Specialized stores
  • OpenTelemetry β€” The vendor-neutral standard (APIs, SDKs, Collector, OTLP protocol) for producing and exporting metrics, logs and traces. β†’ Observability β€Ί OpenTelemetry
  • Open vs closed workload model β€” Open: requests arrive at a rate regardless of responses (real users); closed: a fixed number of users wait for each response before the next. β†’ Performance β€Ί Load testing
  • Optimistic locking β€” Detecting conflicting updates at write time with a version column (UPDATE … WHERE version = ?) instead of locking in advance. β†’ Databases β€Ί Transactions
  • Orchestrator (agent) β€” The component that runs an AI agent's loop: calls the model, executes the tools it chooses, and repeats until done. β†’ Catalog β€Ί Orchestrator
  • Outbox (transactional) β€” Writing an event into an outbox table in the same transaction as the business change, then relaying it to the broker; avoids dual writes. β†’ Messaging β€Ί Patterns

P

  • p50 / p99 (percentiles) β€” The latency below which 50% or 99% of requests complete; p99 describes the tail that users feel. β†’ Estimation β€Ί Percentiles
  • PACELC β€” Extends CAP: if there is a Partition choose Availability or Consistency; Else choose Latency or Consistency. β†’ Distributed systems β€Ί CAP & PACELC
  • Partitioning (sharding) β€” Splitting data across nodes by a key (hash or range) so that each node holds and serves only part of it. β†’ Databases β€Ί Partitioning
  • Partition (Kafka) β€” An ordered, append-only shard of a topic; ordering holds only within a partition, and it is the unit of consumer parallelism. β†’ Catalog β€Ί Kafka
  • Pessimistic locking β€” Locking rows before changing them (SELECT … FOR UPDATE) so concurrent transactions wait; safe under contention, but holds locks. β†’ Databases β€Ί Transactions
  • Pod β€” The smallest deployable unit in Kubernetes: one or more containers sharing a network address and volumes. β†’ Cloud-native β€Ί Kubernetes
  • Prompt injection β€” Untrusted text (a web page, an email, a review) carrying instructions that an LLM follows against its intended behavior. β†’ Catalog β€Ί Safety Mesh
  • Pub/Sub β€” Publish/subscribe: each message published to a topic is delivered to every subscriber. β†’ Catalog β€Ί Pub/Sub

Q

  • QPS (queries per second) β€” Requests or queries per second, the basic unit of load; average QPS β‰ˆ requests per day Γ· 10^5. β†’ Estimation β€Ί Numbers
  • Queueing theory β€” The mathematics of waiting lines; explains why latency explodes as utilization approaches 100%. β†’ Estimation β€Ί Queueing
  • Quorum β€” The minimum number of replicas that must take part in a read or write; with R + W > N, reads overlap the latest write. β†’ Distributed systems β€Ί Consensus

R

  • Raft β€” A consensus algorithm designed to be understandable: a leader replicates a log to followers and commits entries once a majority has them. β†’ Distributed systems β€Ί Consensus
  • RAG (retrieval-augmented generation) β€” Retrieving relevant chunks of your data (usually by vector search) and putting them in the prompt so the model answers from them. β†’ Catalog β€Ί Memory Fabric
  • Rate limiting β€” Capping how many requests a client may make per time window, answering 429 Too Many Requests beyond it. β†’ Case study: rate limiter
  • Read replica β€” A follower copy of a database used to serve reads; adds read capacity but may lag behind the leader. β†’ Databases β€Ί Replication
  • Read-your-writes β€” A consistency guarantee that a user always sees their own latest writes, even if other users may see stale data. β†’ Distributed systems β€Ί Consistency
  • Redis β€” An in-memory data-structure server (strings, hashes, lists, sorted sets, streams) used as cache, rate limiter, queue and more. β†’ Caching β€Ί Redis
  • RED method β€” Monitoring every service by Rate, Errors and Duration. β†’ Observability β€Ί Signals
  • Region β€” A geographic area of a cloud provider containing several availability zones (e.g. SΓ£o Paulo, Virginia). β†’ Reliability β€Ί Disaster recovery
  • Replication lag β€” The delay between a write on the leader and its appearance on a follower; causes stale reads. β†’ Databases β€Ί Replication
  • Resharding β€” Moving data between partitions when nodes are added or keys are redistributed, ideally online and gradually. β†’ Databases β€Ί Partitioning
  • REST β€” An API style built on HTTP resources, verbs and status codes; simple, cacheable and the default for public APIs. β†’ Compute β€Ί API styles
  • Retry β€” Repeating a failed request in the hope that the failure was transient; safe only for idempotent operations, with backoff and a limit. β†’ Reliability β€Ί Patterns
  • Retry storm β€” Retries from many clients multiplying the load on a struggling service and turning a blip into an outage. β†’ Reliability β€Ί Patterns
  • Reverse proxy β€” A server in front of backends that receives client requests and forwards them, often terminating TLS and caching (Nginx, Envoy). β†’ Catalog β€Ί Load Balancer
  • RPO / RTO β€” Recovery point objective (how much data you may lose) and recovery time objective (how long until service is back). β†’ Reliability β€Ί Disaster recovery

S

  • Saga β€” A distributed transaction split into local steps, each with a compensating action that undoes it if a later step fails. β†’ Messaging β€Ί Patterns
  • SBOM (software bill of materials) β€” An inventory of every component and version inside a software artifact, used to track vulnerabilities and licenses. β†’ Network & security β€Ί Zero trust
  • Scheduler β€” A component that triggers jobs at given times or intervals, ensuring each run happens once even with several replicas. β†’ Catalog β€Ί Scheduler
  • Secrets management β€” Storing credentials and keys in a dedicated, audited system (vault) and injecting them at runtime, never in code or images. β†’ Network & security β€Ί Zero trust
  • Semantic cache β€” A cache for LLM answers keyed by the meaning of the question (embedding similarity) rather than its exact text. β†’ Catalog β€Ί LLM Gateway
  • Serverless (FaaS) β€” Running functions on demand without managing servers, billed per invocation and duration, scaling to zero. β†’ Catalog β€Ί Serverless
  • Service discovery β€” How a service finds the current addresses of another (DNS, a registry, Kubernetes Services). β†’ Cloud-native β€Ί Kubernetes
  • Service mesh β€” A layer of proxies beside every service providing mTLS, retries, timeouts, traffic splitting and telemetry without code changes. β†’ Catalog β€Ί Service Mesh
  • Single point of failure (SPOF) β€” A component whose failure stops the whole system; removed by redundancy. β†’ Estimation β€Ί Availability math
  • SLI / SLO / SLA β€” Indicator (what you measure), objective (the target, e.g. 99.9% over 28 days) and agreement (a contract with penalties). β†’ Observability β€Ί SLOs
  • Snapshot isolation β€” Each transaction reads from a consistent snapshot of the database; prevents most anomalies but allows write skew. β†’ Databases β€Ί Transactions
  • Soak / spike / stress test β€” Load tests that hold load for hours (soak), jump suddenly (spike) or increase until the system breaks (stress). β†’ Performance β€Ί Load testing
  • SSE (Server-Sent Events) β€” A one-way stream of events from server to client over a long-lived HTTP response; used for live updates and LLM tokens. β†’ Edge β€Ί Protocols
  • Stateless service β€” A service that keeps no client state between requests, so any replica can serve any request. β†’ Catalog β€Ί App Server
  • Sticky session β€” Routing all requests of a client to the same backend; simplifies in-memory sessions but hurts balance and failover. β†’ Catalog β€Ί Load Balancer
  • Strong consistency β€” Reads always reflect the latest committed write (linearizability or serializability), at a cost in latency or availability. β†’ Distributed systems β€Ί Consistency
  • Subnet β€” A range of addresses inside a VPC in one availability zone; public or private depending on its route table. β†’ Catalog β€Ί Subnet

T

  • Tail latency β€” The slowest requests (p99, p999); amplified when a request fans out to many services. β†’ Estimation β€Ί Percentiles
  • Throughput β€” The amount of work completed per unit of time (requests per second, MB per second). β†’ Estimation β€Ί Numbers
  • Thundering herd β€” Many clients or threads waking up and hitting the same resource at once (after an expiry, a restart, a lock release). β†’ Caching β€Ί Failure modes
  • Timeout β€” A deadline on a call after which the caller gives up; the first defense against slow dependencies. β†’ Reliability β€Ί Patterns
  • Token bucket β€” A rate-limiting algorithm: tokens refill at a steady rate up to a burst capacity, and each request spends one. β†’ Edge β€Ί API gateway
  • Token (LLM) β€” The unit of text a language model reads and writes (roughly ΒΎ of an English word); models are priced and limited in tokens. β†’ AI systems β€Ί LLM gateway
  • Tool calling β€” An LLM returning a structured request to call a function (with JSON arguments) that the application executes. β†’ Catalog β€Ί Tool Registry
  • Tracing (distributed) β€” Following one request across services as a tree of timed spans sharing a trace id. β†’ Catalog β€Ί Tracing
  • TTFT (time to first token) β€” The delay before an LLM starts streaming its answer; dominates perceived latency in chat interfaces. β†’ AI systems β€Ί LLM gateway
  • TTL (time to live) β€” How long a cached item or DNS answer stays valid before it must be fetched again. β†’ Caching β€Ί Invalidation
  • Two-phase commit (2PC) β€” A protocol that commits a transaction across several participants atomically via a coordinator; blocks if the coordinator fails. β†’ Messaging β€Ί Patterns

U

  • Universal Scalability Law β€” Extends Amdahl with a coherence cost: beyond some point, adding nodes reduces throughput. β†’ Estimation
  • USE method β€” Checking every resource for Utilization, Saturation and Errors to find bottlenecks. β†’ Observability β€Ί Signals
  • Utilization β€” The fraction of time a resource is busy (ρ = arrival rate Γ· capacity); latency grows sharply as it approaches 1. β†’ Estimation β€Ί Queueing

V

  • Vector clock β€” A vector of counters, one per node, that detects whether two events are causally ordered or concurrent. β†’ Distributed systems β€Ί Time & clocks
  • Vector database β€” A store optimized for similarity search over embeddings, with ANN indexes and metadata filters. β†’ Catalog β€Ί Vector DB
  • Vertical scaling β€” Moving to a bigger machine (scale up): simple, but with a ceiling and a single point of failure. β†’ Intro β€Ί Scaling journey
  • Virtual nodes β€” Many positions per physical node on a consistent-hashing ring, to spread load evenly and rebalance smoothly. β†’ Edge β€Ί Load balancing
  • VPC (virtual private cloud) β€” An isolated private network in a cloud region, where you define subnets, routes and firewall rules. β†’ Catalog β€Ί VPC
  • VPN β€” An encrypted tunnel joining two networks (or a laptop and a network) over the internet. β†’ Catalog β€Ί VPN

W

  • WAF (web application firewall) β€” A filter for HTTP traffic that blocks attacks (injection, bad bots, abusive patterns) before they reach the application. β†’ Catalog β€Ί WAF
  • Waiting room (virtual queue) β€” A queue in front of a high-demand flow that admits users at the rate the system can sustain, turning a spike into a steady flow. β†’ Case study: ticketing
  • Webhook β€” An HTTP callback: a provider calls your endpoint when something happens (payment confirmed); must be verified and processed idempotently. β†’ Compute β€Ί Integrations
  • WebSocket β€” A full-duplex, long-lived connection between browser and server, used for chat, live updates and games. β†’ Edge β€Ί Protocols
  • Worker β€” A process that takes jobs from a queue and executes them in the background, scaled by backlog. β†’ Catalog β€Ί Worker
  • Write-ahead log (WAL) β€” A log where a database records every change before applying it, used for crash recovery and replication. β†’ Databases β€Ί Replication
  • Write skew β€” An anomaly where two transactions read overlapping data and make disjoint writes that together break an invariant; prevented by serializable isolation. β†’ Databases β€Ί Transactions
  • Write-through / write-behind β€” Cache write policies: write to cache and database together (through), or to the cache first and flush later (behind). β†’ Caching β€Ί Patterns

Z

  • Zero trust β€” A security model in which no network location is trusted: every request is authenticated, authorized and encrypted. β†’ Network & security β€Ί Zero trust
  • Zipf distribution β€” A popularity distribution where a few items get most of the traffic; why caches work and why hot keys happen. β†’ Caching β€Ί Failure modes