Resilience Patterns
When a downstream service degrades or fails, the naive default β keep sending requests and wait for a response β reliably produces cascading failures. The calling service's threads block, its queue fills, and the failure propagates upstream to clients who had nothing to do with the broken dependency.
Resilience patterns are design decisions that interrupt this propagation. They do not prevent failures; they limit how far a failure travels through the system1.
flowchart LR
client --> gateway
gateway -->|"timeout<br/>circuit breaker<br/>bulkhead"| downstream["downstream<br/>(slow)"]
gateway -->|"fast failure<br/>or fallback"| client The patterns below are complementary β production systems use them in combination, not in isolation. Throughout the page we use one running example from the store: the order service calls the product service (through an OpenFeign client, as in Hands-on 1) to check prices before saving an order.
| Pattern | Question it answers | Protects against |
|---|---|---|
| Timeout | How long am I willing to wait? | Hanging calls |
| Retry | Is it worth trying again? | Transient failures |
| Circuit breaker | Should I even try? | A dependency that is known to be down |
| Fallback | What do I answer instead? | Total loss of a feature |
| Bulkhead | How much of my capacity can one dependency take? | Resource exhaustion spreading |
| Rate limiting | How much traffic do I accept? | Spikes and abusive clients |
| Health checks | Should traffic reach this instance? | Routing to broken instances |
Timeout
A timeout is a deadline applied to an outbound call. If the downstream service does not respond within the deadline, the call is abandoned and an error is returned to the caller immediately.
Without a timeout, a single hanging dependency can hold threads indefinitely. With a timeout, the maximum damage any single slow call can do is bounded.
sequenceDiagram
participant C as Client
participant G as Gateway
participant P as Payment Service
C->>+G: POST /checkout
G->>+P: POST /charge
Note over P: processing...
Note over G: deadline exceeded (500 ms)
G-->>-C: 504 Gateway Timeout
P-->>-G: (response arrives late β discarded) Choosing timeout values
- Percentile-based: set the timeout at the P99.9 latency of the downstream service under normal conditions. Anything slower than that is already abnormal.
- Budget-aware: if the upstream call has its own timeout of 1 s, all downstream calls within that request β retries and their waits included β must complete well under 1 s combined.
- Distinct per dependency: a database query, a third-party API, and an internal microservice have very different latency profiles. A single global timeout value is almost always wrong.
There are usually two timeouts on an HTTP call: the connect timeout (how long to wait to open the TCP connection) and the read timeout (how long to wait for the response once connected). Configure both.
Timeouts on the store's Feign client
# order-service application.yaml (Spring Cloud OpenFeign)
spring:
cloud:
openfeign:
client:
config:
product: # the @FeignClient name
connect-timeout: 500 # ms
read-timeout: 1000 # ms
If the product service takes more than one second, the order service gets an exception instead of a blocked thread, and can decide what to do next.
The timeout budget across hops
Timeouts nest: each hop must finish inside the deadline of the hop that called it. Suppose the gateway waits at most 3 s for the order service, and the order service calls product with the settings of this page: read timeout 1 s, 3 attempts, waits of 0.5 s and 1 s. In the worst case the order service needs 1 + 0.5 + 1 + 1 + 1 = 4.5 s β longer than anyone waits for it.
sequenceDiagram
participant C as Client
participant G as Gateway (waits 3 s)
participant O as Order service
participant P as Product service (slow)
C->>G: POST /orders
G->>O: POST /orders
O->>P: attempt 1
Note over O: read timeout at 1 s, then wait 0.5 s
O->>P: attempt 2
Note over O: read timeout at 2.5 s, then wait 1 s
Note over G: 3 s gone: gives up
G-->>C: 504 Gateway Timeout
O->>P: attempt 3 (nobody waits for it)
Note over O: fails at 4.5 s: wasted work and load Fit the inner budget inside the outer one: 2 attempts (1 + 0.5 + 1 = 2.5 s), a shorter read timeout, or a deadline passed along with the request so each hop knows how much time is left (gRPC does this with deadlines).
Timeouts are only the first step
A timeout on its own just converts a hang into a fast error. Combine it with a retry policy (below) for transient failures, and with a circuit breaker and a fallback for longer outages.
Retry with Exponential Backoff
A retry re-attempts a failed request after a waiting period. Retries are appropriate for transient failures β brief network blips, momentary service restarts, or transient resource exhaustion β where the same request is likely to succeed on a subsequent attempt.
sequenceDiagram
participant C as Client
participant S as Service
C->>+S: Request (attempt 1)
S-->>-C: 503 Service Unavailable
Note over C: wait 1 s
C->>+S: Request (attempt 2)
S-->>-C: 503 Service Unavailable
Note over C: wait 2 s
C->>+S: Request (attempt 3)
S-->>-C: 200 OK β Exponential backoff
Retrying immediately on failure is dangerous: if many clients all retry simultaneously after a service restart, the resulting spike can overwhelm the recovering service β a thundering herd. Exponential backoff spaces retries out:
where \(n\) counts the retries from 0 (the wait before the first retry is \(c\)):
| Before retry | Formula | Wait (c = 1 s, max_wait = 16 s) | Time since the first failure |
|---|---|---|---|
| 1st (\(n\) = 0) | \(1 \cdot 2^0\) | 1 s | 1 s |
| 2nd (\(n\) = 1) | \(1 \cdot 2^1\) | 2 s | 3 s |
| 3rd (\(n\) = 2) | \(1 \cdot 2^2\) | 4 s | 7 s |
| 4th (\(n\) = 3) | \(1 \cdot 2^3\) | 8 s | 15 s |
| 5th (\(n\) = 4) | \(1 \cdot 2^4\) | 16 s (reaches max_wait) | 31 s |
| 6th and later | capped | 16 s | +16 s each |
Jitter
Even with exponential backoff, clients that all failed at the same moment will retry at the same moments β a synchronised wave of retries. Adding a random jitter breaks this synchronisation:
Marc Brooker's analysis at AWS2 compared several variants and recommends full jitter, which randomises the entire wait instead of adding a small random amount:
With 1,000 clients failing at the same instant, full jitter spreads their retries uniformly over the interval instead of sending 1,000 requests at exactly t = 1 s, 3 s and 7 s. (Brooker found full jitter and "decorrelated" jitter both far better than none; full jitter is the simplest.)
sequenceDiagram
participant C as 1,000 clients
participant S as Product service (just restarted)
Note over C,S: backoff without jitter
C->>S: t = 0 s: 1,000 requests fail together
C->>S: t = 1 s: 1,000 retries in the same millisecond
S-->>C: overloaded again: 503 for most
C->>S: t = 3 s: the same wave, again
Note over C,S: backoff with full jitter
C->>S: retries spread over 0 to 1 s: about 1 per millisecond
S-->>C: a steady load it can absorb: 200 OK What to retry β and what not to
| Response | Retry? | Why |
|---|---|---|
| Connection refused, connection reset, timeout | Yes | Typical transient failure |
503 Service Unavailable, 429 Too Many Requests | Yes, with backoff (respect Retry-After if present) | The server is asking you to come back later |
500 Internal Server Error | Maybe, once | Often a bug that will fail again |
400, 401, 403, 404, 422 | No | The request itself is wrong; retrying cannot fix it |
Retry amplification
Retries multiply load exactly when the system is weakest. If the gateway, the order service and the product client each make up to 3 attempts, one user click can become 3 Γ 3 Γ 3 = 27 calls to a struggling product service. Retry at one layer only (usually the one closest to the failing dependency), cap the number of attempts, and stop retrying when the circuit breaker is open.
flowchart LR
U["1 click"] --> G["gateway<br/>3 attempts"]
G -->|"3 calls"| O["order service<br/>3 attempts each"]
O -->|"9 calls"| F["Feign client<br/>3 attempts each"]
F -->|"27 calls"| P["product service<br/>(already struggling)"] Do not retry non-idempotent operations
Retrying a POST /orders that creates a new order may create duplicates. Only retry operations that are idempotent (GET, PUT, DELETE) or that are explicitly designed for retry safety (idempotency keys, deduplication tokens).
An operation is idempotent when doing it twice has the same effect as doing it once. The common way to make a POST safe to retry is an idempotency key: the client generates a unique ID (e.g. a UUID) per logical operation and sends it in a header; the server stores the keys it has already processed and, on a repeated key, returns the stored result instead of creating a second order.
sequenceDiagram
participant C as Client
participant O as Order service
participant DB as Postgres
C->>O: POST /orders (Idempotency-Key: k-42)
O->>DB: key k-42 seen? no β insert order + key
O--xC: 201 Created (response lost)
C->>O: retry POST /orders (Idempotency-Key: k-42)
O->>DB: key k-42 seen? yes
O-->>C: 201 Created (same order, no duplicate) Circuit Breaker
The circuit breaker3 is named by analogy with the electrical component: when the current load exceeds safe limits, the breaker trips and interrupts the circuit. In software, when the failure rate of the calls to a downstream service reaches a threshold, the circuit breaker opens and immediately rejects calls to that service β without attempting the network call at all.
This prevents the calling service from wasting threads and connections on a dependency that is known to be unavailable.
stateDiagram-v2
[*] --> Closed
Closed --> Open : 50% or more of the last 10 calls failed or were slow
Open --> HalfOpen : 30 s in Open, then the next call
HalfOpen --> Closed : the 3 probe calls stay below the thresholds
HalfOpen --> Open : the probes fail or are slow again Closed (normal operation) : Requests pass through. The breaker records the outcome of the last calls (a sliding window). Once enough calls are recorded, if the share that failed β or that were slower than a threshold β reaches the configured rate, the breaker trips to Open.
Open (failing fast) : All requests are rejected immediately with an error (CallNotPermittedException in Resilience4j) β no network call is made. This protects the caller's resources and gives the downstream service time to recover. After the configured wait duration, the breaker moves to Half-Open.
Half-Open (testing recovery) : A limited number of probe calls are allowed through; any other call is still rejected. When the probes are done, the breaker computes their failure and slow-call rates: below the thresholds, the dependency has recovered and the breaker closes; otherwise it returns to Open for another wait.
The sequence below plays one cycle with the configuration of the example further down:
sequenceDiagram
participant O as Order service
participant B as Breaker (product)
participant P as Product service
Note over B: Closed
O->>B: findProduct
B->>P: GET /products/42
P--xB: no answer: read timeout after 1 s
Note over B: 5 of the last 10 calls failed: Open
O->>B: findProduct
B-->>O: CallNotPermittedException in under 1 ms
Note over B: 30 s later, the next call: Half-Open
O->>B: findProduct (probe 1 of 3)
B->>P: GET /products/42
P-->>B: 200 OK in 50 ms
Note over B: 3 probes fast and OK: Closed Configuration knobs
| Parameter (Resilience4j property) | Default | Typical value | Effect |
|---|---|---|---|
Failure threshold (failure-rate-threshold) | 50% | 50% | Share of failed calls in the window that opens the breaker |
Window (sliding-window-size, minimum-number-of-calls) | 100 calls, 100 | 10β20 calls | How many recent calls are judged; no decision before the minimum |
Slow calls (slow-call-duration-threshold, slow-call-rate-threshold) | 60 s, 100% | below the timeout, 50% | Slow calls open the breaker too, before they time out |
Wait in Open (wait-duration-in-open-state) | 60 s | 10β60 s | How long the dependency gets to recover |
Half-Open probes (permitted-number-of-calls-in-half-open-state) | 10 | 3β5 | How cautiously to test recovery |
The defaults are conservative: with a window of 100 calls, the breaker of a service that receives 1 call per second needs well over a minute to decide. Slow calls are judged by their own rate β with the default 100%, they open the breaker only when every call in the window is slow.
Spring Boot: the Resilience4j library4 implements all of these patterns with @CircuitBreaker, @Retry, @Bulkhead, @RateLimiter and @TimeLimiter annotations. With Spring Boot 4 (the version of the hands-on), the dependencies are io.github.resilience4j:resilience4j-spring-boot4 and spring-boot-starter-aspectj; with Spring Boot 3 they were resilience4j-spring-boot3 and spring-boot-starter-aop.
Circuit breaker and retry around the product call
// Resilience4j's default order: Retry ( CircuitBreaker ( call ) ).
// The fallback sits on the OUTERMOST layer, so it runs once: after
// the retries are exhausted, or at once when the breaker is open.
@Retry(name = "product", fallbackMethod = "priceFallback")
@CircuitBreaker(name = "product")
public ProductOut findProduct(String id) {
return productController.findById(id).getBody(); // Feign call
}
// Same signature + the exception.
private ProductOut priceFallback(String id, Throwable t) {
throw new ResponseStatusException(
HttpStatus.SERVICE_UNAVAILABLE, "catalogue unavailable, try again later");
}
resilience4j:
circuitbreaker:
instances:
product:
sliding-window-size: 10 # the last 10 calls
minimum-number-of-calls: 10 # judge only after 10 calls
failure-rate-threshold: 50 # >= 50% failed β Open
slow-call-duration-threshold: 500ms # slower than this = slow (below the 1 s timeout)
slow-call-rate-threshold: 50 # >= 50% slow β Open
wait-duration-in-open-state: 30s # Open β Half-Open
permitted-number-of-calls-in-half-open-state: 3
retry:
instances:
product:
max-attempts: 3 # 1 call + 2 retries
wait-duration: 500ms
enable-exponential-backoff: true
exponential-backoff-multiplier: 2 # 500 ms, then 1 s
enable-randomized-wait: true
randomized-wait-factor: 0.5 # each wait Β± 50% (jitter)
retry-exceptions:
- feign.RetryableException # timeouts and connection errors
- feign.FeignException$ServiceUnavailable # 503
ignore-exceptions:
- io.github.resilience4j.circuitbreaker.CallNotPermittedException # breaker open: fail fast
The breaker's state is exported as a metric (resilience4j_circuitbreaker_state) through Actuator, so you can graph it in Grafana (Hands-on 6).
Three details of this configuration matter. The slow-call threshold (500 ms) is below the read timeout (1 s): a call slower than the timeout never completes, so a threshold above it would never see a slow call. The retry lists the exceptions worth retrying and ignores CallNotPermittedException: when the breaker is open there is nothing to wait for. And the fallback is on @Retry, the outer annotation: placed on @CircuitBreaker, it would run inside the retry and turn every failure into the fallback's exception before the retry sees it β so the retry would either retry the fallback (if it retried everything) or never retry at all.
Play it: a slow product service
The simulator puts the order service of the overview β 200 threads, 60 orders per second β in front of a product service that degrades for one minute, and lets you switch on the timeout, the retry and the breaker of the configuration above.
Try it
With no pattern, the 3-second calls keep about 176 threads busy (Little's law: 60 Γ 3 s = 180) and the pool hits its 200 threads in 8 of the 60 seconds: only 98.4% of GET /orders succeed, although they never call product. Switch on the timeout: about 60 threads (60 Γ 1 s) and GET /orders back to 100% β but every order now fails after 1 s. Add the retry: each order holds its thread for up to 4.5 s, the pool is full during 51 seconds, only 71.2% of GET /orders succeed and product receives 128 calls per second instead of 60 β worse than no pattern at all. Add the breaker: it opens 1.1 s after the fault begins, orders fail in under a millisecond, about 2 threads stay busy, the probes at 41 s fail and the breaker closes at t = 72.2 s, 2.2 s after product recovers. Then pick product down: with timeout and retry but no breaker, product receives 3.0 calls per order while it refuses them all.
Fallback and graceful degradation
Failing fast is better than hanging, but an error is still an error. A fallback is what the caller does instead when a call fails or the breaker is open. The goal is graceful degradation: lose a feature, not the whole page.
| Dependency down | Possible fallback in the store |
|---|---|
| Recommendation service | Show the "best sellers" list cached in Redis (Hands-on 8) |
| Product service, for reads | Serve the last cached product data, marked as possibly stale |
| Notification service | Publish the e-mail request to Kafka and send it later (Event-Driven Architecture) |
| Payment service | No safe fallback: answer 503 quickly with a clear message β never pretend the payment succeeded |
A fallback must be safe and simple
A fallback that calls another remote service can fail too (and needs its own protection). And some operations β charging a card, reserving stock β must not be faked: failing clearly is the correct degraded behaviour.
Bulkhead
The bulkhead pattern β named after watertight compartments in a ship hull β isolates resources by dependency. Rather than sharing a single thread pool across all downstream calls, each dependency gets its own pool.
flowchart TD
gateway --> poolA["Bulkhead A<br/>(max 10 concurrent calls)"]
gateway --> poolB["Bulkhead B<br/>(max 10 concurrent calls)"]
poolA --> payment["Payment Service<br/>(degraded β οΈ)"]
poolB --> inventory["Inventory Service<br/>(healthy β
)"] If Payment Service degrades and fills all 10 slots of Bulkhead A, Bulkhead B is completely unaffected. Inventory Service continues to handle requests at full speed. Without bulkheads, a single slow dependency saturates the shared pool and degrades all other calls equally. Step by step, with the order service's 200 threads:
sequenceDiagram
participant O as Order service (200 threads)
participant A as Bulkhead payment (10)
participant P as Payment (slow)
participant I as Inventory
O->>A: charge (calls 1 to 10)
A->>P: 10 calls waiting 3 s each
O->>A: charge (call 11)
A-->>O: BulkheadFullException at once, fallback 503
O->>I: reserve stock (its own bulkhead has room)
I-->>O: 200 OK in 30 ms
Note over O,I: payment holds at most 10 threads, the other 190 keep working The cost of bulkheads is capacity that cannot be shared: with ten pools of 10, a burst on one dependency is refused at 10 concurrent calls even while the other 90 slots sit idle, whereas one pool of 100 would have absorbed it. Sizing each bulkhead is the work (Little's law again: calls per second Γ normal latency, plus a margin). The trade-off is almost always worth it for critical-path dependencies.
Resilience4j offers two kinds of bulkhead: a semaphore bulkhead (max-concurrent-calls, default 25, limits how many threads may be inside the call at once β cheap, the default; max-wait-duration: 0 rejects at once when full) and a thread-pool bulkhead (the call runs in a dedicated pool). The same idea applies outside the JVM: separate database connection pools per workload, or separate Kubernetes deployments for critical and non-critical traffic.
Rate Limiting
Rate limiting caps the number of requests a client (or the system) can make within a time window, protecting services from traffic spikes and abusive clients.
Token bucket algorithm
The token bucket is the most common rate limiting algorithm. It models a bucket that fills at a constant rate with tokens; each request consumes one token; requests that arrive when the bucket is empty are rejected or queued.
flowchart LR
refill["Token source<br/>(refill rate: 100 req/s)"] -->|adds tokens| bucket["Token Bucket<br/>(capacity: 200)"]
request["Incoming<br/>Request"] -->|consumes 1 token| bucket
bucket -->|tokens available| allowed["β
Allowed"]
bucket -->|bucket empty| rejected["β 429 Too Many Requests"] | Property | Effect |
|---|---|
| Refill rate | Steady-state maximum throughput |
| Bucket capacity (burst) | How much instantaneous traffic above the rate is tolerated |
A bucket capacity larger than the refill rate allows short bursts while still enforcing a long-term average. Spring Cloud Gateway's RequestRateLimiter filter (backed by Redis) and AWS API Gateway use a token bucket. Nginx limit_req uses the closely related leaky bucket: requests drain at a fixed rate and the burst parameter sets how many may wait in the queue.
Rate limiting at the store's edge (Nginx)
# 10 requests/s per client IP; up to 20 extra requests of a burst are
# served at once (nodelay) and their slots free up at 10/s; beyond that, 429
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
server {
location / {
limit_req zone=api burst=20 nodelay;
limit_req_status 429;
proxy_pass http://gateway;
}
}
This fits in front of the gateway replicas of Hands-on 7. Rate limits per user or per API key (rather than per IP) are usually applied in the gateway, where the authenticated identity is known.
Health Checks
Health check endpoints allow orchestrators (Kubernetes, Docker Compose, load balancers) to detect failed instances and stop routing traffic to them before clients notice.
Two distinct types serve different purposes:
Liveness : Is the process alive? Returns 200 OK if the application is running and not deadlocked. If a liveness check fails, the orchestrator restarts the container.
Readiness : Is the service ready to serve traffic? Returns 200 OK only if the instance has finished starting and can do useful work (e.g. its own database is reachable). If a readiness check fails, the orchestrator removes the instance from the load balancer pool without restarting it.
Startup (Kubernetes) : Has the application finished starting? Liveness and readiness checks are not run until it succeeds, so a slow-starting JVM is not killed during boot.
The sequence plays a 10-second database blip on a pod whose readiness includes the database and whose liveness does not:
sequenceDiagram
participant K as Kubernetes
participant P as order pod
participant D as Postgres
Note over D: blip: unreachable for 10 s
K->>P: GET /actuator/health/readiness
P-->>K: 503 DOWN (db)
Note over K: removes the pod from the Service, no restart
K->>P: GET /actuator/health/liveness
P-->>K: 200 UP (the process is fine)
Note over D: reachable again
K->>P: GET /actuator/health/readiness
P-->>K: 200 UP
Note over K: the pod receives traffic again Spring Boot Actuator exposes both at /actuator/health/liveness and /actuator/health/readiness. In Spring Boot 4 (the hands-on's version) these probe groups are enabled by default, in any environment; in Spring Boot 2.3 to 3.x they were automatic only on Kubernetes, and elsewhere (e.g. Docker Compose) you set management.endpoint.health.probes.enabled=true. By default they reflect only the application's internal state; add a dependency to readiness explicitly, e.g. management.endpoint.health.group.readiness.include=readinessState,db.
Probes for the order service on Kubernetes
# inside the container spec of the Deployment
startupProbe:
httpGet: { path: /actuator/health/liveness, port: 8080 }
failureThreshold: 30 # up to 30 Γ 5 s = 150 s to start
periodSeconds: 5
livenessProbe:
httpGet: { path: /actuator/health/liveness, port: 8080 }
periodSeconds: 10
readinessProbe:
httpGet: { path: /actuator/health/readiness, port: 8080 }
periodSeconds: 5
See Hands-on 5 for the Deployments these probes belong to.
Never put external dependencies in the liveness check
If liveness checks the database and the database blips, Kubernetes restarts every replica at the same time β turning a 10-second database hiccup into a full outage while all JVMs boot again. Liveness should answer only "is this process stuck?". Be careful even with readiness: if every replica depends on the same database, they all become unready together and the service disappears instead of returning clear errors.
Defence in depth
These patterns are layered, not chosen in isolation. A production-grade call chain combines all of them:
flowchart LR
client --> rl["Rate limiter<br/>at the edge (Nginx, gateway)"]
rl --> retry
subgraph order["order service, around each call to product"]
retry["Retry + backoff<br/>(outermost)"] --> cb["Circuit breaker<br/>(counts every attempt)"]
cb --> bh["Bulkhead<br/>(caps the threads on product)"]
bh --> to["Timeout<br/>(bounds each attempt)"]
end
to --> downstream["product service"] Each layer handles a different failure scenario:
| Layer, outermost first | What it handles | Why in this position |
|---|---|---|
| Rate limiter (inbound) | Traffic spikes, abusive clients | Refuses excess work before it costs anything |
| Retry + backoff | Transient errors | Outside the breaker: an open breaker stops the retries at once |
| Circuit breaker | Known-failing dependencies | Sees every attempt, timeouts included, as a call to judge |
| Bulkhead | Slow dependencies bleeding into other calls | Caps the threads that the attempts can hold |
| Timeout | Unbounded waiting | Innermost: bounds each attempt, not the sum |
The order is a design decision
This is the order of Resilience4j's Spring Boot integration, whose default aspect order is Retry ( CircuitBreaker ( RateLimiter ( TimeLimiter ( Bulkhead ( call ) ) ) ) ) (the Feign read timeout plays the time limiter's role here, inside the call). Microsoft's standard resilience handler for .NET's HttpClient stacks the same way: rate limiter β total timeout β retry β circuit breaker β attempt timeout5. Whatever the library, check that retries do not bypass the breaker and that attempts Γ timeout + waits fit in the caller's own deadline (the timeout budget).
Connections
- Hands-on 1 β the Feign clients that need timeouts and breakers.
- Hands-on 7 β Nginx in front of the gateway, where edge rate limiting lives.
- Hands-on 10 β retries and dead letter queues for asynchronous consumers.
- SLI / SLO / SLA β the budget these patterns protect.
Key takeaways
- Every remote call needs a timeout (connect + read), chosen per dependency and inside the caller's budget β attempts Γ timeout + waits included.
- Retry only transient failures and idempotent operations, with exponential backoff and full jitter, at one layer only; use idempotency keys to make
POSTsafe. - A circuit breaker (Closed β Open β Half-Open) fails fast on a dependency known to be down or slow β set its slow-call threshold below the timeout; pair it with a safe fallback on the outermost layer.
- A timeout plus a retry without a breaker can be worse than nothing: each call holds its thread for every attempt and wait, and the dependency receives several times the load.
- Bulkheads keep one slow dependency from consuming all threads or connections; rate limiting (token / leaky bucket) protects the service from excess traffic.
- Liveness restarts, readiness removes from the load balancer; never check external dependencies in liveness.
- The patterns are layered; their order and combined timeouts must be designed, then verified with chaos experiments.
-
NYGARD, M. Release It! Design and Deploy Production-Ready Software, 2nd ed. Pragmatic Programmers, 2018.Β β©
-
BROOKER, M. Exponential Backoff And Jitter. AWS Architecture Blog, 2015.Β β©
-
FOWLER, M. CircuitBreaker. martinfowler.com, 2014.Β β©
-
MICROSOFT. Build resilient HTTP apps: key development patterns. Microsoft Learn (standard resilience handler defaults).Β β©