Resilient LLM Calls in TypeScript: Retries, Breakers, Fallbacks

1. What you'll build

An ask(prompt) function that calls Claude Opus 5 with exponential-backoff retries, trips a circuit breaker after repeated failures, and answers from Claude Sonnet 5 whenever the primary model is overloaded, rate-limited, unreachable, or gone. About 70 lines, built on Cockatiel and the Anthropic TypeScript SDK.

2. Prerequisites

• Node.js 24 LTS (verified on 24.20.0). Node runs .ts files directly since 22.18, so there's no build step. Cockatiel 4 requires Node 22 or newer.
• @anthropic-ai/sdk 0.122.0 and cockatiel 4.0.0, the current releases as of this writing.
• typescript 7.0.2 and @types/node, used only for tsc --noEmit; Node strips the types at runtime.
• An Anthropic API key exported as ANTHROPIC_API_KEY, with access to claude-opus-5 and claude-sonnet-5.
• Commands are for macOS/Linux. On Windows, set the environment variables with $env:NAME = "value" in PowerShell.

3. Set up the project

mkdir resilient-llm && cd resilient-llm
npm init -y
npm pkg set type=module
npm install @anthropic-ai/sdk cockatiel
npm install -D typescript @types/node

type: module matters: Cockatiel 4 ships ESM only. Add a minimal tsconfig.json so your editor and tsc understand .ts imports:

{
  "compilerOptions": {
    "target": "es2022",
    "module": "nodenext",
    "strict": true,
    "noEmit": true,
    "allowImportingTsExtensions": true,
    "skipLibCheck": true
  },
  "include": ["src"]
}

4. Define the retry and circuit-breaker policies

Create src/resilient.ts. The first job is deciding which errors count as transient. The SDK throws typed subclasses of Anthropic.APIError with a status field, which makes this a one-liner:

import Anthropic from "@anthropic-ai/sdk";
import {
  BrokenCircuitError,
  ConsecutiveBreaker,
  ExponentialBackoff,
  circuitBreaker,
  fallback,
  handleWhen,
  retry,
  wrap,
} from "cockatiel";

export const PRIMARY_MODEL = process.env.PRIMARY_MODEL ?? "claude-opus-5";
export const FALLBACK_MODEL = process.env.FALLBACK_MODEL ?? "claude-sonnet-5";

// maxRetries: 0 hands retries to Cockatiel. The SDK default of 2 would
// multiply every Cockatiel attempt by three.
const client = new Anthropic({ maxRetries: 0, timeout: 60_000 });

// Worth retrying: network errors, 408/409/429, and any 5xx (529 = overloaded).
// 400/401/403/404 are our mistake or a dead model; retrying just burns time.
const isTransient = (err: unknown): boolean =>
  err instanceof Anthropic.APIConnectionError ||
  (err instanceof Anthropic.APIError &&
    (err.status === 408 ||
      err.status === 409 ||
      err.status === 429 ||
      (err.status ?? 0) >= 500));

// 3 retries after the first call, starting around 500ms and capped at 8s, with jitter.
export const retryPolicy = retry(handleWhen(isTransient), {
  maxAttempts: 3,
  backoff: new ExponentialBackoff({ initialDelay: 500, maxDelay: 8_000 }),
});

// Open after 5 consecutive transient failures; let one probe through after 30s.
export const breaker = circuitBreaker(handleWhen(isTransient), {
  halfOpenAfter: 30_000,
  breaker: new ConsecutiveBreaker(5),
});

// Retry outside, breaker inside. An open circuit throws BrokenCircuitError,
// which isTransient doesn't match, so the retry loop stops immediately.
const primaryPolicy = wrap(retryPolicy, breaker);

retryPolicy.onRetry(({ attempt, delay }) =>
  console.warn(`[retry] ${PRIMARY_MODEL} failed; retry ${attempt}/3 in ${Math.round(delay)}ms`),
);
breaker.onBreak(() => console.warn(`[breaker] open: fast-failing ${PRIMARY_MODEL} for 30s`));
breaker.onHalfOpen(() => console.warn(`[breaker] half-open: probing ${PRIMARY_MODEL}`));
breaker.onReset(() => console.warn(`[breaker] closed: ${PRIMARY_MODEL} healthy again`));

Two details: maxAttempts counts retries after the initial call, so this policy makes up to four requests. And because the breaker sits inside the retry, every failed attempt feeds ConsecutiveBreaker; one fully failed request is four of the five strikes.

5. Add the model fallback

Append this to src/resilient.ts. The fallback policy is built per call because its factory can't see the wrapped function's arguments and needs the prompt. It holds no state, so that's free:

// Fall back when the primary is unhealthy or gone, not when the request itself is bad.
const shouldFallBack = (err: unknown): boolean =>
  isTransient(err) ||
  err instanceof BrokenCircuitError ||
  err instanceof Anthropic.NotFoundError;

export interface Answer {
  model: string;
  text: string;
}

async function callModel(model: string, prompt: string): Promise<Answer> {
  const message = await client.messages.create({
    model,
    max_tokens: 1024,
    messages: [{ role: "user", content: prompt }],
  });
  const text = message.content
    .filter((block) => block.type === "text")
    .map((block) => block.text)
    .join("");
  return { model: message.model, text };
}

export async function ask(prompt: string): Promise<Answer> {
  const withFallback = fallback(handleWhen(shouldFallBack), () => {
    console.warn(`[fallback] ${PRIMARY_MODEL} unavailable; answering with ${FALLBACK_MODEL}`);
    return retryPolicy.execute(() => callModel(FALLBACK_MODEL, prompt));
  });

  return await withFallback.execute(() =>
    primaryPolicy.execute(() => callModel(PRIMARY_MODEL, prompt)),
  );
}

NotFoundError is on the fallback list on purpose: a retired or mistyped model ID returns 404, and that's exactly when traffic should move. BadRequestError (400) is not on the list; a malformed request fails on the secondary too, so let it throw.

The fallback call reuses retryPolicy but skips the breaker. The breaker tracks the primary's health, and sharing it would let a sick primary block a healthy secondary.

Now src/index.ts. Node's type stripping requires the .ts extension on relative imports:

import { ask } from "./resilient.ts";

const { model, text } = await ask(
  "In one sentence, what does a circuit breaker do in a distributed system?",
);
console.log(`[${model}] ${text}`);

6. Verify it works

Type-check, then run:

npx tsc
node src/index.ts

tsc prints nothing on success. On a healthy day the primary answers and no policy logs fire:

[claude-opus-5] A circuit breaker stops sending requests to a failing service so the failure can't cascade, then periodically tests whether it has recovered.

Now force the fallback path by pointing the primary at a model that doesn't exist. The API returns a 404, which skips retries and goes straight to Sonnet:

$ PRIMARY_MODEL=claude-nope-1 node src/index.ts
[fallback] claude-nope-1 unavailable; answering with claude-sonnet-5
[claude-sonnet-5] A circuit breaker detects repeated failures from a dependency and temporarily blocks calls to it, giving it time to recover instead of cascading the failure.

During a real 529 storm you'll see [retry] ... retry 1/3 in 612ms lines before either the primary recovers or [fallback] fires. If several requests fail back to back, [breaker] open appears and later requests skip the primary for 30 seconds.

Node 24.1 through 24.19 print ExperimentalWarning: Type Stripping is an experimental feature on startup. It's harmless; 24.20 dropped it.

7. Troubleshooting

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" You're on Node 20 or an early 22.x. Type stripping shipped by default in 22.18 and 23.6. Upgrade to 24 LTS, or run the file with npx tsx src/index.ts on older versions.

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '.../src/resilient' imported from .../src/index.ts The import is missing its extension. Node doesn't resolve ./resilient to ./resilient.ts; write the .ts explicitly. The allowImportingTsExtensions flag in tsconfig.json keeps tsc from objecting.

Error: Could not resolve authentication method. Expected one of apiKey, authToken, credentials, config, or profile to be set. ANTHROPIC_API_KEY isn't visible to the process. Run export ANTHROPIC_API_KEY=sk-ant-... in the same shell, or prefix the command with it. The client reads the variable at construction time, so a key set after import won't be picked up.

BadRequestError: 400 {"type":"error","error":{"type":"invalid_request_error", ...}} The request itself is invalid, so neither the retry nor the fallback policy handles it, by design. Read the message field; it names the offending parameter (for example a max_tokens above the model's limit). Fix the request rather than widening shouldFallBack.

8. Next steps

Honor retry-after on 429s. The SDK's built-in retries do this and Cockatiel's don't; swap ExponentialBackoff for a DelegateBackoff that reads the header off the failed error in the retry context. For production, replace ConsecutiveBreaker with SamplingBreaker so a 20% failure rate over 30 seconds opens the circuit instead of five unlucky requests in a row, and wire onBreak/onReset into your metrics. Cross-provider fallback is the same shape: point the fallback factory at a different SDK and keep the policies unchanged.

The Claude API also has a server-side fallback for a different failure: a request the model declines rather than an HTTP error. See Refusals and fallback for the fallbacks: "default" beta.