Settle builds every option with IBM Bob, then measures them 2026-09-27 03:46 UTC · base 8e30082
The question

How do we make the top customers endpoint fast?

3 of 3 options built by IBM Bob in parallel, each in its own branch, then measured with the same 15 second load test at concurrency 8.

Verdict

No option meets every constraint.

Closest is Index and rewrite, which misses p95 latency. Relax a constraint or add an option.

Decision map

Each dot is one option Bob built. Drag the two lines to set your limits. Anything in the shaded corner is good enough, and the smallest change in there wins.

1 ms10 ms100 ms1 sfresh15 s30 s45 s60 s75 s p95 latency, log scale → ↑ staleness ≤ 50 ms ≤ 15 s Cache 2.1 ms · 60.2 s stale Materialized view 16 ms · 5 s stale Index and rewrite 189 ms · fresh Today (no change) 698 ms · fresh
MeasuredToday (no change)CacheMaterialized viewIndex and rewrite
p95 latencyneeds ≤ 50 ms, median of 3 runs 698 ms (658 ms–858 ms)2 ms (2 ms–2 ms)✓16 ms (15 ms–17 ms)✓189 ms (179 ms–193 ms)✕
Requestsfailed requests disqualify 587 · 0 failed274,267 · 0 failed✓32,342 · 0 failed✓2,234 · 0 failed✓
Median latency 662 ms1 ms10 ms159 ms
Throughput 12.7 req/s6025.5 req/s718 req/s49.7 req/s
Stalenessneeds ≤ 15 s instant60.2 s✕5 s✓instant✓
Code changedapplication code, tests excluded +0 −0+42 −2+34 −12+29 −10
Tests added +0 lines+105 lines+36 lines+42 lines
Files touched 0232
New dependencies nonenonenonenone
Tests pass · 3pass · 10✓pass · 4✓pass · 4✓
Existing testsmust be left untouched —untouched✓changed 1✕untouched✓
Built by Bob in —2m 14s2m 26s2m 21s

✓ meets the constraint · ✕ misses it · first column is today's code, for reference

Trust this comparison

Base commit8e3008204c32, every option branched from it
Load test15 s at concurrency 8, 3 runs per option, median reported, options measured one at a time
Disqualifiersmeasurement errors, no successful requests, more than 1% failed requests, a missing freshness result, or changed existing tests
Cache✓ 0 of 274267 failed · ✓ tests untouched · p95 per run 2 / 2 / 2 ms
Materialized view✓ 0 of 32342 failed · ✕ changed demo/orders-api/test/api.test.ts · p95 per run 15 / 17 / 16 ms
Index and rewrite✓ 0 of 2234 failed · ✓ tests untouched · p95 per run 179 / 193 / 189 ms
Today's code587 requests, 0 failed
Machine11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz · 8 cores · 16 GB · win32 10.0.26200 · Node v24.15.0

The options

Cache

Cache the endpoint's response in memory with a 60 second time to live.

Branch settle/2026-09-27T03-36-05/cache

  • demo/orders-api/src/cache.ts
  • demo/orders-api/src/server.ts
  • demo/orders-api/test/cache.test.ts test
The code Bob wrote
src/cache.ts⋯+/** Minimal in-process TTL cache used by the top-customers endpoint. */++interface Entry<T> {+  value: T;+  expiresAt: number; // epoch ms+}++export class TtlCache<K, V> {+  private readonly store = new Map<K, Entry<V>>();++  constructor(private readonly ttlMs: number) {}++  get(key: K): V | undefined {+    const entry = this.store.get(key);+    if (!entry) return undefined;+    if (Date.now() > entry.expiresAt) {+      this.store.delete(key);+      return undefined;+    }+    return entry.value;+  }++  set(key: K, value: V): void {+    this.store.set(key, { value, expiresAt: Date.now() + this.ttlMs });+  }++  /** Remove all entries — useful in tests to avoid stale state. */+  clear(): void {+    this.store.clear();+  }+}++export const TOP_CUSTOMERS_TTL_MS = 60_000;src/server.ts⋯ import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; import type { PGlite } from "@electric-sql/pglite";+import { TtlCache, TOP_CUSTOMERS_TTL_MS } from "./cache.ts";  export interface TopCustomer {⋯ } -export function buildServer(db: PGlite): Server {+export function buildServer(db: PGlite, cache = new TtlCache<number, TopCustomer[]>(TOP_CUSTOMERS_TTL_MS)): Server {   return createServer(async (req, res) => {     try {⋯       if (req.method === "GET" && url.pathname === "/stats/top-customers") {         const limit = Math.min(Math.max(Number(url.searchParams.get("limit") ?? 10), 1), 100);-        return send(res, 200, await topCustomers(db, limit));+        const cached = cache.get(limit);+        if (cached) {+          return send(res, 200, cached);+        }+        const result = await topCustomers(db, limit);+        cache.set(limit, result);+        return send(res, 200, result);       }

Materialized view

Precompute the ranking in a Postgres materialized view and refresh it in the background every 5 seconds.

Branch settle/2026-09-27T03-36-05/matview

  • demo/orders-api/src/db.ts
  • demo/orders-api/src/index.ts
  • demo/orders-api/src/server.ts
  • demo/orders-api/test/api.test.ts test
The code Bob wrote
src/db.ts⋯      ANALYZE;++    CREATE MATERIALIZED VIEW top_customers_mv AS+      SELECT c.id AS customer_id,+             c.name,+             sum(o.amount_cents)::int AS revenue_cents,+             count(*)::int AS orders+      FROM orders o+      JOIN customers c ON c.id = o.customer_id+      WHERE o.status = 'paid'+        AND o.created_at > now() - interval '30 days'+      GROUP BY c.id, c.name+      ORDER BY revenue_cents DESC;++    CREATE UNIQUE INDEX top_customers_mv_pk ON top_customers_mv (customer_id);   `);   return db; }++// Refreshes the materialized view and returns a handle that can stop the loop.+export function startRefresh(db: PGlite, intervalMs = 5_000): { stop: () => void } {+  const id = setInterval(() => {+    // CONCURRENTLY requires a unique index and lets reads continue during refresh.+    db.exec("REFRESH MATERIALIZED VIEW CONCURRENTLY top_customers_mv;").catch(() => {+      // Ignore transient errors (e.g. db closing during tests).+    });+  }, intervalMs);+  // Don't block Node.js exit if the caller forgets to stop.+  if (typeof id === "object" && "unref" in id) (id as NodeJS.Timeout).unref();+  return { stop: () => clearInterval(id) };+}src/index.ts⋯-import { openDb } from "./db.ts";+import { openDb, startRefresh } from "./db.ts"; import { buildServer } from "./server.ts"; ⋯ const started = Date.now(); const db = await openDb();+startRefresh(db); buildServer(db).listen(port, () => {   console.log(`orders-api listening on ${port} (seeded in ${Date.now() - started} ms)`);src/server.ts⋯ } -// Top customers by paid revenue over the last 30 days. This is the slow-// endpoint the Settle demo debates how to speed up.+// Top customers by paid revenue over the last 30 days.+// Reads from the precomputed materialized view; the background refresher keeps+// it at most ~5 s stale so results are always close to real-time. export async function topCustomers(db: PGlite, limit: number): Promise<TopCustomer[]> {   const { rows } = await db.query<TopCustomer>(-    `SELECT c.id AS customer_id,-            c.name,-            sum(o.amount_cents)::int AS revenue_cents,-            count(*)::int AS orders-     FROM orders o-     JOIN customers c ON c.id = o.customer_id-     WHERE o.status = 'paid'-       AND o.created_at > now() - interval '30 days'-     GROUP BY c.id, c.name+    `SELECT customer_id, name, revenue_cents, orders+     FROM top_customers_mv      ORDER BY revenue_cents DESC      LIMIT $1`,

Index and rewrite

Add the right index or indexes and rewrite the query so Postgres does less work. No caching and no precomputed tables.

Branch settle/2026-09-27T03-36-05/index

  • demo/orders-api/src/db.ts
  • demo/orders-api/src/server.ts
  • demo/orders-api/test/index-freshness.test.ts test
The code Bob wrote
src/db.ts⋯     ); +    -- Covering index for the top-customers query:+    --   equality on status, range on created_at (newest first),+    --   INCLUDE avoids heap fetches for customer_id and amount_cents+    --   so the aggregate can be satisfied by an index-only scan.+    CREATE INDEX orders_status_created_covering+      ON orders (status, created_at DESC)+      INCLUDE (customer_id, amount_cents);+     SELECT setseed(0.42); src/server.ts⋯ export async function topCustomers(db: PGlite, limit: number): Promise<TopCustomer[]> {   const { rows } = await db.query<TopCustomer>(-    `SELECT c.id AS customer_id,+    // Aggregate on orders first (index-only scan on orders_status_created_covering),+    // then join the small aggregated set to customers for the name.+    // This lets Postgres resolve the GROUP BY entirely from the index and only+    // touch the customers table for at most LIMIT rows.+    `WITH ranked AS (+       SELECT o.customer_id,+              sum(o.amount_cents)::int AS revenue_cents,+              count(*)::int           AS orders+       FROM   orders o+       WHERE  o.status     = 'paid'+         AND  o.created_at > now() - interval '30 days'+       GROUP  BY o.customer_id+       ORDER  BY revenue_cents DESC+       LIMIT  $1+     )+     SELECT r.customer_id,             c.name,-            sum(o.amount_cents)::int AS revenue_cents,-            count(*)::int AS orders-     FROM orders o-     JOIN customers c ON c.id = o.customer_id-     WHERE o.status = 'paid'-       AND o.created_at > now() - interval '30 days'-     GROUP BY c.id, c.name-     ORDER BY revenue_cents DESC-     LIMIT $1`,+            r.revenue_cents,+            r.orders+     FROM   ranked r+     JOIN   customers c ON c.id = r.customer_id+     ORDER  BY r.revenue_cents DESC`,     [limit],   );

Design doc appendix

## Appendix: measured comparison

**Question:** How do we make the top customers endpoint fast?

Each option was built by IBM Bob in its own branch from `8e30082`, then measured with the same load test (15 s at concurrency 8) and the same staleness probe.

| | Today (no change) | Cache | Materialized view | Index and rewrite |
|---|---|---|---|---|
| p95 latency | 698 ms (658 ms–858 ms) | 2 ms (2 ms–2 ms) | 16 ms (15 ms–17 ms) | 189 ms (179 ms–193 ms) |
| Requests | 587 · 0 failed | 274,267 · 0 failed | 32,342 · 0 failed | 2,234 · 0 failed |
| Median latency | 662 ms | 1 ms | 10 ms | 159 ms |
| Throughput | 12.7 req/s | 6025.5 req/s | 718 req/s | 49.7 req/s |
| Staleness | instant | 60.2 s | 5 s | instant |
| Code changed | +0 −0 | +42 −2 | +34 −12 | +29 −10 |
| Tests added | +0 lines | +105 lines | +36 lines | +42 lines |
| Files touched | 0 | 2 | 3 | 2 |
| New dependencies | none | none | none | none |
| Tests | pass · 3 | pass · 10 | pass · 4 | pass · 4 |
| Existing tests | — | untouched | changed 1 | untouched |
| Built by Bob in | — | 2m 14s | 2m 26s | 2m 21s |

**Constraints:** `max_p95_ms: 50`, `max_staleness_seconds: 15`, `tests_must_pass: true`

**Verdict:** No option meets every constraint. Closest is Index and rewrite, which misses p95 latency. Relax a constraint or add an option.

**Branches:**
- Cache: `settle/2026-09-27T03-36-05/cache`
- Materialized view: `settle/2026-09-27T03-36-05/matview`
- Index and rewrite: `settle/2026-09-27T03-36-05/index`

_Generated by Settle on 2026-09-27._