8e30082
3 of 3 options built by IBM Bob in parallel, each in its own branch, then measured with the same 15 second load test at concurrency 8.
No option meets every constraint.
Closest is Index and rewrite, which misses p95 latency. Relax a constraint or add an option.
Decision map
Each dot is one option Bob built. Drag the two lines to set your limits. Anything in the shaded corner is good enough, and the smallest change in there wins.
| Measured | Today (no change) | Cache | Materialized view | Index and rewrite |
|---|---|---|---|---|
| p95 latencyneeds ≤ 50 ms, median of 3 runs | 698 ms (658 ms–858 ms) | 2 ms (2 ms–2 ms)✓ | 16 ms (15 ms–17 ms)✓ | 189 ms (179 ms–193 ms)✕ |
| Requestsfailed requests disqualify | 587 · 0 failed | 274,267 · 0 failed✓ | 32,342 · 0 failed✓ | 2,234 · 0 failed✓ |
| Median latency | 662 ms | 1 ms | 10 ms | 159 ms |
| Throughput | 12.7 req/s | 6025.5 req/s | 718 req/s | 49.7 req/s |
| Stalenessneeds ≤ 15 s | instant | 60.2 s✕ | 5 s✓ | instant✓ |
| Code changedapplication code, tests excluded | +0 −0 | +42 −2 | +34 −12 | +29 −10 |
| Tests added | +0 lines | +105 lines | +36 lines | +42 lines |
| Files touched | 0 | 2 | 3 | 2 |
| New dependencies | none | none | none | none |
| Tests | pass · 3 | pass · 10✓ | pass · 4✓ | pass · 4✓ |
| Existing testsmust be left untouched | — | untouched✓ | changed 1✕ | untouched✓ |
| Built by Bob in | — | 2m 14s | 2m 26s | 2m 21s |
✓ meets the constraint · ✕ misses it · first column is today's code, for reference
8e3008204c32, every option branched from itCache the endpoint's response in memory with a 60 second time to live.
demo/orders-api/src/cache.tsdemo/orders-api/src/server.tsdemo/orders-api/test/cache.test.ts testsrc/cache.ts⋯+/** Minimal in-process TTL cache used by the top-customers endpoint. */++interface Entry<T> {+ value: T;+ expiresAt: number; // epoch ms+}++export class TtlCache<K, V> {+ private readonly store = new Map<K, Entry<V>>();++ constructor(private readonly ttlMs: number) {}++ get(key: K): V | undefined {+ const entry = this.store.get(key);+ if (!entry) return undefined;+ if (Date.now() > entry.expiresAt) {+ this.store.delete(key);+ return undefined;+ }+ return entry.value;+ }++ set(key: K, value: V): void {+ this.store.set(key, { value, expiresAt: Date.now() + this.ttlMs });+ }++ /** Remove all entries — useful in tests to avoid stale state. */+ clear(): void {+ this.store.clear();+ }+}++export const TOP_CUSTOMERS_TTL_MS = 60_000;src/server.ts⋯ import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; import type { PGlite } from "@electric-sql/pglite";+import { TtlCache, TOP_CUSTOMERS_TTL_MS } from "./cache.ts"; export interface TopCustomer {⋯ } -export function buildServer(db: PGlite): Server {+export function buildServer(db: PGlite, cache = new TtlCache<number, TopCustomer[]>(TOP_CUSTOMERS_TTL_MS)): Server { return createServer(async (req, res) => { try {⋯ if (req.method === "GET" && url.pathname === "/stats/top-customers") { const limit = Math.min(Math.max(Number(url.searchParams.get("limit") ?? 10), 1), 100);- return send(res, 200, await topCustomers(db, limit));+ const cached = cache.get(limit);+ if (cached) {+ return send(res, 200, cached);+ }+ const result = await topCustomers(db, limit);+ cache.set(limit, result);+ return send(res, 200, result); }
Precompute the ranking in a Postgres materialized view and refresh it in the background every 5 seconds.
demo/orders-api/src/db.tsdemo/orders-api/src/index.tsdemo/orders-api/src/server.tsdemo/orders-api/test/api.test.ts testsrc/db.ts⋯ ANALYZE;++ CREATE MATERIALIZED VIEW top_customers_mv AS+ SELECT c.id AS customer_id,+ c.name,+ sum(o.amount_cents)::int AS revenue_cents,+ count(*)::int AS orders+ FROM orders o+ JOIN customers c ON c.id = o.customer_id+ WHERE o.status = 'paid'+ AND o.created_at > now() - interval '30 days'+ GROUP BY c.id, c.name+ ORDER BY revenue_cents DESC;++ CREATE UNIQUE INDEX top_customers_mv_pk ON top_customers_mv (customer_id); `); return db; }++// Refreshes the materialized view and returns a handle that can stop the loop.+export function startRefresh(db: PGlite, intervalMs = 5_000): { stop: () => void } {+ const id = setInterval(() => {+ // CONCURRENTLY requires a unique index and lets reads continue during refresh.+ db.exec("REFRESH MATERIALIZED VIEW CONCURRENTLY top_customers_mv;").catch(() => {+ // Ignore transient errors (e.g. db closing during tests).+ });+ }, intervalMs);+ // Don't block Node.js exit if the caller forgets to stop.+ if (typeof id === "object" && "unref" in id) (id as NodeJS.Timeout).unref();+ return { stop: () => clearInterval(id) };+}src/index.ts⋯-import { openDb } from "./db.ts";+import { openDb, startRefresh } from "./db.ts"; import { buildServer } from "./server.ts"; ⋯ const started = Date.now(); const db = await openDb();+startRefresh(db); buildServer(db).listen(port, () => { console.log(`orders-api listening on ${port} (seeded in ${Date.now() - started} ms)`);src/server.ts⋯ } -// Top customers by paid revenue over the last 30 days. This is the slow-// endpoint the Settle demo debates how to speed up.+// Top customers by paid revenue over the last 30 days.+// Reads from the precomputed materialized view; the background refresher keeps+// it at most ~5 s stale so results are always close to real-time. export async function topCustomers(db: PGlite, limit: number): Promise<TopCustomer[]> { const { rows } = await db.query<TopCustomer>(- `SELECT c.id AS customer_id,- c.name,- sum(o.amount_cents)::int AS revenue_cents,- count(*)::int AS orders- FROM orders o- JOIN customers c ON c.id = o.customer_id- WHERE o.status = 'paid'- AND o.created_at > now() - interval '30 days'- GROUP BY c.id, c.name+ `SELECT customer_id, name, revenue_cents, orders+ FROM top_customers_mv ORDER BY revenue_cents DESC LIMIT $1`,
Add the right index or indexes and rewrite the query so Postgres does less work. No caching and no precomputed tables.
demo/orders-api/src/db.tsdemo/orders-api/src/server.tsdemo/orders-api/test/index-freshness.test.ts testsrc/db.ts⋯ ); + -- Covering index for the top-customers query:+ -- equality on status, range on created_at (newest first),+ -- INCLUDE avoids heap fetches for customer_id and amount_cents+ -- so the aggregate can be satisfied by an index-only scan.+ CREATE INDEX orders_status_created_covering+ ON orders (status, created_at DESC)+ INCLUDE (customer_id, amount_cents);+ SELECT setseed(0.42); src/server.ts⋯ export async function topCustomers(db: PGlite, limit: number): Promise<TopCustomer[]> { const { rows } = await db.query<TopCustomer>(- `SELECT c.id AS customer_id,+ // Aggregate on orders first (index-only scan on orders_status_created_covering),+ // then join the small aggregated set to customers for the name.+ // This lets Postgres resolve the GROUP BY entirely from the index and only+ // touch the customers table for at most LIMIT rows.+ `WITH ranked AS (+ SELECT o.customer_id,+ sum(o.amount_cents)::int AS revenue_cents,+ count(*)::int AS orders+ FROM orders o+ WHERE o.status = 'paid'+ AND o.created_at > now() - interval '30 days'+ GROUP BY o.customer_id+ ORDER BY revenue_cents DESC+ LIMIT $1+ )+ SELECT r.customer_id, c.name,- sum(o.amount_cents)::int AS revenue_cents,- count(*)::int AS orders- FROM orders o- JOIN customers c ON c.id = o.customer_id- WHERE o.status = 'paid'- AND o.created_at > now() - interval '30 days'- GROUP BY c.id, c.name- ORDER BY revenue_cents DESC- LIMIT $1`,+ r.revenue_cents,+ r.orders+ FROM ranked r+ JOIN customers c ON c.id = r.customer_id+ ORDER BY r.revenue_cents DESC`, [limit], );
## Appendix: measured comparison **Question:** How do we make the top customers endpoint fast? Each option was built by IBM Bob in its own branch from `8e30082`, then measured with the same load test (15 s at concurrency 8) and the same staleness probe. | | Today (no change) | Cache | Materialized view | Index and rewrite | |---|---|---|---|---| | p95 latency | 698 ms (658 ms–858 ms) | 2 ms (2 ms–2 ms) | 16 ms (15 ms–17 ms) | 189 ms (179 ms–193 ms) | | Requests | 587 · 0 failed | 274,267 · 0 failed | 32,342 · 0 failed | 2,234 · 0 failed | | Median latency | 662 ms | 1 ms | 10 ms | 159 ms | | Throughput | 12.7 req/s | 6025.5 req/s | 718 req/s | 49.7 req/s | | Staleness | instant | 60.2 s | 5 s | instant | | Code changed | +0 −0 | +42 −2 | +34 −12 | +29 −10 | | Tests added | +0 lines | +105 lines | +36 lines | +42 lines | | Files touched | 0 | 2 | 3 | 2 | | New dependencies | none | none | none | none | | Tests | pass · 3 | pass · 10 | pass · 4 | pass · 4 | | Existing tests | — | untouched | changed 1 | untouched | | Built by Bob in | — | 2m 14s | 2m 26s | 2m 21s | **Constraints:** `max_p95_ms: 50`, `max_staleness_seconds: 15`, `tests_must_pass: true` **Verdict:** No option meets every constraint. Closest is Index and rewrite, which misses p95 latency. Relax a constraint or add an option. **Branches:** - Cache: `settle/2026-09-27T03-36-05/cache` - Materialized view: `settle/2026-09-27T03-36-05/matview` - Index and rewrite: `settle/2026-09-27T03-36-05/index` _Generated by Settle on 2026-09-27._