Lessons already paid for
Every entry below cost real time — a broken prod deploy, a debugging afternoon, or a silent bug that shipped anyway. They're baked into the starter kit as tests, gates, or documented fixes, so you don't pay for them again.
Aurora DSQL isn't Postgres — it just looks like it
DSQL speaks the Postgres wire protocol, which is exactly what makes it dangerous: everything looks familiar until it isn't.
No SET ROLE, no standard GRANT, no table-ownership changes
DSQL uses its own IAM-based grant syntax and won't let you reassign table ownership — you have to drop and recreate. Standard Postgres admin muscle memory actively works against you here.
ALTER TABLE ADD COLUMN silently ignores DEFAULT and constraints
Add the column bare, then backfill with UPDATE, then remember every read needs COALESCE(column, default) — because NULL + 1 is NULL, not the default you thought you set.
Multi-statement DDL needs a single psql -f pass, not sequential connections
Run dependent CREATE TABLE statements as separate calls and DSQL can lose the dependency ordering.
New tables don't inherit Lambda role permissions automatically
Every CREATE TABLE needs an explicit GRANT afterward, or the app 500s on first query with a permissions error that looks like a code bug.
A bootstrap run doesn't register itself in the migrations table
Run a one-off --setup script once, then run your normal migration flow, and the runner tries to re-apply the bootstrap script — which isn't idempotent — and fails. You have to backfill the migrations row by hand before continuing.
Security middleware that blocks your own users
A SQLi filter that scans request bodies for characters like ;, |, and -- is good instinct — and it will reject a completely legitimate document upload.
Clean test fixtures never contain attack-adjacent punctuation
A résumé bullet, a recipe ("8–10 min; salt & pepper"), or an itinerary with an em dash and a price range can trip a filter that never gets exercised by normal test data. It ships clean, passes every test, and then a real user's first upload gets a 400 with no useful explanation.
The fix is scope, not weakening the filter everywhere
Free-form content routes (uploads, notes, prose) get punctuation-tolerant scanning for XSS/path-traversal only; everything else keeps the full pattern set — plus a dedicated unit test asserting real attack payloads still get blocked and realistic prose always passes.
What unit tests structurally cannot catch
Unit tests hit your app directly, in-process. They never touch API Gateway, a cold Lambda start, a real IAM-authenticated DB connection, or a real browser. A green unit suite tells you the logic is right — nothing about whether the system works.
A "smoke test" that silently skipped every authenticated check
The test runner registered a fresh user, but login was gated behind both email verification and admin approval — neither of which the throwaway user had. Every authenticated assertion silently no-op'd, and a broken endpoint deployed green. Fix: a self-verifying test user, plus a rule — a test suite that can silently skip is not a gate, it's a false green.
An admin dashboard that "worked" in every test but showed nothing in the browser
Three independent things all had to be true — the route registered, the user actually flagged admin in the live database, and the frontend rendering a visible entry point — and each failed silently on its own. Unit tests checked the route in isolation and missed the other two.
Why there's a Playwright layer, not just unit tests
The pattern in every seam bug: it lives between components, not inside any one of them. End-to-end coverage drives a real browser against the real deployed stack and catches the class of bug that "all tests green" otherwise hides — shipped alongside unit tests, not instead of them.
AI grounding is content-shape-dependent, not just model-dependent
Specific to building on Bedrock (or any RAG pipeline): the "how confident does a retrieved chunk need to be" threshold isn't a single number — it depends on what kind of text you embedded.
A threshold tuned on prose breaks on terse content
Tuning against prose gives clean separation between grounded and unrelated questions. Deploy the same threshold against terse content like résumé bullets or spec sheets, and grounded questions can score below the floor — a false negative that looks like a data problem but is actually a threshold problem.
The permanent fix
Never tune a grounding threshold against synthetic test embeddings — measure real separation against both prose and terse content, set the floor in the combined gap, and gate every deploy behind a live test that runs against the real model and fails loudly (not silently skips) if it can't verify actual grounded recall.
Model and service availability isn't static
Two traps that look like your code broke when actually the platform moved.
Foundation model IDs get retired
An invoke that worked last month can start throwing "resource not found" not because of a permissions regression, but because the model ID itself was deprecated. The fix is a one-line check against the current model list before assuming an IAM problem.
Account-level usage agreements can be CLI-only
Some providers require an account-level usage-agreement step that has nothing to do with your IAM policy, and the console page for it can be deprecated in favor of a CLI-only flow with its own sharp edges — using the wrong token format throws a validation error that gives no hint what's actually wrong.
None of this is marketing copy about "best practices." Every entry above is a documented, dated production incident with a root cause and a permanent fix — a failing test, a scoped guard, or a rule that's now enforced automatically. A boilerplate gives you a starting structure. This gives you the starting structure plus the debugging you'd otherwise do yourself, the first time, in production, on your own clock.