Workshop 5: Live demo script

Share-screen guide. This is the most demo-heavy session. We use AI well together: I drive on screen, students follow along in their own clones in small chunks. Total demo time: ~40 minutes across blocks 3–5. We record results in the W5 logbook section together as we go. Anything beyond the core tasks is optional polish.

Setup before the demo

• AI tool of choice open in the side panel (Claude / ChatGPT / Cursor).
• Editor + terminal split.
• Prompt templates open: workshops/05-ai/starter/refactor-prompt.md and test-gen-prompt.md.
• workshops/05-ai/starter/active-comprehension-prompt.md open, and the comprehension-preserving prompt already set as your system prompt for the demo session so students see it shaping the answers.
• Repo state: clean, all tests passing (or the flaky ones passing on this run).

---

Step 0: Set the comprehension-preserving system prompt (~2 min)

1. Open starter/active-comprehension-prompt.md. Read the prompt out loud.

   "Before any task, I set this as my system prompt. It tells the AI to explain its reasoning, state assumptions and tradeoffs, offer an alternative, ask me clarifying questions, and quiz me at the end. The point is to stop the AI from thinking for me."

2. Name the risk plainly:

   "The danger with AI isn't that it's wrong, we catch wrong with tests. It's that it's right and you accept code you never understood. That's cognitive offloading. You can't debug or extend what you don't understand. This prompt, plus proving comprehension after each task, is how we avoid it."

3. Leave it on for every step below. Call out, live, when the AI asks you a question or surfaces an assumption, that is the prompt working.

---

Step 1: Refactor the god function in reports.ts (~10 min)

1. Open apps/api/src/routes/reports.ts. Scroll. Show its length.

   "~130 lines. One function. We're going to ask AI to split it apart, and then verify it didn't break anything."

2. Open starter/refactor-prompt.md. Read it out loud, slowly.

   "Notice what we're doing here. We're giving the AI the file, the conventions, AND a constraint: don't change behavior. That last constraint is what makes the output safe to apply."

3. Paste the prompt template into your AI tool. Paste the full contents of reports.ts. Run it.

4. While it generates, talk through what we're hoping for:

   • Smaller functions for each concern (CSV building, date formatting, timezone handling)
   • Behavior preserved (tests still pass)

5. Apply the output. Run the existing tests:

   cd apps/api && npx jest --testPathPattern=reports

6. If tests pass: high-five the chat.

7. If tests fail: paste the failure back to the AI: "This test failed after your refactor: [paste]. Fix the refactor." Iterate.

8. Key teaching moment: show students that the AI's first answer often isn't right. The skill is iterating on the prompt.

9. Prove comprehension, live. Close the AI panel. From memory, name each helper the AI extracted and what it produces. Then change one helper by hand (rename it, tweak a behavior) with no AI and re-run the tests.

   "If I can't name what it did and change it myself, I didn't understand it, I just accepted it. That's the move I want you copying: explain it back, or modify it without AI."

---

Step 2: Generate tests for auth.service.ts (~7 min)

1. Open apps/api/src/services/auth.service.ts. Show that there are zero tests for it.

2. Open starter/test-gen-prompt.md. Walk through what the prompt does.

3. Paste into AI. Paste the source file. Paste apps/api/tests/projects.test.ts as the style guide.

4. Apply the output to apps/api/tests/auth.service.test.ts.

5. Run the tests:

   cd apps/api && npx jest --testPathPattern=auth.service

6. Expect failures. AI typically gets one or two things wrong:

   • Imports something that doesn't exist (@orbittasks/api instead of relative paths).
   • Asserts on JWT payload structure directly when it should use verifyToken.
   • Misses an edge case.

7. Walk through fixing one failure live. Push the rest back to the AI with the failure messages.

8. Key teaching moment: the AI is a junior pair programmer. Treat its output like a PR you have to merge.

9. Prove comprehension, live. Pick one passing test, change its assertion to something you know is wrong, and predict the failure out loud before running it. Run it. If the failure matches your prediction, you understood the test. If it surprises you, you didn't, and that's the gap blind acceptance would have shipped.

---

Step 3: Mock the SDK clients to collapse the slow API tests (~7 min)

This is the highest-value win of the session. Spend the time here.

1. Run the full API suite and time it:

   cd apps/api && npm run test:api

   It takes ~12 minutes. Show students why: the integration tests make real HTTP through the SDK clients.

2. Point at the cost center: these tests, plus reports.test.ts, all drive real client calls:

   • tests/integration/billing.rollup.test.ts
   • tests/integration/email.campaigns.test.ts
   • tests/integration/webhooks.fanout.test.ts
   • tests/integration/search.reindex.test.ts
   • tests/integration/notifications.blast.test.ts
   • tests/reports.test.ts

3. Show that the services are already constructor-injectable, e.g.:

   "BillingService takes client: BillingClient = new BillingClient(). The seam is already there. We just inject a mock instead of the real client."

4. Ask the AI to inject mocked clients for the six clients (billing, email, search, webhooks, notifications, audit) so the tests stop making real HTTP:

   "These integration tests make real HTTP through the SDK clients. The services accept an injectable client in their constructor (e.g. BillingService(client: BillingClient = new BillingClient())). Generate mocked clients for billing, email, search, webhooks, notifications, and audit, and inject them in the tests so no real network calls happen. Keep the assertions the same."

5. Apply the mocks. Re-run and time it:

   cd apps/api && npm run test:api

   Expected drop: from ~12 minutes to seconds. This is the payoff Workshop 3 set up: the seam was added then, AI wires up the mocks now.

---

Step 4: Refactor the slow FE list test (~5 min)

1. Open apps/web/tests/components/TaskList.test.tsx. Scroll to the "renders a long list of tasks" test.

2. Show it rendering 500 items. Run the test, time it:

   cd apps/web && npx vitest run TaskList --no-coverage

3. Ask the AI to virtualize the TaskList component (don't change the test; change the component to be efficient with N items):

   "Here's the TaskList component. Add window-based virtualization so it can render thousands of items efficiently. Use only React built-ins, no new dependencies."

4. Apply the change to TaskList.tsx. Re-run the test. Expected drop: from ~2s to under 200ms.

---

Step 5: Fix the flaky CommentList test (~5 min)

1. Open apps/web/tests/components/CommentList.test.tsx. Run the "renders a relative timestamp" test a few times. Show that it occasionally fails depending on time of day.

2. Ask the AI to fix it. The preferred fix is injecting a fixed now, formatRelativeDays(input, now = new Date()) already takes a now param, so the test can pass a fixed date instead of relying on the real clock:

   "This test depends on the real clock via formatRelativeDays. The function already takes a now parameter (formatRelativeDays(input, now = new Date())). Pass a fixed now (or thread it through the component prop) so the assertion is deterministic."

   If injecting now isn't reachable from the test, the alternative is faking the clock:

   "Alternatively, replace the real clock with vi.useFakeTimers() so the assertion is deterministic."

3. Apply the fix. Re-run the test 10 times in a loop:

   for i in {1..10}; do npx vitest run CommentList --no-coverage --silent; done

4. Every run passes. Teaching moment: the AI sometimes gives you a real, durable fix on the first try. Sometimes it doesn't. Both happen.

---

Step 6: Fix the flaky Login test (~5 min)

1. Open apps/web/tests/pages/Login.test.tsx. Scroll to the "Signing in…" test.

2. Run it a few times. Sometimes it passes, sometimes it doesn't.

3. Ask the AI:

   "This test fires a click and synchronously asserts on the resulting state. React batches state updates so the rerender is queued. Fix the test using await waitFor or findByText so it waits for the rerender."

4. Apply the fix. Run repeatedly. All pass.

5. Capture for Workshop 7: the pattern "synchronous assertion on async state update" is something to ban via a lint rule. We'll do that in Workshop 7.

---

After the live demo

"We just used AI on six real problems in this codebase together, with a system prompt that kept us in the loop, and we proved we understood the output every time. With your own comprehension prompt set (handout Part 0), we keep going together on the next task, following along in your own clones. The handout has the rubric and the comprehension checks; we record the results in the W5 logbook section as we go. 10 minutes."

That's the handoff to the next block. We keep working together rather than splitting off into solo work. Any task past the core ones is optional polish.

---

What students will not realize without me saying it

1. The prompts are first-class code. Save them. Version them. Reuse them. That includes the comprehension-preserving system prompt.
2. The AI is wrong sometimes in ways that compile but lie. Always run the tests.
3. The bigger risk is the code that's right and you didn't understand. That's cognitive offloading, and it's invisible because nothing breaks today. The tell: could you re-derive or explain it without the AI? If not, you offloaded comprehension. Explain it back or modify it without AI to close the gap.
4. Failure modes are predictable once you've used AI for a week or two. The categories in starter/evaluation-rubric.md will become second nature.
5. The team policy you write in Workshop 7 codifies all of the above. Today is the input data.