Story Engine — Graphene novel-mode seasons

The Story Engine is the system that generates serialized audio fiction inside HiveJournal. Two modes:

• Journal mode — original flow. AI personas write daily journal entries; a narrator weaves them into a podcast. Cast-driven.
• Novel mode — narrator-only long-form prose. No personas, no journals. Each season is a serialized novel with chapters; each chapter renders to its own MP3 played back as a chapter playlist.

This document covers novel mode end-to-end. Journal mode is documented inline in INDEX.md.

If you're reading this to understand the architecture before extending it, the related plan in docs/ai/plans/story-bible-and-chapter-takes.md is the design doc that drove the implementation.

The flow at a glance

1. Create season (novel mode)
     ↓ pick a universe (optional) — inherits canonical lore
     ↓ chapter plan auto-generated
     ↓ Story Bible auto-bootstrapped from plan + universe canon
2. Generate next chapter
     ↓ auto-refine pipeline (or one-shot generate)
        ├─ take 1 (bible-aware GPT call)
        ├─ run default critics
        ├─ apply findings, queue bible deltas
        ├─ refined take 2 with critic notes
        ├─ promote take 2 → live
        └─ render chapter audio (ElevenLabs, with TTS cache reuse)
3. Repeat for each chapter (or bulk-refine all + queue audio for all)
4. Listener visits /seasons/[id]
     ↓ chapter playlist player auto-resumes from last position
     ↓ end-of-novel rating prompt (👍/👎 + stars)

Costs are roughly $0.04 GPT + $0.60 ElevenLabs for a fresh 6-chapter novel; running critics and a refined take adds ~$0.005/chapter.

The Story Bible

season_story_bibles (migration 116) is the canon document the chapter generator reads BEFORE every chapter. Two layers of canon:

Structured canon (mechanically enforced)

Stored as JSONB columns. Every chapter prompt sees these in a token-budgeted block (~1500 tokens default).

Per-character verbal_tics + speech_avoid + mbti + age are dialogue levers — the chapter prompt renders these inline so each character sounds distinct. backstory and relationships are shown only on the chapter where a character first appears (orientation budget, not bloat).

Free-form lore_notes (texture, not enforcement)

season_story_bibles.lore_notes (migration 124) is markdown. Paste from NotebookLM exports, hand-written world docs, draft scenes, the city's geography, the agency's history — anything that doesn't fit the structured shape. Token-budgeted (~800 tokens combined with universe lore at chapter time, ~1200 at bootstrap).

The split exists because:

• Structured canon is what the model is told to honor across every chapter. Surgical, slow to author, mechanically enforced via bibledelta + apply-canon-on-bootstrap.
• lore_notes is what the model can lean on for texture. Loose, fast to author, helpful but not required to cite.

Versioning

Every commit snapshots into season_story_bible_versions. The Current view is always the latest; History tab shows the full timeline with commit messages.

Operations

Universes (shared lore)

StoryUniverse (in story-universes.ts) is a typed registry of shared canon worlds. Multiple novels can live in one universe and inherit:

• Canonical characters / themes / settings / motifs / threads / rules (merged into the bible at bootstrap, deduped by key)
• A prompt_directive injected into BOTH chapter-plan generation AND bible bootstrap as MANDATORY CANON
• Free-form lore_notes markdown that flows into chapter prompts alongside per-novel notes
• A suggested default_framework_key

story_seasons.universe text (migration 123) tags a season with its universe key. Schema is mode-agnostic — journal-mode seasons could opt in too, but UI wiring is novel-only in v1.

The Turing Logs

First (and currently only) universe. Tech-dystopian, in the mode of Black Mirror. Every novel is "a recovered record from the Department of Cognitive Affairs."

Canonical:

• Dr. Evelyn Harper (Senior Behavioral Architect, INTJ, 54) — main recurring character; secret about her daughter
• Director Marcus Vance (her superior, openly cruel)
• Auditor Theodore Pell (internal affairs, was Subject 41-B)
• Department of Cognitive Affairs HQ, Reflection Rooms, the unnamed Mid-Atlantic city
• Motifs: jar of baby teeth, "a recommendation has been made," cold tea, "graduates"
• Voice rules (no contractions for state actors, passive constructions), do-not-use (gore, comic relief, neat catharsis, "dystopia")

Universe-level lore_notes carries ~3.5k chars covering agency history, city geography, what citizens know vs. don't, the founding incident, and untouchable canon.

Adding a new universe

1. Add a StoryUniverse const in story-universes.ts.
2. Register it in STORY_UNIVERSES.
3. It auto-appears in the create-season picker via GET /universes.

Frameworks

StoryFramework (in story-frameworks.ts) is a data-driven recipe: prompt directive + intensity curve + structural hints + beats. Frameworks are picked at the chapter take level, not at season creation. Default is three_act.

tech_dystopian is the natural pair for the Turing Logs universe (set as its default_framework_key).

Chapter Takes

story_episode_takes (migration 118) — every prose generation lands here as status='draft'. Promotion is the only path that mutates story_episodes.chapter_prose. One row per attempt, snapshotted with the prompt actually sent to GPT (prompt_snapshot) and the bible version it was generated against (bible_version_id).

Statuses: draft → live | archived | rejected. Partial unique index ensures one live take per episode.

Iterate-with-notes loop

episode_take_critiques + episode_critique_decisions (migration 119) mirror the season-level critique loop at the take level:

1. Run one or more critics against a draft take (default: tension_editor, dialogue_doctor, repetition_hawk)
2. Each finding gets a per-paragraph card with apply / ignore / override-with-text buttons
3. "Iterate with N notes" generates a fresh take with the applied notes folded into the prompt as CRITIC NOTES ADDRESSING THE PREVIOUS TAKE
4. Consumed decisions get marked applied_to_take_id so they don't double-fire on the next iteration

The full audit trail per chapter is visible in the 📜 Critique history collapsible panel inside ChapterTakesModal.

Auto-refine pipeline

Two flavors:

Single chapter — POST /:id/auto-refine-next-chapter. Triggers next untriggered chapter, generates take 1, runs default critics, applies findings, generates refined take 2 with notes, promotes, renders audio. ~60-90s background. Surfaces as ✨ Auto-refine next in admin and ✨ Create Next Episode on /seasons/[id] for super-admins.

Bulk — POST /:id/auto-refine-all-chapters. Loops over every untriggered chapter, runs the full refine flow, skips audio. ~30-60s per chapter; ~3-5 minutes for a 6-chapter novel. Audio is queued separately via POST /:id/queue-all-chapter-audio so admins can review prose before paying for TTS.

Failed chapters in bulk get auto-untriggered to prevent loop-on-broken- chapter; the loop continues to the next.

Audio rendering

Per-chapter as primary output

renderSeasonAudio (in season-audio.ts) detects novel mode and skips the master compiled.mp3 step entirely. Each chapter gets its own MP3 stamped on story_episodes.chapter_audio_url (migration 115). Adding a new chapter only renders that chapter; older chapters' MP3s are reused as-is.

Segment-level TTS cache reuse

season_audio_segments.audio_url is set once on a successful render. renderChapterAudio and renderSeasonAudio skip ElevenLabs when a segment's audio_url is already set (downloads cached MP3 from storage instead). rebuildSegmentsFromCache preserves the cache across prose changes via tuple match (episode_number, segment_type, voice_id, voice_direction, text_content). Force-re-render is available via { force: true } in the body or ⇧Shift-click in the UI.

Chapter playlist player

ChapterPlaylistPlayer (on /seasons/[id]) plays per-chapter MP3s sequentially with auto-advance. wantsToPlayRef guards the pause-before-ended state-flip cycle so auto-advance actually plays. Click-to-jump on chapter rows; cumulative elapsed time across the whole novel.

Listener tracking

Session tracking + version-aware listens

chapter_listen_sessions (migration 121) — one row per (identity, episode, take) combo. Identity is user_id (auth) or session_id (localStorage UUID for anonymous). Captures both how many listens and which version they listened to by snapshotting take_id at listen-start. Future listens after a take change record under the new take so admins can A/B compare engagement across prose iterations.

Player fires throttled events: start on play (per chapter mount), progress every 10s while playing, final progress on pause, complete on chapter end. Fire-and-forget with keepalive: true so the last event survives page-unload.

Resume-where-you-left-off

last_progress_seconds (migration 122) tracks current position (replaced each event), distinct from max_progress_seconds (ratcheted; deepest ever). GET /:id/resume-cursor returns the listener's most-recently-touched session so the player can seek to that position on mount. Cross-device for authenticated users; per- browser for anonymous (by design — anonymous can't sync).

A take_changed flag in the cursor surfaces a "story has been updated since you last listened" hint when the chapter has been promoted to a new take since the listener's last visit.

End-of-novel ratings

season_listener_ratings (migration 120) — one row per identity per season. Two-phase modal at end of playlist: 👍/👎, then if 👍 a 1-5 star picker + "what worked?" textarea; if 👎 just a "what didn't work?" textarea (no stars — saves the listener effort). Re-rating upserts.

Public aggregate badge below the playlist: 👍 87% · ★ 4.3 (24) — rendered only when count ≥ 5 to avoid small-sample noise. Admin view in SuperAdminPanel shows full free-text feedback for moderation.

Reference: file map

Backend services

• story-bible.ts — bible CRUD, bootstrap, backfill, prompt block builder, applyDelta
• story-bible-deltas.ts — delta queue + approval
• story-frameworks.ts — framework registry + prompt directive renderer
• season-novel-script.ts — full-season novel script generator
• season-novel-chapter.ts — per-chapter generation, generateChapterRaw
• season-novel-chapter-audio.ts — per-chapter audio render
• season-novel-chapter-auto.ts — auto-refine orchestrators (single + bulk + audio queue)
• episode-takes.ts — chapter takes service
• episode-take-critiques.ts — chapter critique loop
• chapter-listen-tracking.ts — listen sessions + resume cursor + stats aggregator
• season-listener-ratings.ts — end-of-novel ratings

Backend constants / registries

• story-universes.ts — universe registry (Turing Logs)
• script-critics.ts — critic registry

Frontend

• components/seasons/StoryBibleModal.tsx
• components/seasons/ChapterTakesModal.tsx
• components/seasons/EpisodesAdminModal.tsx
• components/seasons/SuperAdminPanel.tsx
• components/seasons/ChapterPlaylistPlayer.tsx
• components/seasons/ListenerRatingModal.tsx
• components/seasons/ListenerRatingsAdminModal.tsx
• components/seasons/ListenStatsAdminModal.tsx
• app/seasons/[id]/page.tsx
• app/dashboard/admin/seasons/page.tsx

Migrations (story-engine relevant)

Conventions

• New universes: code-defined (under version control); migrate to story_universes table only when > 5 universes exist or admins need to edit them at runtime.
• New frameworks: same — add to story-frameworks.ts, restart backend.
• Lore for an existing universe: edit the universe const in story-universes.ts. Per-novel texture goes in the bible's lore_notes field via the UI.
• Admin-only RLS for everything bible/take/critique/rating-related. Public reads only on aggregate ratings (above the noise threshold) and on live takes for visible seasons.
• Audio costs are dominated by ElevenLabs; segment cache reuse + per- chapter rendering are the two big cost levers. Prefer "bulk-refine prose, then queue audio when satisfied" over "auto-refine each chapter to completion."