How Story State Is Maintained Across Frames in Calliope: A Deep Dive into the Fern Strategy

Story state in Calliope is persisted as a JSON object in the PostgreSQL-backed state_props column of the Story table, updated by strategies like FernStrategy after each LLM generation, and injected into subsequent prompts to guarantee narrative continuity across frames.

Calliope is an open-source generative storytelling framework (chrisimmel/calliope) that constructs narratives one frame at a time. The system maintains story state across frames by treating state as a database-backed JSON object that evolves with each interaction, ensuring that characters, settings, and plot points remain consistent throughout the narrative.

Where Story State Lives in the Database

The canonical source of truth for story continuity resides in the Story table defined in calliope/tables/story.py. Here, the state_props column stores the evolving narrative context as a PostgreSQL JSONB object.

# calliope/tables/story.py

state_props = JSONB(null=True)   # ← JSON‑B column that stores the evolving story state

Because Calliope uses the Piccolo ORM, this column accepts native Python dictionaries and persists them as structured JSON. The framework also tracks which story is currently active for a given client (sparrow) via calliope/models/sparrow_state.py, ensuring API requests target the correct Story row.

Initializing Story State on the First Frame

When a narrative begins (frame_number == 0), the strategy invokes _init_story to generate the initial state. During this bootstrap phase, the LLM returns a StoryStateModel JSON object that seeds the database record:

# calliope/strategies/fern.py (inside _init_story)

story.state_props = json_response   # json_response is the initial state dict

await story.save()

This initialization establishes the baseline genre, cast, and settings that subsequent frames will reference and mutate.

Updating State After Frame Generation

After generating each frame, the Fern strategy (the most feature-complete implementation in calliope/strategies/fern.py) extracts the updated state from the LLM response and persists it to the database:

# calliope/strategies/fern.py

if story_state:
    # `story_state` is a `StoryStateModel` returned from the LLM.

    print(f"Updating story state to: {story_state}")
    story.state_props = story_state.model_dump()   # ← store it as JSON

    await story.save()                            # ← persist to DB

The update follows a strict sequence:

1. The LLM returns ExtendStoryResponseModel, which contains the story_state field.
2. model_dump() converts the Pydantic model to a plain dictionary compatible with Piccolo's JSONB column.
3. await story.save() commits the transaction, immediately making the new state visible to concurrent workers.

Feeding Stored State into LLM Prompts

To maintain continuity, the strategy reads the stored state_props and injects it into the system messages of the next LLM request:

# calliope/strategies/fern.py

story_state = json.dumps(story.state_props, indent=2) if story.state_props else ""

messages = [
    {"role": "system", "content": "Situation: {situation}"},
    {"role": "system", "content": "You are a storyteller ..."},
    {"role": "system", "content": f"<STORY_STATE>{story_state}</STORY_STATE>"},
    # … other messages …

]

By wrapping the serialized state in <STORY_STATE> tags, the framework provides the LLM with the full cumulative context (genre, character arcs, prior events) required to generate coherent continuations.

Accessing State Through the REST API

Client applications can retrieve the current state via the REST API exposed in calliope/routes/v2/stories.py. The endpoint GET /v2/stories/{id} returns the state_props field as part of the payload assembled in calliope/tasks/handlers.py:

# calliope/tasks/handlers.py (response payload)

payload = {
    "id": story.cuid,
    "title": story.title,
    "frames": [...],
    "state_props": story.state_props,   # ← included for the client

}

This allows external clients to inspect narrative context or resume stories across sessions.

Summary

• Storage Location: Story state lives in the state_props JSONB column of the Story table (calliope/tables/story.py), managed by Piccolo ORM.
• Update Mechanism: The FernStrategy updates state by calling story.state_props = story_state.model_dump() followed by await story.save() after each LLM generation.
• Continuity Guarantee: The strategy reads story.state_props back into system messages, ensuring the LLM receives complete historical context for every new frame.
• Scalability: Because state persists in PostgreSQL rather than memory, it survives process restarts and scales horizontally across worker nodes.

Frequently Asked Questions

What database column type does Calliope use for story state?

Calliope uses a PostgreSQL JSONB column named state_props defined in calliope/tables/story.py. This allows the framework to store semi-structured narrative data (characters, settings, plot points) while maintaining query performance and ACID compliance.

How does the Fern strategy handle the initial story state?

When frame_number == 0, the Fern strategy calls _init_story, which prompts the LLM to generate an initial StoryStateModel. The strategy assigns this JSON object directly to story.state_props and immediately calls await story.save() to establish the baseline state for the narrative arc.

Why is story state stored in the database rather than memory?

Storing state in the database row (via await story.save() in calliope/strategies/fern.py) ensures durability across server restarts and distributes cleanly across multiple workers. Because the Story record is the single source of truth, any subsequent frame request—regardless of which worker processes it—retrieves the most recent state from PostgreSQL.

What specific model does the LLM return to update the state?

The LLM returns an ExtendStoryResponseModel containing a story_state field of type StoryStateModel. The Fern strategy validates this Pydantic model and serializes it via model_dump() before persisting to the state_props column, ensuring type safety and schema consistency.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}