# How ai-memory Handles Fallback Search When Compiled Wiki Pages Miss

> Discover how ai-memory uses raw observation fallback search to ensure users always get relevant data when compiled wiki pages miss.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: internals
- Published: 2026-09-12

---

**When compiled wiki pages return no results, ai-memory automatically triggers a raw-observation fallback search to ensure users always receive relevant data.**

ai-memory is an open-source memory system designed to persist and retrieve contextual information for AI applications. When queries against pre-processed documentation fail to return hits, the system implements a sophisticated fallback mechanism that searches raw observation snippets instead. This ensures that recent knowledge captured from hooks remains accessible even before it gets compiled into indexed wiki pages.

## The Fallback Search Mechanism

The fallback process operates as a seamless second layer that activates only when primary searches exhaust the compiled page index. According to the source code in [`crates/ai-memory-mcp/src/server.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/server.rs), the system distinguishes between **compiled pages** (pre-processed Markdown stored in the FTS index) and **raw observations** (recent text snippets captured from integration hooks).

The mechanism follows a strict conditional flow:

1. **Primary search execution** – The server first calls `MemoryQuery::search_pages_for_project_at` to retrieve compiled-page hits.
2. **Miss detection** – The code explicitly checks `if !hits.is_empty()` to determine whether any results exist.
3. **Fallback activation** – Only when the hit list is empty does the system proceed to raw-observation search.

## Implementation Details

### Detecting Primary Search Misses

The core logic resides in [`crates/ai-memory-mcp/src/server.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/server.rs) between lines 1010-1048. After attempting to retrieve compiled pages, the system evaluates the result set:

```rust
// Raw-observation fallback when compiled-page search misses. Works
// for a single resolved project (default / workspace+project) AND
// for explicit `scopes` — the recommended scope‑bleed mitigation —

```

If the `hits` vector contains no elements, the execution path diverges to the fallback routine rather than returning an empty response.

### Handling Scoped vs. Unscoped Queries

The fallback adapts its behavior based on whether the query includes explicit scope parameters:

- **With explicit scopes** – The system iterates over each resolved `(workspace, project)` pair and performs a separate observation search per scope.
- **Without scopes** – The request's effective workspace and project are resolved first, followed by a single-project observation search.

This dual-path approach prevents scope bleed while ensuring comprehensive coverage across multi-tenant environments.

### Executing Raw Observation Searches

When fallback activates, the reader invokes `search_observations_for_project` from [`crates/ai-memory-store/src/reader.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/reader.rs). This function queries the observations table directly, retrieving recent text snippets that haven't yet been processed into wiki pages.

The results undergo rank-based sorting and truncation to the query limit. For multi-scope queries, observation hits from each scope merge and re-sort to maintain consistent relevance ordering across boundaries.

### Packaging Results in MemoryQueryResponse

The final stage populates the `raw_hits` field of the `MemoryQueryResponse` structure. As implemented in lines 1044-1048 of the server implementation:

- Fallback hits populate `raw_hits`
- Standard result groups (`hits`, `global_hits`, etc.) remain untouched
- The response maintains the same schema regardless of which search layer provided the data

## Why the Fallback Matters

Compiled pages represent a static view of knowledge generated from Markdown sources and indexed through FTS. However, this creates a potential gap: observations captured via hooks exist in the system before compilation occurs, or may persist after compiled pages are deleted.

The **raw-observation fallback** guarantees a "no-silence" property for the system. Users querying recent activity or deleted documentation still receive relevant results from the raw observation backlog, bridging the temporal gap between data capture and wiki compilation.

## Key Source Files and Functions

| File | Role |
|------|------|
| [`crates/ai-memory-mcp/src/server.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/server.rs) | Contains the MCP query endpoint implementation, fallback detection logic (lines 1010-1048), and `raw_hits` population. |
| [`crates/ai-memory-store/src/reader.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/reader.rs) | Provides `search_observations_for_project`, the low-level routine querying the observations table for raw snippets. |
| [`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md) | Documents the overall data flow between store, writer, and wiki components, explaining the necessity of fallback mechanisms. |

## Summary

- **ai-memory** implements a two-tier search system: compiled pages first, raw observations as fallback.
- The fallback triggers only when `MemoryQuery::search_pages_for_project_at` returns an empty hit list.
- **Scoped queries** execute observation searches per `(workspace, project)` pair, while unscoped queries use a single effective project resolution.
- Raw results populate the `raw_hits` field in `MemoryQueryResponse`, preserving other result groups unchanged.
- This mechanism ensures recent knowledge remains searchable regardless of wiki compilation status.

## Frequently Asked Questions

### How does ai-memory know when to trigger the fallback search?

The system checks `if !hits.is_empty()` after executing the primary compiled-page search. Only when this condition evaluates to false—indicating zero compiled-page matches—does the code path proceed to execute `search_observations_for_project` for the raw observation fallback.

### What is the difference between compiled pages and raw observations in ai-memory?

**Compiled pages** consist of pre-processed Markdown content stored in the FTS (Full-Text Search) index, representing static documentation. **Raw observations** are dynamic text snippets captured from hooks that haven't been processed into wiki format, stored separately in the observations table until compilation occurs.

### Does the fallback search work differently for multi-scope queries?

Yes. When explicit `scopes` are provided, the fallback runs independently for each resolved `(workspace, project)` pair, then merges and re-sorts the results. For single-scope queries, it resolves the effective workspace and project once before performing a single observation search, preventing cross-project data leakage while maintaining coverage.

### Where are the fallback results stored in the query response?

Fallback hits populate the `raw_hits` field of the `MemoryQueryResponse` struct. The standard `hits` and `global_hits` fields remain reserved for compiled-page results, allowing client applications to distinguish between curated wiki content and raw observation data if needed.