Authority-Aware Recall System: Tier-Pinned Pages and Front-Matter Tags in ai-memory

The authority-aware recall system combines FTS5 full-text search with metadata-driven ranking, using tier levels, pinned status, and front-matter tags to prioritize high-authority content while applying decay to low-priority pages.

The ai-memory repository implements a sophisticated retrieval layer that extends standard full-text search with authority signals. By integrating tier declarations, pinned exemptions, and configurable tag weights directly into the search pipeline, the system ensures that critical documentation remains discoverable while less relevant content gradually demotes based on explicit metadata and access patterns.

How the Authority-Aware Recall System Works

At its core, the system intercepts standard FTS5 query results and re-ranks them using a composite authority score. This process involves three distinct metadata signals that work together to determine final result ordering.

The Authority-Aware Candidate Window

When a query hits the /api/v1/search endpoint, the engine first constructs a candidate window of recent matches. According to CHANGELOG.md at line 1332, this window serves as the initial filter before authority scoring applies. The system preserves high-authority pages within this window while allowing lower-priority matches to fall outside the retrieval threshold if they lack supporting metadata signals.

Tier and Pinned Front-Matter Metadata

Pages declare their authority level through YAML front-matter fields parsed in crates/ai-memory-wiki/src/wiki.rs. As documented in docs/ARCHITECTURE.md at line 286, two specific fields control ranking:

• tier: Accepts hierarchical values like bootstrap, core, or user, establishing base authority levels
• pinned: A boolean flag that, when set to true, exempts the page from decay algorithms and applies a significant ranking boost

Pinned pages maintain permanent visibility in search results regardless of temporal access patterns, while tier values provide the foundation for the composite scoring algorithm.

Tag-Based Relevance Signals

Front-matter tags function as adjustable relevance weights during the Retrieval-Relevance-Fusion (RRF) step. Referenced at line 115 in docs/ARCHITECTURE.md, these tags operate as positive or negative signals:

• Positive tags (e.g., ["security"], ["api"]) add weight to the document score
• Negative tags or tag removal reduce authority weight
• Arbitrary tag arrays allow domain-specific relevance tuning without modifying source code

The Three-Stage Search Pipeline

The authority-aware recall system processes queries through a strict pipeline implemented in crates/ai-memory-store/src/search.rs:

1. Collection: The engine gathers raw FTS5 matches, either globally or project-scoped
2. Window Application: The authority-aware candidate window filters matches to a manageable size while preserving high-authority entries
3. Re-ranking: The system calculates composite scores using:
   • Tier multiplier: Higher tiers (core > user) receive elevated base scores
   • Pinned constant: Boolean pinned: true adds a large fixed boost
   • Tag weights: Each front-matter tag contributes positive or negative adjustments

Implementing Authority-Aware Search in Rust

The following examples demonstrate how to interact with the authority-aware recall system using the ai-memory Rust API.

Performing a Project-Scoped Authority Search

use ai_memory_store::api::search::{SearchRequest, SearchResponse};
use ai_memory_store::client::MemoryClient;

let client = MemoryClient::new("http://localhost:49374")?;
let req = SearchRequest {
    query: "authentication token".to_string(),
    project_id: Some("my_project".into()),
    limit: Some(20),
    // Authority-aware ranking enabled by default
    ..Default::default()
};

let resp: SearchResponse = client.search(req).await?;
for hit in resp.hits {
    println!("▶ {} (score={})", hit.path, hit.score);
}

Creating Tier-Pinned Content

use ai_memory_wiki::{Wiki, Page};

let wiki = Wiki::new("/home/user/ai-memory/wiki")?;
let md = Page {
    frontmatter: serde_json::json!({
        "title": "Core Authentication API",
        "tier": "core",
        "pinned": true,
        "tags": ["security", "api"]
    }),
    body: "## Overview\nThe API uses ...".into(),

};

wiki.write_page("auth/api.md", md).await?;

Modifying Authority Tags Programmatically

let mut fm = wiki.read_page("auth/api.md")?.frontmatter;
let tags: Vec<String> = fm["tags"]
    .as_array()
    .unwrap()
    .iter()
    .map(|v| v.as_str().unwrap().to_string())
    .collect();

let mut new_tags = tags;
new_tags.retain(|t| t != "security"); // Remove positive authority signal
fm["tags"] = serde_json::json!(new_tags);

wiki.update_frontmatter("auth/api.md", fm).await?;

Key Source Files and Implementation Details

Understanding the authority-aware recall system requires examining specific source locations:

• docs/ARCHITECTURE.md (lines 115, 286): Defines the RRF scoring algorithm and tier/pinned metadata specifications
• CHANGELOG.md (line 1332): Documents the candidate window implementation for authority-aware FTS
• crates/ai-memory-store/src/search.rs: Implements the search request handler and composite scoring logic
• crates/ai-memory-wiki/src/wiki.rs: Handles front-matter parsing, normalization, and canonicalization of tier and pinned fields

Summary

• The authority-aware recall system extends FTS5 search with metadata-driven ranking to ensure high-priority content surfaces first
• Tier levels (bootstrap, core, user) establish base authority hierarchies, while pinned pages receive decay exemptions and ranking boosts
• Front-matter tags act as adjustable positive or negative signals during the Retrieval-Relevance-Fusion scoring phase
• The implementation spans docs/ARCHITECTURE.md for specifications and crates/ai-memory-store/src/search.rs for the actual ranking pipeline
• All authority signals are declared via YAML front-matter parsed by crates/ai-memory-wiki/src/wiki.rs and processed during the candidate window phase

Frequently Asked Questions

How does the pinned status affect search ranking compared to tier levels?

Pinned status applies a large constant boost to the document score and exempts the page from temporal decay algorithms, whereas tier levels provide multiplicative base scores. A pinned: true page with tier: "user" will outrank an unpinned tier: "core" page in most scenarios, though both signals contribute to the final composite score calculated during the RRF step.

Can tags negatively impact a page's authority score?

Yes. The Retrieval-Relevance-Fusion implementation treats front-matter tags as bidirectional signals. Removing positive tags (like security) or adding explicit negative markers reduces the document's authority weight during re-ranking, allowing the system to demote outdated or deprecated content without deleting the underlying files.

What is the default behavior for pages without explicit tier or pinned metadata?

Pages lacking explicit front-matter declarations receive default neutral values. According to the parsing logic in crates/ai-memory-wiki/src/wiki.rs, unspecified tiers default to standard user-level authority, and pinned defaults to false. These pages rely entirely on FTS5 match quality and temporal decay patterns without metadata boosts.

Where is the authority-aware candidate window size configured?

The candidate window parameters are defined in the search implementation within crates/ai-memory-store/src/search.rs, with architectural documentation referencing the window logic at line 1332 of CHANGELOG.md. This window limits initial FTS results before authority scoring applies, ensuring the re-ranking stage processes only the most relevant candidates while preserving high-authority outliers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}