Configuring Knowledge Base Backend in AutoResearchClaw: Markdown vs Obsidian

Set knowledge_base.backend: obsidian in your .arc.yaml file to generate Obsidian-compatible Markdown with wikilinks and tags, or leave it as markdown (default) for plain Markdown output.

AutoResearchClaw stores pipeline artefacts in a knowledge base (KB) under a configurable root folder, and configuring the knowledge base backend determines whether files are emitted as standard Markdown or enhanced for Obsidian compatibility. This choice affects how your research notes integrate with your documentation workflow and PKM (Personal Knowledge Management) tools.

Understanding the Two Backend Options

AutoResearchClaw supports two distinct output formats controlled by the knowledge_base.backend setting. Both produce valid Markdown, but the Obsidian variant includes additional metadata syntax that enables advanced navigation in Obsidian.

Plain Markdown Backend (Default)

The markdown backend generates clean Markdown files with YAML front-matter containing metadata such as id, title, stage, and tags. According to the source code in [researchclaw/knowledge/base.py](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/knowledge/base.py#L90-L94), this is the default behavior when no backend is specified or when backend: markdown is explicitly set.

Obsidian-Enhanced Backend

The obsidian backend extends the plain Markdown format by appending tag lines and wikilink syntax. The private helper _obsidian_enhancements (lines L74-L83 in researchclaw/knowledge/base.py) adds:

• Inline hashtag tags (e.g., #goal_define #stage-01)
• A "Related" section with Obsidian wikilinks (e.g., [[run-2024-09]])

This enhancement occurs conditionally in write_kb_entry at lines L105-L109, only when the backend parameter equals "obsidian".

Configuration File Structure

Declare your preferred backend in the knowledge_base section of your .arc.yaml or config.yaml:

knowledge_base:
  backend: obsidian      # or "markdown"

  root: docs/kb          # path relative to project root

  obsidian_vault: ""     # reserved for future use, currently optional

The configuration schema is strictly enforced. In [researchclaw/config.py](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/config.py#L96-L99), the KB_BACKENDS constant defines the allowed values as ["markdown", "obsidian"]. The KnowledgeBaseConfig dataclass (lines L165-L170) stores these three fields, and the validate_config function (lines L82-L85) raises an error if an invalid backend string is provided.

How the Backend Selection Works Internally

When pipeline stages complete, the function write_stage_to_kb constructs a KBEntry dataclass and delegates file writing to write_kb_entry. The backend flows from configuration to execution as follows:

1. Config loading: RCConfig.load("config.arc.yaml") populates cfg.knowledge_base.backend
2. Validation: validate_config verifies the backend against KB_BACKENDS
3. Runtime dispatch: write_kb_entry(kb_root, entry, backend=backend) receives the backend string
4. Conditional formatting: If backend == "obsidian", the code calls _obsidian_enhancements before writing

This centralized approach means changing the backend in your config file automatically switches the format for every KB entry without modifying pipeline logic.

Practical Implementation Examples

Switching to Obsidian Backend

To enable Obsidian-style metadata, update your configuration and re-run your pipeline:

# config.yaml

knowledge_base:
  backend: obsidian
  root: docs/kb

The resulting files will include both standard front-matter and Obsidian-specific extensions:

---
id: goal-define-run-abc
title: Define Goal
stage: 01-goal_define
run_id: 2024-09-01-xyz
created: 2024-09-01T12:34:56+00:00
tags:
  - goal_define
  - stage-01
  - run-2024-09
---

# Define Goal

<stage output content>

#goal_define #stage-01 #run-2024-09
Related: [[run-2024-09]]

Manual KB Entry Creation

You can programmatically write entries outside the standard pipeline using the core API:

from pathlib import Path
from researchclaw.knowledge.base import KBEntry, write_kb_entry

kb_root = Path("docs/kb")
entry = KBEntry(
    category="questions",
    entry_id="sample-001",
    title="Sample Question",
    content="What is the effect of X on Y?",
    source_stage="01-goal_define",
    run_id="run-123",
    tags=["sample", "question"],
    links=["run-123"]  # utilized only with obsidian backend

)

# Generate plain Markdown

write_kb_entry(kb_root, entry, backend="markdown")

# Generate Obsidian-enhanced Markdown

write_kb_entry(kb_root, entry, backend="obsidian")

Runtime Backend Inspection

To verify which backend is active during execution:

from researchclaw.config import RCConfig

cfg = RCConfig.load("config.arc.yaml")
print("KB backend:", cfg.knowledge_base.backend)   # Outputs: markdown or obsidian

Key Source Files and Functions

Understanding these source locations helps with debugging and customization:

• researchclaw/knowledge/base.py: Contains KBEntry dataclass, write_kb_entry, _obsidian_enhancements, and write_stage_to_kb
• researchclaw/config.py: Defines KB_BACKENDS, KnowledgeBaseConfig, and validation logic
• researchclaw/wizard/quickstart.py: Default quickstart template sets backend: "markdown" at lines L90-L92
• tests/test_rc_config.py: Contains test_validate_config_rejects_invalid_knowledge_base_backend ensuring only valid backends pass validation

Summary

• Default behavior: The markdown backend produces standard YAML front-matter without Obsidian-specific syntax.
• Obsidian integration: Set backend: obsidian to append hashtag tags and wikilinks compatible with Obsidian graph view.
• Validation: Invalid backend values trigger errors during config validation in researchclaw/config.py.
• API consistency: All pipeline stages use write_stage_to_kb, which respects the centralized backend configuration automatically.

Frequently Asked Questions

What is the default knowledge base backend in AutoResearchClaw?

The default backend is markdown, as defined in the quickstart wizard at researchclaw/wizard/quickstart.py lines 90-92 and implemented in write_kb_entry in researchclaw/knowledge/base.py. When no backend is specified, the system generates plain Markdown files with YAML front-matter but no Obsidian-specific extensions.

How do I validate my backend configuration?

AutoResearchClaw validates the backend during config loading via the validate_config function in researchclaw/config.py (lines 82-85). The validator checks your configured backend against the KB_BACKENDS constant, which only permits the strings "markdown" and "obsidian". Invalid values raise a configuration error immediately upon loading.

Can I use Obsidian files in other Markdown editors?

Yes. Both backends produce standard Markdown files readable in any editor. The obsidian backend adds extra syntax—specifically inline hashtags like #tag and wikilinks like [[page]]—which appear as plain text in standard Markdown viewers but render as interactive links in Obsidian. The core content and YAML front-matter remain portable across all tools.

Where is the backend validation logic located?

The validation logic resides in researchclaw/config.py. The KB_BACKENDS constant (lines 96-99) enumerates valid options, while the KnowledgeBaseConfig dataclass (lines 165-170) stores the configuration structure. The validate_config function performs the actual validation check at lines 82-85, ensuring only supported backend strings reach the knowledge base writing functions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}