Where to Find Higher-Level Tools like Conversation and Document Handlers in SymbolicAI

The higher-level tools like Conversation and document handlers in SymbolicAI are located in the symai/extended package, which provides ready-to-use utilities for chat-style interactions, vector-based document storage, and various content parsers.

The extensityai/symbolicai repository organizes its advanced functionality into a dedicated extended module. These tools inherit from the Expression base class defined in symai/symbol.py, giving them native access to LLM-aware Symbol handling and lazy evaluation capabilities.

The symai/extended Package: Home of High-Level Utilities

The symai/extended directory houses user-facing, high-level Expression subclasses that combine multiple lower-level primitives—such as memory systems, processors, and interfaces—into cohesive workflows. These classes are designed for immediate use without requiring deep knowledge of the underlying Symbol mechanics.

Key characteristics of this package:

• All classes extend Expression from symai/symbol.py
• They integrate with symai/memory.py for stateful operations
• They consume configuration from symai/backend/settings.py for API keys and engine selection

Core Higher-Level Tools and Their Locations

Conversation: Chat-Style Memory and Interaction

The Conversation class, defined in [symai/extended/conversation.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/conversation.py), provides a sliding-window memory system for multi-turn dialogues. It extends SlidingWindowStringConcatMemory from symai/memory.py and wires together several components:

• File ingestion: Uses FileReader from symai/components.py via the store_file method
• URL scraping: Leverages Interface("naive_scrape") via the store_url method
• Persistence: Implements save_conversation_state and load_conversation_state using Python's pickle module

from symai.extended.conversation import Conversation

# Initialise with an optional system prompt and a file to preload

conv = Conversation(
    init="You are a helpful AI assistant.",
    file_link="README.md",          # automatically reads the file content

    auto_print=True                # prints each LLM response

)

# Send a query – the underlying LLM engine is selected from the config

reply = conv.forward("Summarize the purpose of Symbolic AI.")

# reply is a Symbol; its value is printed automatically because auto_print=True

Persist & restore

# Save the whole conversation (including memory) to disk

Conversation.save_conversation_state(conv, "my_chat.pkl")

# Later… load and continue

new_conv = Conversation()
new_conv = new_conv.load_conversation_state("my_chat.pkl")
new_conv.forward("What did we talk about earlier?")

VectorDB: Vector-Based Document Store

The VectorDB class, located in [symai/extended/vectordb.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/vectordb.py), implements a vector database as an Expression subclass. It handles document storage, embedding generation, and similarity search.

Key implementation details:

• Loads configuration from symai/backend/settings.py to select the embedding engine (defaults to a local model if EMBEDDING_ENGINE_API_KEY is absent)
• Initializes the embedding function via _init_embedding_model
• Normalizes input through helpers _unwrap_documents, _to_texts, _embed_batch, and _raise_texts_unassigned
• Supports persistence via save and load methods, plus clear for cleanup

from symai.extended.vectordb import VectorDB

# Create a DB – it will lazily load the configured embedding engine

db = VectorDB()

# Add arbitrary documents (dicts, strings, or any JSON‑serialisable structure)

docs = [
    {"title": "Neuro‑Symbolic AI", "text": "Combines symbolic reasoning with neural nets."},
    {"title": "Vector Search", "text": "Efficient similarity lookup using embeddings."},
]
db.add_documents(docs)

# Perform a similarity query

query = "How do neural networks integrate with symbolic rules?"
results = db(query)          # returns a Symbol wrapping the top‑k most similar docs

print(results.value)        # → list of matching document dicts

FileMerger: Recursive File Concatenation

Found in [symai/extended/file_merger.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/file_merger.py), the FileMerger class recursively reads files with selectable extensions, skips excluded names, and returns a single merged text Symbol. This utility is used by the Conversation class for ingesting local codebases or documentation.

from symai.extended.file_merger import FileMerger

merger = FileMerger(file_endings=[".py", ".md"])
merged_symbol = merger.forward(root_path="symai")   # merges all .py/.md under symai/

print(merged_symbol.value[:500])                  # preview first 500 chars

Document Parsers: BibTeX and ArXiv

SymbolicAI provides specialized parsers for academic content:

BibTeX Parser ([symai/extended/bibtex_parser.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/bibtex_parser.py)) converts BibTeX strings into structured Symbol representations.

from symai.extended.bibtex_parser import BibTexParser

bib_string = """
@article{smith2023,
  title={Symbolic AI in practice},
  author={Smith, John},
  journal={Journal of AI Research},
  year={2023}
}
"""
parser = BibTexParser()
bib_symbol = parser.forward(bib_string)
print(bib_symbol.value)   # → list of dicts with parsed fields

ArXiv PDF Parser ([symai/extended/arxiv_pdf_parser.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/arxiv_pdf_parser.py)) downloads an arXiv PDF, extracts raw text, and returns it as a Symbol.

Supporting Infrastructure

Several lower-level modules provide the backbone for these high-level tools:

• symai/components.py – Contains FileReader, the thin wrapper used by both Conversation and FileMerger to read file paths and return Symbol objects.
• symai/memory.py – Houses SlidingWindowStringConcatMemory, the base class extended by Conversation to manage rolling transcripts.
• symai/backend/settings.py – Stores configuration including API keys and default engines, accessed by VectorDB to initialize embedding models.
• symai/symbol.py – Defines the Expression and Symbol base classes that power all extended tools with LLM-aware handling and lazy evaluation.

Summary

• Location: All higher-level tools like Conversation and document handlers reside in the symai/extended package of the extensityai/symbolicai repository.
• Conversation ([symai/extended/conversation.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/conversation.py)): Provides sliding-window memory, file/URL ingestion, and state persistence for multi-turn dialogues.
• VectorDB ([symai/extended/vectordb.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/vectordb.py)): Implements vector-based document storage with configurable embedding engines and similarity search.
• FileMerger ([symai/extended/file_merger.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/file_merger.py)): Recursively merges codebase files into a single Symbol for ingestion.
• Document Parsers: Specialized handlers for BibTeX ([symai/extended/bibtex_parser.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/bibtex_parser.py)) and arXiv PDFs ([symai/extended/arxiv_pdf_parser.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/arxiv_pdf_parser.py)).

Frequently Asked Questions

Where is the Conversation class defined in SymbolicAI?

The Conversation class is defined in [symai/extended/conversation.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/conversation.py). It extends SlidingWindowStringConcatMemory from symai/memory.py and integrates FileReader from symai/components.py to handle file ingestion, URL scraping via Interface("naive_scrape"), and state persistence through pickle-based save/load methods.

How does VectorDB handle document embeddings?

VectorDB, located in [symai/extended/vectordb.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/vectordb.py), loads configuration from symai/backend/settings.py to select an embedding engine, defaulting to a local model if EMBEDDING_ENGINE_API_KEY is absent. It normalizes input documents via helper methods _unwrap_documents and _to_texts, then computes embeddings in batches using _embed_batch before storing them for similarity search.

What is the difference between FileMerger and FileReader?

FileMerger ([symai/extended/file_merger.py](https://github.com/extensityai/symbolicai/blob/main/symai/extended/file_merger.py)) is a high-level utility that recursively traverses directories, filters by file extensions, and concatenates multiple files into a single Symbol. FileReader ([symai/components.py](https://github.com/extensityai/symbolicai/blob/main/symai/components.py)) is a lower-level component that simply reads a single file path and returns its content as a Symbol, serving as the building block used by FileMerger and Conversation.

Can I persist conversation state across sessions?

Yes, the Conversation class provides built-in persistence through save_conversation_state and load_conversation_state class methods. These methods use Python's pickle module to serialize the entire conversation object—including the sliding-window memory and message history—to disk, allowing you to restore the exact state later and continue the dialogue without losing context.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}