# How to Export Palace Data Using the MemPalace Exporter Module

> Learn how to export palace data with MemPalace exporter module. Convert ChromaDB storage to Markdown safely and efficiently with export_palace().

- Repository: [MemPalace/mempalace](https://github.com/MemPalace/mempalace)
- Tags: how-to-guide
- Published: 2026-06-07

---

**The MemPalace exporter module in [`mempalace/exporter.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/exporter.py) provides the `export_palace()` function to convert ChromaDB palace storage into a browsable Markdown directory structure with built-in safety checks against symlink attacks and memory-efficient batch processing.**

The MemPalace repository provides a dedicated exporter module that converts binary palace storage into structured Markdown documentation. To export palace data using the MemPalace exporter module, developers invoke the `export_palace()` function from [`mempalace/exporter.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/exporter.py), which streams drawer contents directly from the underlying ChromaDB collection without loading the entire dataset into memory. This approach generates a hierarchical file system representation while implementing security safeguards against path traversal and symlink attacks.

## Understanding the Exporter Architecture

The core export logic resides in [`mempalace/exporter.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/exporter.py), which orchestrates data retrieval from [`mempalace/palace.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/palace.py) and safe file system operations. When you export palace data using the MemPalace exporter module, the system executes a seven-stage pipeline that maintains constant memory usage regardless of palace size.

### The Export Pipeline

1. **Collection Retrieval**: Calls `get_collection` from [`mempalace/palace.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/palace.py) to establish a connection to the ChromaDB collection containing all drawers.
2. **Directory Validation**: Verifies the output path using `_reject_symlink` to prevent symbolic link attacks and creates the destination folder with restrictive permissions.
3. **Paginated Streaming**: Retrieves drawers in configurable batches (defaulting to 1000 per batch) to bound memory consumption.
4. **Metadata Grouping**: Organizes each batch by the `wing` and `room` metadata fields stored in drawer documents.
5. **Markdown Generation**: Creates an [`index.md`](https://github.com/MemPalace/mempalace/blob/main/index.md) file cataloging all wings alongside individual Markdown files for each room containing block-quoted drawer content and metadata tables.
6. **File Handle Management**: Tracks previously opened room files to determine whether to create (`w`) or append (`a`) content.
7. **Statistics Collection**: Aggregates drawer counts during iteration, returning a summary dictionary upon completion.

## Programmatic Export Usage

Invoke the export functionality directly from Python code to integrate palace exports into larger workflows.

### Basic Script Implementation

```python
from mempalace.exporter import export_palace

# Path to the ChromaDB palace directory

palace_dir = "/home/user/.mempalace/palace"

# Destination for Markdown output

output_dir = "/tmp/palace_export"

# Execute export and retrieve statistics

stats = export_palace(palace_dir, output_dir)

print(f"Exported {stats['total_drawers']} drawers to {output_dir}")

```

### Command Line Execution

While [`mempalace/cli.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/cli.py) provides a thin CLI wrapper, you can also execute the exporter directly via Python's `-c` flag or inline scripts:

```bash
python -c "
from mempalace.exporter import export_palace
export_palace(
    palace_path='/home/user/.mempalace/palace',
    output_dir='/tmp/palace_export'
)
"

```

## Safety Mechanisms and File System Security

The MemPalace exporter module implements defensive programming patterns to handle untrusted input data safely.

### Path Sanitization and Component Escaping

The `_safe_path_component` helper sanitizes wing and room names to ensure valid filesystem components, preventing directory traversal attacks through malicious metadata.

### Symlink Attack Prevention

The exporter employs `_reject_symlink` to refuse writing into paths that are symbolic links, eliminating a class of time-of-check-time-of-use (TOCTOU) vulnerabilities. The `_safe_open_for_write` function further hardens file creation by utilizing `O_NOFOLLOW` flags where the operating system supports them, ensuring opened files are not symlink substitutions.

## Output Structure and Generated Files

When you export palace data using the MemPalace exporter module, the system generates a hierarchical Markdown structure:

- **Index File**: A top-level [`index.md`](https://github.com/MemPalace/mempalace/blob/main/index.md) listing every wing with room counts and total drawer statistics.
- **Wing Directories**: Subdirectories for each wing with names escaped via `_safe_path_component`.
- **Room Files**: Individual `<room>.md` files within wing directories containing:
  - A Markdown heading identifying the room
  - Block-quoted drawer content
  - A metadata table documenting source, filed date, and added-by fields

This structure enables human browsing while preserving the relational organization of your memory palace.

## Summary

- The `export_palace()` function in [`mempalace/exporter.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/exporter.py) converts ChromaDB palace data into Markdown documentation.
- Export operations stream data in 1000-drawer batches to maintain constant memory usage for large palaces.
- Built-in safety helpers `_safe_path_component`, `_reject_symlink`, and `_safe_open_for_write` protect against path traversal and symlink attacks.
- The generated output includes an [`index.md`](https://github.com/MemPalace/mempalace/blob/main/index.md) file and hierarchical wing/room Markdown files with drawer metadata.
- The module integrates with [`mempalace/palace.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/palace.py) for database access and supports both programmatic and command-line invocation.

## Frequently Asked Questions

### What file format does the MemPalace exporter generate?

The exporter produces Markdown (`.md`) files. It creates an [`index.md`](https://github.com/MemPalace/mempalace/blob/main/index.md) at the root level containing wing and room statistics, plus individual room files organized within wing subdirectories. Each room file contains block-quoted drawer content alongside metadata tables showing source, filing date, and contributor information.

### How does the exporter handle very large palace datasets?

Rather than loading the entire palace into RAM, the exporter streams drawers in paginated batches of 1000 by default. This pagination occurs within the main processing loop in [`mempalace/exporter.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/exporter.py), ensuring memory consumption remains bounded regardless of total drawer count.

### Is the exporter safe to run against untrusted palace data?

Yes, the MemPalace exporter module implements multiple security layers. The `_safe_path_component` function sanitizes directory names derived from metadata, while `_reject_symlink` and `_safe_open_for_write` prevent symlink attacks and TOCTOU race conditions during file creation. These mechanisms allow safe execution even when palace metadata contains malicious path strings.

### Where is the primary export logic implemented?

The main export functionality resides in [`mempalace/exporter.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/exporter.py), specifically within the `export_palace()` function. This module imports `get_collection` from [`mempalace/palace.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/palace.py) to access the underlying ChromaDB storage, while CLI access is available through [`mempalace/cli.py`](https://github.com/MemPalace/mempalace/blob/main/mempalace/cli.py).