REST API Endpoint Structure in Open-Notebook: Routers and Service Layer Pattern Explained

The open-notebook backend implements a clean three-tier architecture where FastAPI routers handle HTTP transport, service modules provide reusable client abstractions, and domain objects encapsulate SurrealDB persistence logic.

This architecture separates concerns between the web interface and data layer, making the codebase maintainable and testable. The lfnovo/open-notebook repository uses FastAPI for its HTTP framework and organizes code into distinct router, service, and domain layers. Understanding this structure helps developers extend endpoints or build clients that consume the API correctly.

Router Layer Architecture

All public HTTP endpoints live under api/routers/ and follow a consistent FastAPI pattern. Each router file declares an APIRouter instance, defines Pydantic request/response models, and delegates business logic to the domain layer or service wrappers.

Route Organization and File Structure

The router files map to core domain resources:

• api/routers/sources.py: Handles GET /sources, POST /sources, PUT /sources/{id}, and multipart file uploads. Uses parse_source_form_data to process form data and returns SourceResponse models.
• api/routers/notes.py: Manages GET /notes, POST /notes, and PUT /notes/{id} endpoints. Directly instantiates Note domain objects and serializes responses via NoteResponse.
• api/routers/notebooks.py: Contains CRUD operations plus relationship endpoints like POST /notebooks/{nb}/sources/{src}. Builds SurrealQL queries via repo_query and returns NotebookResponse objects.
• api/routers/models.py, search.py, insights.py: Follow the same skeleton for LLM model metadata, vector search, and AI-generated insights.

Common Router Responsibilities

Every router handles five distinct concerns:

1. Parameter handling: Extracts FastAPI Query, Form, and UploadFile parameters from requests.
2. Input validation: Converts string booleans and JSON strings using utilities like parse_source_form_data.
3. Domain interaction: Calls await Notebook.get(id) or await Source.save() to trigger business logic.
4. Error mapping: Translates domain exceptions (NotFoundError, InvalidInputError) into FastAPI HTTPException instances with appropriate status codes.
5. Response shaping: Constructs Pydantic models (SourceResponse, NoteResponse) that conform to the API contract.

Service Layer Pattern

The service files (api/sources_service.py, api/notes_service.py, api/notebook_service.py) sit adjacent to the routers and provide a client-side API for the frontend. These classes wrap the HTTP client (api.client.api_client) and translate JSON payloads into rich domain objects, offering a programmatic interface that mirrors the REST endpoints.

Service Method Flow and Data Transformation

Service methods follow a four-step pipeline:

1. HTTP invocation: Calls api_client.get_sources(notebook_id=...) or similar methods.
2. Deserialization: Transforms JSON dictionaries into domain objects like Source, Asset, or Note.
3. Enrichment: Wraps raw objects with metadata classes (SourceWithMetadata) to add computed properties like embedded_chunks counts.
4. Return: Returns plain Python objects or async result wrappers (SourceProcessingResult) for background jobs.

# From api/sources_service.py (lines 72-99)

def get_all_sources(self, notebook_id: Optional[str] = None) -> List[SourceWithMetadata]:
    sources_data = api_client.get_sources(notebook_id=notebook_id)
    sources = []
    for src in sources_data:
        source = Source(
            title=src["title"],
            topics=src["topics"],
            asset=Asset(
                file_path=src["asset"]["file_path"] if src["asset"] else None,
                url=src["asset"]["url"] if src["asset"] else None,
            ),
        )
        source.id = src["id"]
        sources.append(SourceWithMetadata(source, src.get("embedded_chunks", 0)))
    return sources

Integration Flow: How the Layers Connect

The architecture follows a strict dependency direction from HTTP to database:

1. Application bootstrap: api/main.py imports all routers and registers them with app.include_router(sources.router, prefix="/api").
2. Router to domain: Routers call methods on domain objects (open_notebook/domain/notebook.py) rather than writing raw SQL.
3. Domain to repository: Domain objects use open_notebook/database/repository.py functions (repo_query, ensure_record_id) to execute SurrealQL, keeping database details isolated.
4. Service to router: Frontend code in frontend/ uses service methods that internally call the same HTTP endpoints, ensuring single-source-of-truth for request shapes.
5. Async job handling: Routers submit background work via CommandService.submit_command_job() and return command IDs, while services provide polling helpers like is_source_processing_complete().

Practical Code Examples

Async Source Creation in Routers

The POST /sources endpoint in api/routers/sources.py supports both synchronous and asynchronous processing paths:

@router.post("/sources", response_model=SourceResponse)
async def create_source(
    form_data: tuple[SourceCreate, Optional[UploadFile]] = Depends(parse_source_form_data)
):
    source_data, upload_file = form_data
    
    if source_data.async_processing:
        # Async path: queue job and return immediately

        command_id = await CommandService.submit_command_job(...)
        source.command = ensure_record_id(command_id)
        await source.save()
        return SourceResponse(
            ..., 
            command_id=command_id, 
            status="new"
        )
    else:
        # Sync path: block until processing completes

        result = await asyncio.to_thread(
            execute_command_sync, 
            "open_notebook", 
            "process_source", 
            command_input.model_dump()
        )
        return SourceResponse(..., embedded=embedded_chunks > 0)

Service Layer Consumption

The corresponding service method in api/sources_service.py (lines 62-99) handles the client-side translation:

def create_source(self, ...):
    source_data = api_client.create_source(
        notebook_id=notebook_id,
        source_type=source_type,
        async_processing=async_processing,
        ...
    )
    
    source = Source(...)
    
    # Wrap async results with metadata

    if source_data.get("command_id"):
        return SourceProcessingResult(
            source, 
            True, 
            source_data["command_id"], 
            source_data["status"], 
            source_data["processing_info"]
        )
    return source

Notebook Detail Retrieval

Routers use SurrealQL graph queries to compute aggregated counts before returning responses:

# From api/routers/notebooks.py (lines 50-73)

@router.get("/notebooks/{notebook_id}", response_model=NotebookResponse)
async def get_notebook(notebook_id: str):
    query = """
        SELECT *, count(<-reference.in) as source_count, 
               count(<-artifact.in) as note_count
        FROM $notebook_id
    """
    result = await repo_query(query, {"notebook_id": ensure_record_id(notebook_id)})
    nb = result[0]
    
    return NotebookResponse(
        id=str(nb["id"]), 
        name=nb["name"], 
        description=nb["description"],
        archived=nb["archived"], 
        created=str(nb["created"]), 
        updated=str(nb["updated"]),
        source_count=nb["source_count"], 
        note_count=nb["note_count"]
    )

Summary

• Router files in api/routers/ define FastAPI endpoints, handle HTTP concerns, and return Pydantic response models.
• Service files in api/ provide reusable Python APIs that wrap HTTP calls, deserialize JSON into domain objects, and add metadata wrappers.
• Domain objects in open_notebook/domain/ contain business logic and use the repository pattern for SurrealDB access.
• Repository utilities in open_notebook/database/repository.py (repo_query, ensure_record_id) isolate database query syntax from business logic.
• Async processing uses CommandService to queue background jobs, with routers returning command IDs and services providing polling methods.

Frequently Asked Questions

What is the purpose of the service layer if routers already handle HTTP requests?

The service layer provides a programmatic Python API for the frontend and internal consumers. While routers expose HTTP endpoints, service classes like SourcesService translate raw JSON into typed domain objects (Source, Asset) and handle client-side concerns such as deserialization and error retry logic. This prevents the frontend from duplicating HTTP boilerplate and ensures consistent data modeling across the application.

How do routers handle file uploads and form data?

Routers use FastAPI's Depends injection with custom parsers like parse_source_form_data to handle multipart/form-data. This function converts incoming form fields and UploadFile objects into validated Pydantic models (SourceCreate) before the route handler executes, separating HTTP parsing from business logic as seen in api/routers/sources.py.

Where is the database query logic actually implemented?

Database queries are encapsulated in open_notebook/database/repository.py through the repo_query function and ensure_record_id utility. Domain objects call these functions rather than writing raw SurrealQL, creating a clean separation between the FastAPI layer and the SurrealDB persistence mechanism.

Can the service layer be used independently of the frontend?

Yes. The service layer is designed as a standalone client API that any Python code can import. While the frontend uses these services to communicate with the backend, other scripts or microservices can instantiate SourcesService or NotebookService to interact with the API without managing HTTP details manually.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}