# How the AI Assistant Is Integrated into the AppFlowy Editor: A Technical Deep Dive > Discover how the AppFlowy editor integrates its AI assistant with a three-layer architecture. Learn about Flutter UI, Dart services, and Rust backend for AI inference. - Repository: [AppFlowy-IO/AppFlowy](https://github.com/AppFlowy-IO/AppFlowy) - Tags: deep-dive - Published: 2026-03-03 --- **AppFlowy embeds AI capabilities directly into the editor through a three-layer architecture: Flutter UI components insert special AI nodes, Dart services stream completions via protobuf, and Rust backend managers orchestrate local and cloud model inference.** The AppFlowy-IO/AppFlowy repository implements a tightly coupled AI assistant system that allows users to generate, improve, and explain content without leaving the document editor. This integration spans the Flutter frontend, Dart service layer, and Rust core, enabling real-time streaming of AI responses through custom document nodes. Understanding this architecture reveals how modern open-source editors can seamlessly blend large language model capabilities with collaborative document editing. ## Architecture Overview: Three-Layer Integration The AI assistant system operates across three distinct layers that communicate through well-defined interfaces: 1. **Editor UI Layer (Flutter/Dart)** – Handles toolbar interactions, custom AI node rendering, and state management via `AiWriterCubit` 2. **AI Service Layer (Dart)** – Manages protobuf communication through `AppFlowyAIService`, establishing native ports for streaming responses 3. **Core AI Manager (Rust)** – Orchestrates model selection, chat history, and retrieval-augmented generation (RAG) through the `AIManager` in the `flowy-ai` crate This architecture ensures that swapping between cloud providers and local AI models requires no changes to the editor UI code. ## Editor UI Layer: Inserting AI Nodes When a user clicks an AI toolbar button—such as "Improve writing" or "Explain"—the system creates a specialized document node using the logic defined in **`ai_writer_toolbar_item.dart`**. The `_insertAiNode` function creates an `ai_writer` node at the current cursor position: ```dart // frontend/appflowy_flutter/lib/plugins/document/presentation/editor_plugins/ai/ai_writer_toolbar_item.dart void _insertAiNode(EditorState editorState, AiWriterCommand command) async { final selection = editorState.selection?.normalized; if (selection == null) return; final transaction = editorState.transaction ..insertNode( selection.end.path.next, aiWriterNode( selection: selection, command: command, ), ) ..selectionExtraInfo = {selectionExtraInfoDisableToolbar: true}; await editorState.apply(transaction, options: const ApplyOptions(recordUndo: false, inMemoryUpdate: true), withUpdateSelection: false); } ``` The **`AiWriterCommand`** enum (defined in `ai_writer_entities.dart`) encodes the specific operation type, whether improving grammar, fixing spelling, or generating explanations. The inserted node renders through **`ai_writer_block_component.dart`**, displaying loading states, streaming text, and action buttons within the document flow. ## State Management: The AiWriterCubit Once inserted, the AI node registers itself with **`AiWriterCubit`**, a BLoC-pattern state manager that bridges the UI and backend services. The cubit collects selected text, initiates the AI stream, and handles user actions like stop or retry. The registration process establishes the connection between the document node and the AI service: ```dart // frontend/appflowy_flutter/lib/plugins/document/presentation/editor_plugins/ai/operations/ai_writer_cubit.dart void register(Node node) async { if (node.isAiWriterInitialized) return; aiWriterNode = node; onCreateNode?.call(); await setAiWriterNodeIsInitialized(editorState, node); final command = node.aiWriterCommand; final (run, prompt) = await _addSelectionTextToRecords(command); if (!run) return exit(); runCommand(command, prompt, null, null); } ``` The cubit maintains the `aiWriterNode` reference and manages the node's lifecycle, ensuring that document updates remain responsive even while streaming large completions. It also handles the `TextRobot` helper that progressively applies AI-generated text to the document. ## Dart-to-Rust Communication: Streaming Completions `AiWriterCubit` delegates network operations to **`AppFlowyAIService`**, the concrete implementation of the `AIRepository` abstract class. This service constructs protobuf messages and establishes a native port for receiving streamed responses from Rust. The `streamCompletion` method builds a `CompleteTextPB` payload and initiates the request: ```dart // frontend/appflowy_flutter/lib/ai/service/appflowy_ai_service.dart Future<(String, CompletionStream)?> streamCompletion({ required String text, required CompletionTypePB completionType, ... }) async { final stream = AppFlowyCompletionStream(...); final payload = CompleteTextPB( text: text, completionType: completionType, streamPort: fixnum.Int64(stream.nativePort), ); return AIEventCompleteText(payload).send().fold( (task) => (task.taskId, stream), (error) => null, ); } ``` **`AppFlowyCompletionStream`** creates a `RawReceivePort` that receives chunked responses from the Rust backend through FFI (Foreign Function Interface). The stream routes events to callbacks—including `processMessage`, `processAssistMessage`, and `onError`—which the cubit consumes to update the document in real time. ## Rust Backend: The AIManager Orchestration On the native side, requests arrive at **`AIManager`** in [`frontend/rust-lib/flowy-ai/src/ai_manager.rs`](https://github.com/AppFlowy-IO/AppFlowy/blob/main/frontend/rust-lib/flowy-ai/src/ai_manager.rs). This core component handles model selection, chat initialization, and response streaming. The manager supports both cloud-based and local AI inference: ```rust // frontend/rust-lib/flowy-ai/src/ai_manager.rs pub async fn stream_chat_message(&self, params: StreamMessageParams) -> Result<(), FlowyError> { let chat = self.get_or_create_chat(¶ms.chat_id).await?; let mut stream = chat.stream_message(params).await?; while let Some(chunk) = stream.next().await { self.external_service.notify_did_send_message(&chat.id, &chunk).await?; } Ok(()) } ``` **`AIManager`** uses `ModelSelectionControl` to determine whether to route requests to `ChatServiceMiddleware` (cloud) or `LocalAIController` (local Ollama/LLM instances). The `reload_with_workspace_id` method monitors workspace settings to start or stop local AI plugins based on user preferences, enabling seamless switching between cloud and on-device inference. ## Settings and Model Configuration User preferences flow through the settings UI in **`settings_ai_view.dart`** and **`local_ai_setting.dart`**, which dispatch actions to `SettingsAIBloc`. These settings control: - **Local AI enablement** – Toggles on-device inference via Ollama or llama.cpp - **Model selection** – Chooses between cloud providers or local model files - **Workspace-level AI policies** – Determines availability per workspace The `AIManager` exposes these configurations through its `model_control` interface, ensuring that the editor respects user privacy and performance preferences when initiating AI operations. ## Summary - **AI nodes are first-class document blocks** created via toolbar actions in `ai_writer_toolbar_item.dart`, inserting specialized nodes that render inline with other content. - **State management isolates complexity** through `AiWriterCubit`, which handles text extraction, streaming callbacks, and document updates without blocking the UI thread. - **Protobuf over FFI enables real-time streaming** through `AppFlowyAIService`, using `RawReceivePort` to receive chunks from Rust without polling. - **Rust backend abstracts model providers** via `AIManager`, supporting both cloud APIs and local inference through a unified `Chat` interface. - **Pluggable architecture** allows swapping LLM providers by implementing `AIExternalService` or `ChatCloudService` interfaces without modifying editor code. ## Frequently Asked Questions ### How does AppFlowy handle AI streaming without freezing the editor? The editor uses **isolate communication through `RawReceivePort`** to receive AI chunks asynchronously. The `AppFlowyCompletionStream` creates a native port that receives data from Rust on a separate thread, while the `AiWriterCubit` updates the Flutter UI using the `TextRobot` helper to progressively apply text. This ensures the editor remains responsive even during long-running completions. ### Can AppFlowy work with local AI models instead of cloud APIs? Yes, the **`AIManager`** in the Rust layer supports local inference through the `LocalAIController`. Users enable this in `settings_ai_view.dart`, which triggers `reload_with_workspace_id` to initialize local AI plugins. The system routes requests to either `ChatServiceMiddleware` (cloud) or local controllers based on the `ModelSelectionControl` configuration, allowing offline AI usage via Ollama or similar local LLM runners. ### What happens when a user clicks "Improve writing" in the toolbar? The toolbar triggers `_insertAiNode` with `AiWriterCommand.improveWriting`, which inserts an `ai_writer` node into the document. This node registers with `AiWriterCubit`, extracts the selected text, and calls `AppFlowyAIService.streamCompletion`. The service sends a `CompleteTextPB` protobuf message to the Rust backend, which streams improved text back through the native port, updating the document node in real time. ### Where is the AI prompt logic defined in the codebase? Prompt definitions and command types reside in **`ai_writer_entities.dart`** (Dart) and the corresponding protobuf definitions in `frontend/rust-lib/flowy-ai/protobuf/`. The `AiWriterCommand` enum encodes operations like `fixSpellingAndGrammar` and `explain`, while the actual prompt engineering occurs in the Rust layer within [`chat_service_mw.rs`](https://github.com/AppFlowy-IO/AppFlowy/blob/main/chat_service_mw.rs) or local AI controllers, depending on the selected model provider.