how-to-guide

How AstrBot Integrates ComfyUI with Its T2I Pipeline for AI Image Generation

March 12, 2026 AstrBotDevs/AstrBot ↗

AstrBot delegates text-to-image generation to a configurable HTTP endpoint via the HtmlRenderer class, allowing seamless integration with ComfyUI by setting t2i_endpoint to your local ComfyUI server URL.

AstrBot implements a pluggable rendering service for text-to-image (T2I) generation that remains backend-agnostic. Rather than embedding image generation logic directly, the core pipeline delegates rendering to an HtmlRenderer facade that can route requests to remote services like ComfyUI or fall back to local processing.

Architecture of the AstrBot T2I Pipeline

The T2I system operates through a dual-mode architecture defined in astrbot/core/utils/t2i/renderer.py. The HtmlRenderer class acts as a high-level façade that selects between two strategies based on the t2i_strategy configuration key.

How the Pipeline Decides to Trigger Image Generation

The ResultDecorateStage class in astrbot/core/pipeline/result_decorate/stage.py controls when T2I activation occurs. When processing a response containing Plain components, the stage checks if the text exceeds the t2i_word_threshold (default 150 words).

If the threshold is exceeded, the stage invokes the global html_renderer singleton:

url = await html_renderer.render_t2i(
    plain_str,
    return_url=True,
    use_network=self.t2i_use_network,          # Maps to remote strategy

    template_name=self.t2i_active_template,    # Template selection

)

The html_renderer singleton is instantiated in astrbot/core/__init__.py with the configured endpoint:

t2i_base_url = astrbot_config.get("t2i_endpoint", "https://t2i.soulter.top/text2img")
html_renderer = HtmlRenderer(t2i_base_url)

Integrating ComfyUI via the Remote Rendering Strategy

To route T2I requests to ComfyUI, configure the t2i_endpoint in astrbot/core/config/default.py or via user configuration to point to your ComfyUI server URL:

{
  "t2i_endpoint": "http://localhost:8188/text2img"
}

The NetworkRenderStrategy implements the HTTP contract required for backend integration. It constructs a POST request to /generate with the following payload structure:

{
    "tmpl": template_name,
    "json": markdown_text,
    "tmpldata": {},
    "options": {}
}

The strategy handles response parsing by checking for HTTP 200 status. If successful, it returns either a constructed URL ("{endpoint}/{image_id}") or raw image bytes depending on the return_url parameter.

If the remote call fails due to network errors, non-200 responses, or timeouts, HtmlRenderer.render_t2i automatically falls back to LocalRenderStrategy, ensuring service continuity even when ComfyUI is unavailable.

Local Fallback Rendering with Pillow

When t2i_strategy is set to "local" or when the remote endpoint fails, LocalRenderStrategy in astrbot/core/utils/t2i/local_strategy.py executes. This strategy renders Markdown directly to PNG using Pillow without external dependencies, converting the text to an image format suitable for chat platforms.

Configuring Custom Templates for ComfyUI

AstrBot supports custom HTML/Jinja2 templates for T2I generation, managed through the dashboard API in astrbot/dashboard/routes/t2i.py. Templates are stored in data/t2i_templates and can include predefined styles, layouts, or ComfyUI-specific prompt structures.

To create a template:

POST /t2i/templates/create
Content-Type: application/json

{
  "name": "comfy_cinematic",
  "content": "<html><body style='background:#000;color:#fff'><h1>{{ text }}</h1></body></html>"
}

Activate it for subsequent renders:

POST /t2i/templates/set_active
{
  "name": "comfy_cinematic"
}

The template_name parameter is passed through HtmlRenderer.render_t2i to the remote endpoint, allowing ComfyUI workflows to receive fully-formed prompts based on the selected template.

Summary

  • AstrBot's T2I pipeline uses a strategy pattern via HtmlRenderer to support both remote and local rendering modes.
  • ComfyUI integration requires setting t2i_endpoint to your ComfyUI server URL and configuring t2i_strategy to "remote".
  • Automatic fallback to local Pillow rendering occurs if the ComfyUI endpoint fails or returns errors.
  • Custom templates can be defined via the dashboard API and passed to the remote endpoint for specialized styling.
  • Key files include astrbot/core/utils/t2i/renderer.py, network_strategy.py, and astrbot/core/pipeline/result_decorate/stage.py.

Frequently Asked Questions

How do I configure AstrBot to use my local ComfyUI instance for image generation?

Set the t2i_endpoint configuration key to your ComfyUI server URL (e.g., http://localhost:8188/text2img) and ensure t2i_strategy is set to "remote". This routes all T2I requests through the NetworkRenderStrategy instead of the local renderer.

What happens if the ComfyUI backend is offline when AstrBot tries to generate an image?

If the remote endpoint returns a non-200 status code or raises a network exception, HtmlRenderer.render_t2i automatically catches the error and falls back to LocalRenderStrategy, which renders the text as a PNG using Pillow without requiring external services.

Can I use custom HTML templates with ComfyUI through AstrBot's T2I pipeline?

Yes, custom Jinja2 templates can be created via the dashboard API endpoints in astrbot/dashboard/routes/t2i.py and stored in data/t2i_templates. When rendering, the selected template_name is passed to the ComfyUI endpoint via the tmpl field in the JSON payload, allowing your ComfyUI workflow to apply the template styling.

Have a question about this repo?

These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:

Share the following with your agent to get started:
curl -s "https://instagit.com/install.md"
Works with
Claude Codex Cursor VS Code OpenClaw Any MCP Client

Maintain an open-source project? Get it listed too →