Approach Taken by the Screenshot Utility for Capturing UI States in Lemon AI

Lemon AI generates visual UI snapshots by orchestrating a headless Chromium browser via Playwright, injecting authentication tokens into localStorage, applying resilient multi-stage navigation waits, and exporting stabilized screenshots with full metadata context.

The Lemon AI platform depends on deterministic visual regression and state documentation to power its conversation interfaces. The screenshot utility, implemented in src/utils/screen_shot.js, abstracts complex browser automation into a robust service that captures authenticated, fully-rendered web pages as portable image assets.

Core Architecture and Browser Initialization

The utility centers on two primary exports: takeScreenshot for single captures and takeMultipleScreenshots for batch operations. Both rely on Playwright’s Chromium integration to ensure consistent cross-environment rendering.

Headless Chromium Launch

The implementation initializes an isolated browser instance using chromium.launch({ headless }). Each invocation creates a fresh browser context and page with a configurable viewport (width × height), guaranteeing that responsive layouts render identically across capture sessions.

Authentication Pre-Injection

For protected application routes, the utility accepts an accessToken parameter. Before navigating to the target URL, it executes localStorage.setItem('access_token', token) within the page context. This injection authenticates the session without requiring manual login flows, ensuring that private dashboards render with correct user state.

Resilient Navigation and Load Stabilization

Capturing dynamic single-page applications requires more than simple document readiness checks. The utility implements a cascading wait strategy to ensure the DOM is fully realized before image generation.

Progressive Wait Strategies

The navigation logic attempts page loads with three sequential readiness conditions:

• domcontentloaded – fires when the initial HTML document is parsed
• load – waits for all dependent resources including stylesheets and images
• networkidle – pauses execution until network activity ceases for a specified duration

If an earlier strategy fails or times out, the utility automatically falls back to the next condition, providing resilience against slow or asynchronously loaded content.

Post-Load Stabilization

After successful navigation, an optional waitTime parameter (defaulting to 2000 ms) allows JavaScript-driven rendering to complete. The utility also validates that the HTML payload exceeds a trivial size threshold, aborting early with an error if the page fails to render substantive content.

Image Capture and Metadata Assembly

Once the UI stabilizes, the utility invokes page.screenshot() to generate binary image data with configurable output characteristics.

Output Configuration

Screenshots are persisted to public/screenshots/ by default, or a custom outputPath if specified. The function supports:

• Format selection: png or jpeg with adjustable quality settings
• Full-page capture: fullPage: true captures vertically scrolling content beyond the initial viewport dimensions

Contextual Metadata

The function returns a structured result object containing:

• buffer: Raw image data for immediate processing or transmission
• outputPath: Filesystem location of the saved asset
• metadata: Title, final URL, viewport dimensions, and capture options used

This metadata enables downstream services to correlate visual snapshots with specific application states and conversation contexts.

Batch Processing and PDF Generation

For scenarios requiring multiple captures, takeMultipleScreenshots iterates over an array of URLs, reusing the core logic while respecting an optional delay parameter between requests to prevent rate limiting.

The utility also exposes generatePDF, which repurposes the same Playwright workflow to export pages as PDF documents, maintaining identical authentication and navigation strategies for consistency across output formats.

Integration with Lemon AI Services

The screenshot utility operates as a foundational service layer, invoked by both API endpoints and frontend components.

API Endpoints

The Express router in src/routers/conversation/conversation.js exposes POST /screenshots/single and batch routes. These endpoints parse request bodies containing target URLs and conversation IDs, then delegate execution to takeScreenshot or takeMultipleScreenshots, returning JSON responses with screenshot URLs and metadata.

Frontend Rendering

Captured screenshots surface within the chat interface via frontend/src/view/lemon/components/ChatMessages.vue. When a message metadata object contains a screenshot property, the component renders the image inline, providing users with visual context for AI-generated responses. The Vuex store module in frontend/src/store/modules/chat.js persists these screenshot URLs within message state, ensuring visual snapshots remain available across session reloads.

Summary

• Lemon AI utilizes Playwright-driven headless Chromium to generate deterministic UI snapshots with consistent viewport rendering.
• Authentication tokens are pre-injected via localStorage to capture protected application states without manual login.
• Resilient navigation employs cascading wait strategies (domcontentloaded, load, networkidle) and post-load stabilization to ensure complete rendering.
• Screenshots are persisted to public/screenshots/ with configurable formats, full-page options, and comprehensive metadata including buffers and file paths.
• The utility integrates with Express API routes and Vue.js frontend components to embed visual context within conversation flows.

Frequently Asked Questions

What browser engine powers Lemon AI's screenshot utility?

The utility leverages Chromium in headless mode, orchestrated through the Playwright library. This combination provides a consistent, modern rendering engine that supports complex CSS, JavaScript-driven SPAs, and WebGL content across all capture sessions.

How does the utility handle authentication for private pages?

Before navigating to the target URL, the utility checks for an accessToken parameter. If present, it executes localStorage.setItem('access_token', token) within the browser page context. This injection authenticates the session immediately, allowing the screenshot to capture fully-rendered protected dashboards without encountering login screens.

What mechanisms ensure the page is fully loaded before capture?

The implementation uses a cascading wait strategy that attempts three conditions in sequence: domcontentloaded for initial HTML parsing, load for resource completion, and networkidle for network quiescence. If one fails, the next is attempted. Additionally, a configurable waitTime (default 2000 ms) provides post-load stabilization for JavaScript-rendered content.

Where are screenshot files stored and how are they accessed by the frontend?

By default, images are written to the public/screenshots/ directory on the server filesystem. The takeScreenshot function returns an outputPath and public URL that the API layer forwards to the frontend. Components like ChatMessages.vue reference these URLs to render inline images within chat bubbles, while the Vuex store persists the references in conversation state for cross-session availability.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}