# Approach Taken by the Screenshot Utility for Capturing UI States in Lemon AI

> Discover how Lemon AI captures UI states using a headless Chromium browser with Playwright. Learn about authentication injection, navigation waits, and metadata export for stable UI snapshots.

- Repository: [hexdocom/lemonai](https://github.com/hexdocom/lemonai)
- Tags: internals
- Published: 2026-03-03

---

**Lemon AI generates visual UI snapshots by orchestrating a headless Chromium browser via Playwright, injecting authentication tokens into localStorage, applying resilient multi-stage navigation waits, and exporting stabilized screenshots with full metadata context.**

The Lemon AI platform depends on deterministic visual regression and state documentation to power its conversation interfaces. The screenshot utility, implemented in [`src/utils/screen_shot.js`](https://github.com/hexdocom/lemonai/blob/main/src/utils/screen_shot.js), abstracts complex browser automation into a robust service that captures authenticated, fully-rendered web pages as portable image assets.

## Core Architecture and Browser Initialization

The utility centers on two primary exports: `takeScreenshot` for single captures and `takeMultipleScreenshots` for batch operations. Both rely on Playwright’s Chromium integration to ensure consistent cross-environment rendering.

### Headless Chromium Launch

The implementation initializes an isolated browser instance using `chromium.launch({ headless })`. Each invocation creates a fresh browser context and page with a configurable viewport (`width` × `height`), guaranteeing that responsive layouts render identically across capture sessions.

### Authentication Pre-Injection

For protected application routes, the utility accepts an `accessToken` parameter. Before navigating to the target URL, it executes `localStorage.setItem('access_token', token)` within the page context. This injection authenticates the session without requiring manual login flows, ensuring that private dashboards render with correct user state.

## Resilient Navigation and Load Stabilization

Capturing dynamic single-page applications requires more than simple document readiness checks. The utility implements a cascading wait strategy to ensure the DOM is fully realized before image generation.

### Progressive Wait Strategies

The navigation logic attempts page loads with three sequential readiness conditions:
- **`domcontentloaded`** – fires when the initial HTML document is parsed
- **`load`** – waits for all dependent resources including stylesheets and images
- **`networkidle`** – pauses execution until network activity ceases for a specified duration

If an earlier strategy fails or times out, the utility automatically falls back to the next condition, providing resilience against slow or asynchronously loaded content.

### Post-Load Stabilization

After successful navigation, an optional `waitTime` parameter (defaulting to 2000 ms) allows JavaScript-driven rendering to complete. The utility also validates that the HTML payload exceeds a trivial size threshold, aborting early with an error if the page fails to render substantive content.

## Image Capture and Metadata Assembly

Once the UI stabilizes, the utility invokes `page.screenshot()` to generate binary image data with configurable output characteristics.

### Output Configuration

Screenshots are persisted to `public/screenshots/` by default, or a custom `outputPath` if specified. The function supports:
- **Format selection**: `png` or `jpeg` with adjustable quality settings
- **Full-page capture**: `fullPage: true` captures vertically scrolling content beyond the initial viewport dimensions

### Contextual Metadata

The function returns a structured result object containing:
- **`buffer`**: Raw image data for immediate processing or transmission
- **`outputPath`**: Filesystem location of the saved asset
- **`metadata`**: Title, final URL, viewport dimensions, and capture options used

This metadata enables downstream services to correlate visual snapshots with specific application states and conversation contexts.

## Batch Processing and PDF Generation

For scenarios requiring multiple captures, `takeMultipleScreenshots` iterates over an array of URLs, reusing the core logic while respecting an optional `delay` parameter between requests to prevent rate limiting.

The utility also exposes `generatePDF`, which repurposes the same Playwright workflow to export pages as PDF documents, maintaining identical authentication and navigation strategies for consistency across output formats.

## Integration with Lemon AI Services

The screenshot utility operates as a foundational service layer, invoked by both API endpoints and frontend components.

### API Endpoints

The Express router in [`src/routers/conversation/conversation.js`](https://github.com/hexdocom/lemonai/blob/main/src/routers/conversation/conversation.js) exposes `POST /screenshots/single` and batch routes. These endpoints parse request bodies containing target URLs and conversation IDs, then delegate execution to `takeScreenshot` or `takeMultipleScreenshots`, returning JSON responses with screenshot URLs and metadata.

### Frontend Rendering

Captured screenshots surface within the chat interface via [`frontend/src/view/lemon/components/ChatMessages.vue`](https://github.com/hexdocom/lemonai/blob/main/frontend/src/view/lemon/components/ChatMessages.vue). When a message metadata object contains a `screenshot` property, the component renders the image inline, providing users with visual context for AI-generated responses. The Vuex store module in [`frontend/src/store/modules/chat.js`](https://github.com/hexdocom/lemonai/blob/main/frontend/src/store/modules/chat.js) persists these screenshot URLs within message state, ensuring visual snapshots remain available across session reloads.

## Summary

- Lemon AI utilizes **Playwright-driven headless Chromium** to generate deterministic UI snapshots with consistent viewport rendering.
- **Authentication tokens** are pre-injected via `localStorage` to capture protected application states without manual login.
- **Resilient navigation** employs cascading wait strategies (`domcontentloaded`, `load`, `networkidle`) and post-load stabilization to ensure complete rendering.
- Screenshots are persisted to `public/screenshots/` with configurable formats, full-page options, and comprehensive metadata including buffers and file paths.
- The utility integrates with Express API routes and Vue.js frontend components to embed visual context within conversation flows.

## Frequently Asked Questions

### What browser engine powers Lemon AI's screenshot utility?

The utility leverages **Chromium** in headless mode, orchestrated through the **Playwright** library. This combination provides a consistent, modern rendering engine that supports complex CSS, JavaScript-driven SPAs, and WebGL content across all capture sessions.

### How does the utility handle authentication for private pages?

Before navigating to the target URL, the utility checks for an `accessToken` parameter. If present, it executes `localStorage.setItem('access_token', token)` within the browser page context. This injection authenticates the session immediately, allowing the screenshot to capture fully-rendered protected dashboards without encountering login screens.

### What mechanisms ensure the page is fully loaded before capture?

The implementation uses a **cascading wait strategy** that attempts three conditions in sequence: `domcontentloaded` for initial HTML parsing, `load` for resource completion, and `networkidle` for network quiescence. If one fails, the next is attempted. Additionally, a configurable `waitTime` (default 2000 ms) provides post-load stabilization for JavaScript-rendered content.

### Where are screenshot files stored and how are they accessed by the frontend?

By default, images are written to the `public/screenshots/` directory on the server filesystem. The `takeScreenshot` function returns an `outputPath` and public URL that the API layer forwards to the frontend. Components like [`ChatMessages.vue`](https://github.com/hexdocom/lemonai/blob/main/ChatMessages.vue) reference these URLs to render inline images within chat bubbles, while the Vuex store persists the references in conversation state for cross-session availability.