How to Test Changes in Open-SEO: Unit vs E2E Testing Strategy

Testing changes in Open-SEO requires running isolated unit tests with Vitest for individual functions and MCP tools, alongside full-stack end-to-end tests with Playwright that validate the complete application stack including the frontend, Cloudflare Worker, and external API integrations.

Open-SEO maintains a dual-layer testing architecture that balances development velocity with deployment confidence. When you test changes in Open-SEO, you are working with a repository that co-locates fast unit tests beside production code and comprehensive browser automation tests that exercise the entire self-hosted or cloud deployment architecture. The test suite covers everything from individual DataForSEO API wrappers to complete keyword research UI workflows.

Understanding Open-SEO's Testing Architecture

The repository splits validation into two distinct categories based on scope and execution speed.

Unit tests reside next to the implementation files as *.test.ts modules and target individual functions, services, and MCP tool handlers. These tests use Vitest's mocking utilities to replace external dependencies such as DataForSEO, Google Search Console, and Cloudflare Workers, ensuring rapid feedback during development.

End-to-end tests leverage Playwright to execute against a live application stack. The configuration in playwright.config.ts automatically spins up the Vite dev server and validates realistic user flows across the frontend, API layer, and database interactions.

Running the Test Suite from the Command Line

The package.json file defines explicit scripts that orchestrate both testing modes:

{
  "scripts": {
    "test": "vitest run",
    "test:watch": "vitest",
    "test:ci": "vitest run --reporter=dot",
    "test:e2e": "playwright test",
    "ci:check": "run-s lint typecheck test"
  }
}

• pnpm test – Executes all unit tests once using Vitest.
• pnpm test:watch – Starts Vitest in watch mode for continuous feedback during coding.
• pnpm test:ci – Runs unit tests with minimal output formatting, optimized for continuous integration pipelines.
• pnpm test:e2e – Launches Playwright and executes the full browser automation suite.

The ci:check script combines linting, type-checking, and test execution, ensuring that every change passes the same quality gates enforced in the hosted CI pipeline.

Unit Testing with Vitest

Unit tests verify logic in isolation without spinning up the full application stack.

Test File Organization and Naming Conventions

Test files use the *.test.ts naming pattern and live adjacent to the code they validate. For example, the Search Console MCP tool tests reside in src/server/mcp/tools/search-console-tools.test.ts, directly alongside the implementation. This proximity ensures that developers discover relevant tests immediately when modifying functionality.

Mocking External Dependencies

Open-SEO relies on several external services that must be mocked during unit testing. The test suite uses Vitest's vi.mock() to replace module implementations. In src/server/mcp/tools/search-console-tools.test.ts, the tests mock GscService to validate request handling, error paths, and self-hosted mode behavior without authenticating against real Google APIs.

// Example mocking pattern for GSC dependencies
vi.mock("@/server/features/gsc/services/GscService", () => ({
  GscService: { 
    fetchSearchAnalytics: vi.fn().mockResolvedValue({ rows: [] }) 
  }
}));

This approach applies equally to DataForSEO API clients, Cloudflare Workers bindings, and database connections, ensuring tests remain fast and deterministic.

MCP Tool Testing Utilities

The repository includes a specialized helper module at src/server/mcp/tools/tool-test-support.ts that exports makeToolContext(). This utility constructs a mock ToolContext object required by all MCP tool handlers, eliminating boilerplate when writing new tests.

// src/server/mcp/tools/my-new-tool.test.ts
import { describe, it, expect, vi } from "vitest";
import { myNewTool } from "./my-new-tool";
import { makeToolContext } from "./tool-test-support";

describe("my new MCP tool", () => {
  const ctx = makeToolContext();

  it("returns formatted output on success", async () => {
    const result = await myNewTool.handler({ projectId: "proj_1" }, ctx);
    expect(result.structuredContent).toMatchObject({ ok: true });
  });
});

End-to-End Testing with Playwright

E2E tests validate the complete system behavior by automating browser interactions against a running instance of the application.

Configuration and Test Location

The playwright.config.ts file defines the test environment, automatically launching the Vite dev server via vite dev before test execution. Test specifications live in the e2e/ directory at the project root. For example, e2e/keyword-research-navigation.spec.ts drives the keyword research UI and validates that user interactions trigger the correct API calls and render expected results.

Running Specific E2E Tests

You can execute the entire Playwright suite or target individual specifications:

# Run all E2E tests

pnpm test:e2e

# Run a single specification file

pnpm test:e2e e2e/keyword-research-navigation.spec.ts

# Run tests matching a specific pattern

pnpm test:e2e --grep "domain overview"

Local Development Workflow for Testing Changes

Follow this sequence to validate modifications locally before submitting changes:

1. Start the development server – Run pnpm dev to launch Vite and the Cloudflare Worker in local mode. While optional for unit tests, this is required for E2E validation.

2. Execute unit tests – Run pnpm test to verify business logic in isolation. Vitest automatically discovers new *.test.ts files throughout the src/ directory.

3. Validate full-stack behavior – Execute pnpm test:e2e to confirm that UI components correctly interact with backend services and external API mocks.

4. Continuous verification – Use pnpm test:watch during active development to receive immediate feedback on code changes.

When testing changes that touch external APIs, add fixtures for expected request and response shapes within the relevant test file, following the existing patterns in src/server/mcp/tools/search-console-tools.test.ts.

Summary

• Open-SEO uses Vitest for fast unit tests located in *.test.ts files beside source code, with mocks for external services like DataForSEO and Google Search Console.
• Playwright powers E2E tests in the e2e/ folder, configured in playwright.config.ts to automatically spin up the dev server.
• Package scripts provide simple commands: pnpm test for unit tests, pnpm test:e2e for browser automation, and pnpm ci:check for full validation.
• tool-test-support.ts supplies makeToolContext() to streamline MCP tool unit testing.
• Local validation requires running the dev server (pnpm dev) only for E2E tests, while unit tests execute independently in milliseconds.

Frequently Asked Questions

How do I run only the unit tests for a specific file in Open-SEO?

Pass the file path directly to the test command: pnpm test src/server/mcp/tools/search-console-tools.test.ts. Vitest will execute only the tests defined in that specific module, ignoring the rest of the suite for rapid feedback during targeted debugging.

Can I run a single Playwright test instead of the entire E2E suite?

Yes, append the specific spec file path to the test command: pnpm test:e2e e2e/domain-overview-filters.spec.ts. You can also use the --grep flag to filter tests by name or tag, allowing you to validate specific user flows without executing the complete browser automation suite.

How do I mock external APIs when testing MCP tools?

Use vi.mock() at the top of your test file to replace the module exporting the external service. For Google Search Console interactions, mock @/server/features/gsc/services/GscService. For DataForSEO endpoints, mock the respective API client modules. Provide mock implementations that return resolved promises with fixture data matching the expected response shapes.

What is the difference between pnpm test and pnpm test:ci?

pnpm test runs Vitest with the default reporter, displaying detailed test names and failure diagnostics suitable for local development. pnpm test:ci executes the same test suite with the --reporter=dot flag, producing minimal output (dots for passing tests) optimized for CI log readability and performance. Both commands exit with a non-zero status if any assertions fail.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}