What Are the Eight Stages of the MemoryProxy Request Pipeline?

The MemoryProxy request pipeline processes every LLM request through eight sequential stages—authentication, system user detection, session initialization, context injection, rate limiting, request forwarding, memory extraction, and telemetry reporting—to securely enrich and proxy traffic between coding agents and upstream language models.

The TencentDB-Agent-Memory repository implements a transparent proxy layer that intercepts requests between coding agents like Claude Code and large language models. Every request carrying a spaceId (the memory instance identifier) traverses the MemoryProxy request pipeline, a fixed processing chain implemented in TypeScript that handles everything from API key validation to credit-based billing telemetry.

The Eight-Stage Pipeline Architecture

The pipeline executes in strict order for every request that includes a spaceId. Stages 1–5 are ops-level operations that prepare and validate the request before it reaches the upstream LLM, while stages 6–8 handle the actual proxying, persistence, and observability.

1. Authentication (src/auth.ts)

The first stage validates the x-tdai-user-key header present in incoming requests. Located in src/auth.ts, this module resolves the end-user identity and verifies that the request has permission to act on the specified memory space. If authentication fails, the pipeline terminates immediately with an authorization error.

2. System User Detection (src/systemUser.ts)

Implemented in src/systemUser.ts, this stage detects internal service accounts—such as the Memory Bridge—and short-circuits the pipeline. When a system user is identified, the proxy bypasses session initialization and context injection, routing the request directly to the upstream LLM to reduce latency for internal traffic.

3. Session Initialization (src/session/index.ts)

For first-turn conversations, the sessionInit stage presents an interactive form requiring the user to select team → agent → task. The chosen context is stored in the session and later injected into the system prompt. This stage is handled by src/session/index.ts, which manages the state machine for multi-step form completion.

4. Context Injection (src/injection/pipeline.ts)

Before forwarding to the LLM, the injection stage pulls relevant Skill, Knowledge, and Memory assets from Memory Core and enriches the system prompt. Located in src/injection/pipeline.ts, this module either appends context directly to the prompt or "tool-izes" assets for function-calling models, ensuring the LLM has access to relevant institutional knowledge.

5. Rate Limiting (src/rate-limit/limiter.ts)

The rateLimit stage enforces per-instance and per-token-model limits using Redis-based sliding-window counters. Implemented in src/rate-limit/limiter.ts, this stage validates TPM (tokens per minute) and QPM (queries per minute) quotas, returning 429 errors when limits are exceeded, protecting both the proxy and upstream LLM providers from traffic spikes.

6. Request Forwarding (src/handler.ts)

Stage six proxies the enriched request to the upstream LLM—whether OpenAI-compatible or Anthropic endpoints. The src/handler.ts file orchestrates this forward stage, managing HTTP client connections, streaming responses, and error handling for the actual LLM inference.

7. Memory Extraction (src/tdai/tdai-memory.ts)

After receiving the LLM response, the extract stage asynchronously writes the conversation turn back to Memory Core. Located in src/tdai/tdai-memory.ts, this module adds the dialogue slice to L0 short-term memory and triggers background Skill extraction jobs, ensuring the system learns from each interaction without blocking the response to the client.

8. Telemetry Reporting (src/report/report.ts)

The final stage emits usage telemetry to ClickHouse, Langfuse, and Opik observability platforms while updating credit-billing counters. Implemented in src/report/report.ts, this stage ensures accurate cost attribution and system monitoring before the request lifecycle completes.

Pipeline Execution and Tracing

The following examples demonstrate how requests flow through the eight stages in production environments.

Basic OpenAI-Compatible Request

{
  "apiKey": "sk-mem-xyz",
  "url": "http://localhost:8096/proxy/mem-example001/v1/chat/completions",
  "model": "gpt-4o",
  "messages": [
    { "role": "user", "content": "Explain the architecture of MemoryProxy." }
  ]
}

When processed, this request generates the following high-level trace through the pipeline:

auth → systemUser → sessionInit → injection → rateLimit → forward → extract → report

Debugging Pipeline Stages via Logging

Enable debug logging to verify each stage execution:

import { Logger } from "./src/report/logger.ts";

logger.info("pipeline.start", { requestId: "req-123" });
logger.debug("pipeline.auth", { requestId: "req-123", userKey: "abc…" });
// Continues through all eight stages...
logger.debug("pipeline.report", { requestId: "req-123", creditDelta: 12 });

The log output contains discrete entries for each stage, allowing operators to verify that requests completed the full MemoryProxy request pipeline.

Programmatic Session Initialization

When a request lacks a stored session, the proxy returns a form response. Complete the sessionInit stage programmatically:

await fetch("http://localhost:8096/v3/session/refresh-cache", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    session_key: "sess_1",
    agent_source: "claude-code",
    space_id: "mem-example001"
  })
});

The response includes a stage field ("team", "agent_select", etc.) indicating the next form step to render.

Configuring Rate Limits

Modify stage 5 limits through the admin API:

await fetch("http://localhost:8096/v3/admin/rate-limits", {
  method: "PUT",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ input_tpm: 50000, qpm: 200 })
});

Subsequent requests hitting the Redis-based counters in src/rate-limit/limiter.ts will respect these updated quotas.

Key Implementation Files

File	Pipeline Role
src/index.ts	Entry point that wires the HTTP server and registers all route handlers
src/handler.ts	Implements the core request flow and invokes the eight pipeline stages
src/auth.ts	Stage 1: User-key validation against Memory Core
src/systemUser.ts	Stage 2: Internal service-account shortcut detection
src/session/index.ts	Stage 3: Session initialization and form handling
src/injection/pipeline.ts	Stage 4: Asset injection into system prompts
src/rate-limit/limiter.ts	Stage 5: TPM/QPM rate-limiting logic
src/tdai/tdai-memory.ts	Stage 7: Async conversation write-back and L0 memory storage
src/report/report.ts	Stage 8: Telemetry, credit-billing, and usage reporting

Summary

• The MemoryProxy request pipeline consists of eight sequential stages executed for every spaceId-bearing request.
• Stages 1–5 (authentication through rate limiting) are ops-level validations that occur before contacting the upstream LLM.
• Stage 6 forwards the enriched request to OpenAI-compatible or Anthropic endpoints via src/handler.ts.
• Stages 7–8 handle asynchronous memory extraction and telemetry reporting without blocking the client response.
• Each stage has a dedicated implementation file in the src/ directory, with src/auth.ts handling validation and src/report/report.ts closing the lifecycle with billing data.

Frequently Asked Questions

What happens if the authentication stage fails?

If the x-tdai-user-key header is missing or invalid, the auth stage in src/auth.ts terminates the pipeline immediately with an authorization error. The request never reaches session initialization, context injection, or the upstream LLM, ensuring that only validated users consume resources.

How does the pipeline handle internal service accounts?

The systemUser stage detects internal accounts—such as the Memory Bridge—and short-circuits the pipeline by bypassing sessionInit and injection. This optimization reduces latency for trusted internal traffic while still enforcing authentication and rate limiting.

Can I skip the session initialization form for automated agents?

No, the sessionInit stage is mandatory for first-turn conversations within a spaceId. However, once initialized, session context is cached in Memory Core. Automated agents can complete this stage programmatically using the /v3/session/refresh-cache endpoint to select team, agent, and task contexts without human interaction.

Where does rate limiting occur in the pipeline?

Rate limiting occurs at stage 5, implemented in src/rate-limit/limiter.ts, using Redis-based sliding-window counters. This positioning ensures that expensive operations—such as context injection and LLM forwarding—only execute for requests within the configured TPM (tokens per minute) and QPM (queries per minute) quotas.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}