How to Write Direct Mode Tests with the mock_llm Fixture in GenLayer

Use the direct_vm.mock_llm(regex, reply) helper to intercept gl.nondet.exec_prompt calls and return deterministic JSON responses, enabling fast, deterministic unit tests for GenLayer contracts without external LLM dependencies.

Direct mode tests in the genlayer-project-boilerplate repository execute GenLayer contracts in-memory using the direct_vm fixture, eliminating the need for a live Studio deployment. The mock_llm fixture method allows you to simulate LLM responses by pattern-matching regex against prompts and injecting controlled return values, ensuring repeatable test runs for contracts that rely on nondeterministic AI inference.

Understanding the Direct VM Mocking Architecture

The direct_vm fixture provides two primary mocking helpers that replace the underlying gl.nondet implementation with deterministic stubs scoped to the current test:

• direct_vm.mock_llm(regex, reply) — Intercepts calls to gl.nondet.exec_prompt where the prompt matches the supplied regular expression. The reply parameter accepts either a JSON string or plain text that the contract receives as the LLM response.
• direct_vm.mock_web(regex, response) — Intercepts gl.nondet.web.* calls by matching URL patterns against the regex and returning an HTTP-like dictionary with status and body keys.

Both helpers register expectations that persist for the duration of the test function. Call direct_vm.clear_mocks() to reset the registry and define fresh expectations for subsequent contract interactions.

Step-by-Step: Writing Direct Mode Tests with mock_llm

Follow this architectural flow to implement reliable direct mode tests with mock LLM fixtures:

1. Deploy the contract using direct_deploy("<path/to/contract.py>"), which loads the contract into the direct VM and returns a proxy object.
2. Set the transaction sender by assigning direct_vm.sender = <address> or use a pre-configured fixture such as direct_alice.
3. Configure mocks by invoking mock_llm with regex patterns that match your contract's prompt templates before calling the method that triggers gl.nondet.exec_prompt.
4. Execute the contract method, which automatically receives the mocked response instead of calling the real LLM service.
5. Assert results by verifying contract state changes or return values against the injected mock data.

Examples from the GenLayer Boilerplate

Basic LLM Mocking in test_patterns.py

For simple validator patterns, set a catch-all regex to return a consistent JSON structure. In [tests/direct/test_patterns.py at lines 10–13](https://github.com/genlayerlabs/genlayer-project-boilerplate/blob/main/tests/direct/test_patterns.py#L10-L13), the test configures a single mock before deployment:

direct_vm.mock_llm(r".*", '{"winner": 1, "score": "2:1"}')
contract = direct_deploy(CONTRACT_PATH)
result = contract.get_match_result(True)

This pattern works because the regex r".*" matches any prompt sent to gl.nondet.exec_prompt, ensuring the contract receives the JSON object {"winner": 1, "score": "2:1"} regardless of the actual query content.

Multi-Step Resolution in test_views.py

Complex workflows requiring different LLM responses within a single test require clearing mocks between steps. In [tests/direct/test_views.py at lines 38–52](https://github.com/genlayerlabs/genlayer-project-boilerplate/blob/main/tests/direct/test_views.py#L38-L52), the test resolves two sequential bets by updating the mock:

# First match: Mock web and LLM for initial resolution

direct_vm.mock_web(
    r".*bbc\.com/sport/football/scores-fixtures.*",
    {"status": 200, "body": "Match results available."},
)
direct_vm.mock_llm(
    r".*Extract the match result.*",
    json.dumps({"score": "1:0", "winner": 1}),
)
contract.resolve_bet("2024-06-20_spain_italy")

# Reset mocks for second match

direct_vm.clear_mocks()
direct_vm.mock_web(r".*bbc\.com/sport/football/scores-fixtures.*",
    {"status": 200, "body": "Updated results."})
direct_vm.mock_llm(r".*Extract the match result.*",
    json.dumps({"score": "1:1", "winner": 0}),
)
contract.resolve_bet("2024-06-20_denmark_england")

The clear_mocks() call ensures the first set of expectations does not interfere with the second resolution, preventing regex collision when both prompts match similar patterns.

Combining Web Scraping and LLM Parsing

When contracts fetch external data before LLM processing, chain both mock types. The direct VM resolves mock_web for the HTTP request and mock_llm for the subsequent data extraction:

def test_price_oracle(direct_vm, direct_deploy, direct_bob):
    contract = direct_deploy("contracts/price_oracle.py")
    direct_vm.sender = direct_bob
    
    # Simulateprice API response

    direct_vm.mock_web(
        r".*example\.com/price.*",
        {"status": 200, "body": '{"price": 123.45}'}
    )
    
    # Simulate LLM extraction logic

    direct_vm.mock_llm(
        r".*Parse the JSON price.*",
        json.dumps({"price": 123.45})
    )
    
    price = contract.fetch_price()
    assert price == 123.45

Summary

• Direct mode tests run contracts in-memory via the direct_vm fixture, eliminating network latency and external dependencies.
• mock_llm(regex, reply) intercepts gl.nondet.exec_prompt calls by regex-matching the prompt text and returning the specified JSON or string reply.
• Scoped isolation ensures mocks apply only to the current test; use clear_mocks() to reset state between assertions or scenarios.
• Real-world implementations appear in tests/direct/test_views.py (multi-step workflows) and tests/direct/test_patterns.py (simple validations).
• The to_hex address helper used in assertions is defined in [tests/direct/conftest.py](https://github.com/genlayerlabs/genlayer-project-boilerplate/blob/main/tests/direct/conftest.py).

Frequently Asked Questions

How do I match specific prompt content with mock_llm?

Use Python raw strings with regex patterns that target unique substrings in your contract's prompts. For example, r".*Extract the match result.*" matches any prompt containing that phrase. The direct VM selects the first registered mock where the regex matches, so register specific patterns before general ones.

Can I use mock_llm for multiple different responses in one test?

Yes, but you must call direct_vm.clear_mocks() between configurations to prevent pattern overlap. After clearing, register new mock_llm calls with updated regex patterns and JSON payloads for subsequent contract invocations, as demonstrated in the multi-resolution test at test_views.py lines 38–52.

What format should the mock_llm reply parameter use?

The reply parameter accepts either a JSON-serialized string (e.g., json.dumps({"key": "value"})) or a plain string. The contract receives this value directly as if it were the LLM's raw output, so structure it to match your contract's parsing logic for gl.nondet.exec_prompt responses.

Does mock_llm affect all tests or just the current function?

The mock_llm helper is scoped to the individual test function where it is called. The direct VM automatically clears mocks after each test completes, but you can manually reset expectations within a test using direct_vm.clear_mocks() to isolate different phases of a multi-step contract interaction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}