How to Monitor Network Traffic and API Responses in Zendriver

Zendriver enables network monitoring by tapping into Chromium DevTools Protocol (CDP) events, allowing you to register asynchronous handlers for RequestWillBeSent and ResponseReceived or use high-level expectations to capture specific API calls.

Zendriver provides deep visibility into browser network activity through its CDP integration, making it straightforward to monitor network traffic and API responses programmatically. By leveraging the network domain defined in zendriver/cdp/network.py, you can intercept every HTTP request and response, inspect headers, capture response bodies, and filter traffic by resource type—all without external proxies.

Understanding the CDP Network Architecture in Zendriver

Network Domain Definitions

The foundation of network monitoring lies in zendriver/cdp/network.py, which defines CDP events like RequestWillBeSent, ResponseReceived, and LoadingFinished. These events expose full request and response objects containing fields such as request.url, request.method, response.status, and response.headers.

Event Handler Registration

The Tab class in zendriver/core/tab.py provides the add_handler(event_type, callback) method. This delegates to Connection.add_handler in zendriver/core/connection.py, maintaining a registry of asynchronous callbacks that receive CDP event dataclasses in real time.

Real-Time Network Monitoring with Handlers

To monitor all traffic continuously, register handlers before navigation. The callbacks must accept the specific CDP event type as a parameter.

import asyncio
from zendriver import cdp, start, loop

async def main() -> None:
    browser = await start()
    tab = browser.main_tab

    # Register low-level handlers

    tab.add_handler(cdp.network.RequestWillBeSent, send_handler)
    tab.add_handler(cdp.network.ResponseReceived, receive_handler)

    await tab.get("https://www.google.com")
    await tab.sleep(5)

async def send_handler(event: cdp.network.RequestWillBeSent) -> None:
    req = event.request
    print(f"➡ {req.method} {req.url}")
    for k, v in req.headers.items():
        print(f"   {k}: {v}")

async def receive_handler(event: cdp.network.ResponseReceived) -> None:
    resp = event.response
    print(f"⬅ {resp.status} {resp.url}")

if __name__ == "__main__":
    loop().run_until_complete(main())

In zendriver/core/tab.py, the add_handler method maps CDP events to your coroutine functions. Because handlers run asynchronously, they process data without blocking the main script execution.

Capturing Specific API Responses with Expectations

For one-off checks, use the expectation helpers in zendriver/core/expect.py. The expect_response method returns a context manager that waits for a URL pattern match, automatically removing handlers after resolution.

import asyncio
from zendriver import cdp, start

async def main():
    browser = await start()
    tab = browser.main_tab

    # Use high-level expectation to wait for a JSON API request

    async with tab.expect_response(r"https://api.example.com/v1/data") as exp:
        await tab.get("https://example.com")
        response = await exp.response          # cdp.network.Response

        body, is_base64 = await exp.response_body  # raw payload

    print("API status:", response.status)
    print("Payload:", body)

asyncio.run(main())

The BaseRequestExpectation class handles the underlying ResponseReceived event and provides the response_body property, which internally calls cdp.network.get_response_body to fetch the payload.

Filtering Traffic by Resource Type

You can inspect event.request.resource_type to filter for specific request types. The ResourceType enum is defined in zendriver/cdp/network.py.

import re
from zendriver import cdp, start

async def main():
    browser = await start()
    tab = browser.main_tab

    # Only monitor XHR / fetch calls

    def only_xhr(event):
        return event.request.resource_type == cdp.network.ResourceType.XHR

    async def xhr_handler(event):
        if only_xhr(event):
            print("XHR →", event.request.url)

    tab.add_handler(cdp.network.RequestWillBeSent, xhr_handler)
    await tab.get("https://example.com")
    await tab.sleep(3)

asyncio.run(main())

This approach allows you to isolate API calls from image loads, CSS requests, and other browser traffic.

Summary

• Register handlers using tab.add_handler() with CDP events like RequestWillBeSent to monitor network traffic and API responses in real time.
• Inspect response data by handling ResponseReceived events, which provide access to status codes, headers, and URLs via event.response.
• Use expectations from zendriver/core/expect.py for one-shot capture of specific endpoints without manual handler management.
• Filter requests by checking event.request.resource_type against the cdp.network.ResourceType enum.
• Access response bodies through expect_response().response_body or manually via cdp.network.get_response_body.

Frequently Asked Questions

How do I capture response bodies when monitoring network traffic?

Use the expect_response context manager and await exp.response_body, which returns a tuple of (body, is_base64). Alternatively, manually call cdp.network.get_response_body with the request ID from the ResponseReceived event, though the expectation method handles cleanup automatically.

What CDP events are available for network monitoring in Zendriver?

The zendriver/cdp/network.py file defines key events including RequestWillBeSent (fired when a request is about to be sent), ResponseReceived (fired when the HTTP response status and headers are received), and LoadingFinished (fired when the body download completes). Each event provides a dataclass with full request or response metadata.

Can I monitor WebSocket traffic with Zendriver?

While the provided examples focus on HTTP requests, the CDP network domain in zendriver/cdp/network.py also defines WebSocket events such as WebSocketCreated and WebSocketFrameReceived. You can register handlers for these events using the same tab.add_handler() pattern to monitor WebSocket traffic alongside standard API calls.

How do I remove network handlers after registration?

Handlers registered via tab.add_handler() are stored in the connection's registry (zendriver/core/connection.py). When using expectations, handlers are automatically removed when the context manager exits. For manual cleanup, you would need to track the handler references and call the connection's removal methods, though the library typically manages this lifecycle for you.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}