Rate Limiting Considerations for Bulk Operations in notebooklm-py

When performing bulk operations in notebooklm-py, implement exponential backoff using the built-in --retry CLI flag or catch RateLimitError exceptions to handle Google's strict batchexecute RPC quotas.

The notebooklm-py library interfaces with Google's undocumented batchexecute RPC endpoint, which enforces aggressive request quotas that can quickly throttle bulk workflows. Whether you are importing dozens of sources or generating multiple artifacts, understanding the three-layer rate limiting mitigation strategy is essential for building reliable automation.

Core Rate Limiting Architecture

The library implements defense-in-depth against HTTP 429 "Too Many Requests" responses through distinct layers that handle detection, CLI configuration, and generation-specific retry logic.

The RPC Detection Layer (_core.py)

At the network boundary, src/notebooklm/_core.py (lines 52-66) inspects every RPC response for status code 429. When detected, the code extracts the optional Retry-After header and raises a dedicated RateLimitError that carries the recommended retry interval in its retry_after attribute. This ensures that calling code receives structured exception data rather than raw HTTP errors.

CLI Retry Configuration (options.py)

For command-line workflows, src/notebooklm/cli/options.py (lines 83-90) introduces a global --retry N flag. This argument injects retry configuration into the CLI context, triggering automatic exponential backoff for all supported generation commands without requiring manual exception handling in shell scripts.

Generation Retry Wrappers (generate.py)

The artifact generation pipeline centralizes its resilience logic in src/notebooklm/cli/generate.py. A dedicated back-off calculator (lines 61-74) computes sleep intervals, while the generate_with_retry coroutine (lines 77-108) implements the actual retry loop. This wrapper respects the max_retries value, prints progress notices during backoff periods, and returns the final GenerationStatus only after successful completion or exhaustion of retry attempts.

Impact of Rate Limits on Bulk Workflows

Different bulk operation patterns trigger distinct failure modes that require specific handling strategies:

• Bulk source import (client.sources.add_* in rapid succession): Typically fails with a 429 after several rapid calls. The _core.rpc_call raises RateLimitError, which propagates to the caller. When using the example docs/examples/bulk-import.py script, you must catch this exception manually or the script aborts.
• Batch artifact generation (sequential calls to client.artifacts.generate_*): Fails mid-stream during the RPC. CLI generate commands mitigate this automatically when --retry is supplied, with the internal wrapper managing the backoff delays.
• Mixed operations (notebook creation, source addition, and artifact generation): Produces interleaved 429 errors alongside potential authentication failures. The core layer retries 401/403 errors once after token refresh before surfacing rate limit exceptions.

Implementation Strategies for Bulk Operations

Using the Built-in CLI Retry Mechanism

For shell-based automation, append the --retry flag to any generation command to enable automatic exponential backoff:

# Generate 5 videos with up to 4 retry attempts on rate limits

notebooklm generate video -n NB123 --retry 4

This delegates all exception handling to the library's internal retry wrapper, making it the simplest approach for non-interactive bulk jobs.

Handling Rate Limits in Python Scripts

When building custom automation with the Python API, wrap bulk calls in a manual retry loop that honors the retry_after attribute:

import asyncio
from notebooklm import NotebookLMClient, RateLimitError

async def bulk_add_sources(nb_id, urls):
    async with await NotebookLMClient.from_storage() as client:
        for url in urls:
            while True:
                try:
                    await client.sources.add_url(nb_id, url)
                    print(f"✔ Added {url}")
                    break
                except RateLimitError as e:
                    wait = e.retry_after or 5
                    print(f"⏳ Rate limited, waiting {wait}s …")
                    await asyncio.sleep(wait)

# Example usage:

# asyncio.run(bulk_add_sources("nb_123", ["https://example.com/1", "..."]))

This pattern provides granular control over logging and intermediate state management between retry attempts.

Leveraging the Internal Retry Wrapper

Advanced use cases can import the internal generate_with_retry helper directly from the CLI module to apply consistent retry semantics to custom generation logic:

from notebooklm.cli.generate import generate_with_retry
from notebooklm.client import NotebookLMClient

async def generate_audio_with_retry(nb_id, max_retries=3):
    async with await NotebookLMClient.from_storage() as client:
        async def _gen():
            return await client.artifacts.generate_audio(nb_id)
        
        status = await generate_with_retry(_gen, max_retries, "audio")
        return status

This approach reuses the library's exponential backoff calculator while allowing you to define custom generation coroutines.

Best Practices for Avoiding 429 Errors

• Prefer the --retry flag for all CLI bulk commands to ensure consistent backoff behavior.
• Honor the Retry-After header by checking e.retry_after on RateLimitError instances rather than using hardcoded delays.
• Insert micro-delays between independent API calls (e.g., await asyncio.sleep(0.2) between source additions) to reduce the probability of triggering quota thresholds.
• Catch RateLimitError explicitly in production scripts to prevent bulk workflows from terminating on transient throttling events.

Summary

• Detection: src/notebooklm/_core.py converts HTTP 429 responses into structured RateLimitError exceptions with timing metadata.
• CLI: The --retry flag in src/notebooklm/cli/options.py enables automatic exponential backoff for generation commands.
• Wrappers: src/notebooklm/cli/generate.py provides reusable generate_with_retry logic for artifact creation workflows.
• API: Manual retry loops must check RateLimitError.retry_after and implement asyncio.sleep intervals to respect Google's quotas.

Frequently Asked Questions

What happens when I hit a rate limit in notebooklm-py?

The library raises a RateLimitError (defined in src/notebooklm/exceptions.py) that includes the retry_after value parsed from the HTTP response headers. If uncaught, this terminates the current operation; when using the CLI with --retry, the library automatically sleeps and retries according to exponential backoff rules.

How do I configure automatic retries for bulk imports?

The CLI does not automatically retry source import operations. You must either wrap your Python script's client.sources.add_* calls in a try/except block that catches RateLimitError, or modify the docs/examples/bulk-import.py example to implement a backoff loop using the retry_after attribute.

Can I use the CLI retry logic in my own Python scripts?

Yes. Import generate_with_retry from src/notebooklm/cli/generate.py to reuse the same exponential backoff logic that powers the CLI commands. This function accepts a coroutine, a max retry count, and an operation name string, returning the final status after all retries are exhausted or the operation succeeds.

Where is the RateLimitError defined?

The exception class is defined in src/notebooklm/exceptions.py. It extends the base notebooklm exception and stores the integer seconds value from the Retry-After header in its retry_after property, allowing calling code to implement compliant backoff delays.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}