Security Measures in Local-Deep-Research: SSRF Protection, URL Validation, and File Integrity

Local-Deep-Research implements a defense-in-depth security model combining SSRF protection via IP-range blacklisting, strict URL validation against malicious schemes and open redirects, and cryptographic file integrity verification using SHA-256 whitelisting.

The learningcircuit/local-deep-research repository includes a self-contained security layer designed to mitigate common web application vulnerabilities. This article examines the three core defensive mechanisms—SSRF protection, URL validation, and file integrity verification—as implemented in the codebase's dedicated security modules.

SSRF Protection via IP-Range Blacklisting

The SSRF defense logic resides in src/local_deep_research/security/ssrf_validator.py. The primary entry point, validate_url(), parses incoming URLs and enforces a strict http/https scheme policy before resolving hostnames and checking resolved IPs.

Blocking Private Networks and Cloud Metadata

After resolution, every IP address passes through is_ip_blocked(), which rejects traffic destined for:

• Loopback addresses (127.0.0.0/8, ::1)
• RFC 1918 private networks (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16)
• Carrier-Grade NAT (CGNAT) and link-local ranges
• IPv6 Unique Local Addresses (ULA)
• The AWS metadata endpoint 169.254.169.254

These ranges are defined in src/local_deep_research/security/ip_ranges.py, providing a centralized blocklist consumed by the validator. For scenarios requiring local service access, validate_url() accepts an allow_localhost=True parameter to explicitly permit loopback connections.

Test Environment Exceptions

During automated testing, the validator checks for the PYTEST_CURRENT_TEST environment variable. When present, validation is bypassed only for the duration of test execution, allowing local mock servers without weakening production security.

from local_deep_research.security.ssrf_validator import validate_url

# Accepted – external HTTPS site

assert validate_url("https://example.com") is True

# Rejected – loopback address (unless explicitly allowed)

assert validate_url("http://127.0.0.1") is False

# Allowed when we trust a local service (e.g., a self-hosted search engine)

assert validate_url(
    "http://127.0.0.1:8080",
    allow_localhost=True
) is True

URL Validation and Malicious Scheme Prevention

General URL sanitization is handled by src/local_deep_research/security/url_validator.py through the URLValidator class. This module protects against XSS, open redirects, and protocol abuse before any external fetch or redirect operation proceeds.

Dangerous Scheme Blocking

The method is_unsafe_scheme() explicitly blocks malicious protocols including javascript:, data:, vbscript:, about:, blob:, and file:. Conversely, is_safe_url() maintains a whitelist of acceptable schemes—http, https, ftp, and ftps—with optional support for mailto and configurable trusted-domain restrictions. The validator also scans for suspicious patterns such as double-encoding, null-bytes, Unicode escapes, and HTML entities.

Injection and Redirect Hardening

For redirect validation, is_safe_redirect_url() prevents:

• Open redirects to external domains
• CRLF injection attacks
• Protocol-relative URLs (//evil.com)
• Back-slash trickery and path-traversal sequences

from local_deep_research.security.url_validator import URLValidator

# Block a javascript URL (XSS)

assert URLValidator.is_safe_url("javascript:alert('XSS')") is False

# Allow a normal HTTPS link

assert URLValidator.is_safe_url("https://arxiv.org/abs/2301.00001") is True

# Safe redirect inside the same host

host = "https://myapp.example.com/"
assert URLValidator.is_safe_redirect_url("/dashboard", host) is True

# Reject open-redirect to another domain

assert URLValidator.is_safe_redirect_url("https://evil.com", host) is False

File Integrity Verification

To prevent unauthorized file modifications, src/local_deep_research/security/file_write_verifier.py implements runtime integrity checks using the verify_file_write() function.

SHA-256 Whitelist Enforcement

Before any file write operation completes, the verifier computes a SHA-256 digest of the payload and compares it against an expected hash stored in .file-whitelist.txt. If the digests mismatch, the write operation aborts and the system logs a security warning. This guarantees that only pre-approved static resources—such as bundled model files or configuration templates—can be created or overwritten at runtime.

Pre-Commit Hook Integration

The repository includes a pre-commit hook at .pre-commit-hooks/check-file-whitelist-check.sh that enforces whitelist compliance during continuous integration, ensuring that committed files match their registered hashes before deployment.

from local_deep_research.security.file_write_verifier import verify_file_write

# Assume `payload` is a bytes object we just downloaded

# The whitelist contains the expected SHA-256 hash for "model.bin"

if verify_file_write("model.bin", payload):
    with open("model.bin", "wb") as f:
        f.write(payload)
else:
    raise RuntimeError("File integrity check failed – aborting write")

Summary

• SSRF Protection: The ssrf_validator.py module blocks requests to private IP ranges and cloud metadata endpoints via is_ip_blocked(), while allowing test-specific bypasses through environment variables.
• URL Validation: url_validator.py sanitizes input through scheme blacklisting (javascript:, data:), safe-domain whitelisting, and pattern detection for encoding attacks and open redirects.
• File Integrity: file_write_verifier.py computes SHA-256 hashes at runtime, rejecting any file writes that do not match the cryptographic whitelist stored in .file-whitelist.txt.

Frequently Asked Questions

How does Local-Deep-Research prevent SSRF attacks against cloud metadata endpoints?

The is_ip_blocked() function in ssrf_validator.py explicitly denies access to the AWS metadata IP 169.254.169.254 alongside other internal ranges defined in ip_ranges.py. This prevents attackers from using the application to fetch sensitive instance credentials from cloud metadata services.

What URL schemes does the validator consider unsafe?

According to the URLValidator.is_unsafe_scheme() implementation, the system blocks javascript:, data:, vbscript:, about:, blob:, and file: schemes. Only http, https, ftp, and ftps are permitted by default, with optional mailto support.

How does the file integrity verification system work?

When the application attempts to write a file, verify_file_write() calculates a SHA-256 hash of the content and validates it against .file-whitelist.txt. Writes are rejected immediately if the fingerprint is unknown, ensuring that only cryptographically verified static assets can persist on disk.

Can developers bypass SSRF protection for local testing?

Yes, the validate_url() function checks for the PYTEST_CURRENT_TEST environment variable. When this variable is detected, SSRF validation is bypassed exclusively for test execution, allowing integration with local mock servers without exposing the production environment to internal network requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}