URL Capture Validation Rules in Claude-Obsidian: A Deep Dive into Safe URL Handling

Claude-obsidian enforces strict URL capture validation through a multi-stage pipeline that restricts submissions to canonical, public HTTPS URLs while blocking credentials, private networks, and malformed structures.

The claude-obsidian repository implements rigorous security controls for URL capture functionality. Understanding these URL capture validation rules is essential for developers integrating with the capture API or extending its safety mechanisms. The validation logic resides primarily in claude_obsidian/capture.py with credential detection outsourced to claude_obsidian/url_safety.py.

Validation Pipeline Overview

The system processes every submitted URL through a deterministic, side-effect-free pipeline consisting of three distinct stages. Each stage targets specific attack vectors or structural violations before the function returns a canonicalized HTTPS string.

• Structural validation enforces byte limits and character constraints
• Scheme and credential checks verify protocol safety and secret leakage prevention
• Host and network verification ensures public routability and domain safety

Stage 1: Basic Structure and Format Validation

The validate_https_url function in claude_obsidian/capture.py (lines 1082-1089) performs initial sanitization to eliminate malformed or oversized inputs.

The function rejects any URL that violates these constraints:

• Length limit: Must be a non-empty str ≤ 8192 bytes
• Character restrictions: No whitespace, Unicode control characters, or backslashes (\)

These checks prevent buffer overflow scenarios and path traversal attempts before parsing begins.

Stage 2: Scheme and Credential Sanitization

After structural validation, the pipeline enforces protocol restrictions and scans for embedded secrets.

HTTPS-Only Enforcement

The validator checks the URL scheme at lines 1099-1102 in claude_obsidian/capture.py. Only the https scheme (case-insensitive) is permitted; http, ftp, file, and other protocols trigger a URL_SCHEME_FORBIDDEN error.

Userinfo and Secret Detection

Credential leakage prevention relies on url_credential_issue defined in claude_obsidian/url_safety.py (lines 77-91). This function inspects parsed URLs for:

• Userinfo components: Any username:password@ syntax is forbidden
• Sensitive query parameters: The validator compares query keys against _SENSITIVE_QUERY_KEYS, _SENSITIVE_QUERY_PARTS, and _SENSITIVE_QUERY_SUFFIXES constants

Matches trigger URL_USERINFO_FORBIDDEN or URL_SECRET_FORBIDDEN errors, blocking URLs containing access_token, api_key, password, or heuristic variants.

Stage 3: Host and Network Safety Verification

The _validate_public_host helper (lines 55-78 in claude_obsidian/capture.py) implements the most complex security logic, ensuring captured content originates from publicly accessible infrastructure.

IDNA Normalization

The _normalize_host function (lines 45-52) converts internationalized domain names to ASCII-compatible encoding using IDNA standards.

Forbidden Host Patterns

The validator explicitly blocks:

• localhost and any domain ending with .localhost
• Private topology domains: .local, .internal, .lan, .home, .test, .invalid

IP Address Constraints

• Single-label hostnames (no dot) are rejected unless they resolve to an IP address
• IP addresses must be globally routable (address.is_global must return True)
• Port ranges, if specified, must fall within 1-65535

Fragment Prohibition

URLs containing fragment identifiers (#section) are rejected with URL_FRAGMENT_FORBIDDEN, ensuring capture targets refer to complete resources rather than subresource anchors.

Canonical Output Format

Upon passing all validation stages, validate_https_url returns a canonicalized string constructed at lines 1120-1124:

• Netloc: Hostname (IPv6 wrapped in brackets) with port omitted if 443
• Path: Defaults to / if empty
• Query string: Preserved unchanged from the original

Implementation Examples

from claude_obsidian.capture import validate_https_url

# ✅ Valid URL

url = "https://example.com/path?query=foo"
print(validate_https_url(url))

# → "https://example.com/path?query=foo"

# ❌ Invalid scheme

validate_https_url("http://example.com")

# CaptureValidationError: URL_SCHEME_FORBIDDEN – only HTTPS capture URLs are accepted

# ❌ Userinfo present

validate_https_url("https://user:pass@example.com")

# CaptureValidationError: URL_USERINFO_FORBIDDEN – URL userinfo credentials are forbidden

# ❌ Sensitive query param

validate_https_url("https://example.com/download?access_token=abc")

# CaptureValidationError: URL_SECRET_FORBIDDEN – sensitive URL query parameter is forbidden: access_token

# ❌ Private host

validate_https_url("https://localhost/api")

# CaptureValidationError: URL_PRIVATE_HOST – local URL host is forbidden: localhost

# ❌ Non‑public IP

validate_https_url("https://192.168.1.10/data")

# CaptureValidationError: URL_PRIVATE_HOST – non‑public IP is forbidden: 192.168.1.10

# ❌ Fragment included

validate_https_url("https://example.com/page#section")

# CaptureValidationError: URL_FRAGMENT_FORBIDDEN – capture URLs must not contain fragments

Summary

• Structural limits: URLs must be ≤8192 bytes with no whitespace, control characters, or backslashes
• Protocol restriction: Only HTTPS scheme is accepted (case-insensitive)
• Credential blocking: Userinfo and sensitive query parameters (defined in url_safety.py constants) are forbidden
• Network isolation: Private IPs, localhost, and reserved domains (.local, .internal, etc.) are rejected
• Output guarantee: Valid URLs return as canonical HTTPS strings without fragments or default ports

Frequently Asked Questions

Why does claude-obsidian reject URLs with fragments?

The validator prohibits fragment identifiers (#) because they refer to subresource locations within a document rather than the document itself. As implemented in claude_obsidian/capture.py, this ensures the capture system archives complete resources rather than anchored sections, preventing content inconsistency when fragments resolve to dynamic content.

How does the validator detect sensitive query parameters?

Detection occurs in claude_obsidian/url_safety.py through the url_credential_issue function, which compares query keys against known secret patterns stored in _SENSITIVE_QUERY_KEYS, _SENSITIVE_QUERY_PARTS, and _SENSITIVE_QUERY_SUFFIXES constants. Any match triggers a URL_SECRET_FORBIDDEN error before network requests are initiated.

What constitutes a "public" host in the validation context?

A public host must resolve to a globally routable IP address where address.is_global returns True. The _validate_public_host helper (lines 55-78) explicitly rejects localhost, RFC 1918 private addresses, and reserved TLDs like .local or .internal, ensuring captured content originates from internet-facing infrastructure.

Can I override the HTTPS-only restriction for internal testing?

No. The scheme check at lines 1099-1102 in claude_obsidian/capture.py enforces HTTPS without configuration exceptions. This hardcoded restriction prevents accidental credential leakage over unencrypted transport and ensures the canonicalization logic (which assumes TLS port 443 defaults) functions correctly across all deployments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}