# URL Capture Validation Rules in Claude-Obsidian: A Deep Dive into Safe URL Handling

> Explore Claude-Obsidian URL capture validation rules. Learn how it ensures safe handling by blocking credentials private networks and malformed URLs for secure data input.

- Repository: [Agrici.Daniel/claude-obsidian](https://github.com/AgriciDaniel/claude-obsidian)
- Tags: deep-dive
- Published: 2026-08-29

---

**Claude-obsidian enforces strict URL capture validation through a multi-stage pipeline that restricts submissions to canonical, public HTTPS URLs while blocking credentials, private networks, and malformed structures.**

The claude-obsidian repository implements rigorous security controls for URL capture functionality. Understanding these URL capture validation rules is essential for developers integrating with the capture API or extending its safety mechanisms. The validation logic resides primarily in [`claude_obsidian/capture.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/capture.py) with credential detection outsourced to [`claude_obsidian/url_safety.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/url_safety.py).

## Validation Pipeline Overview

The system processes every submitted URL through a deterministic, side-effect-free pipeline consisting of three distinct stages. Each stage targets specific attack vectors or structural violations before the function returns a canonicalized HTTPS string.

- **Structural validation** enforces byte limits and character constraints
- **Scheme and credential checks** verify protocol safety and secret leakage prevention
- **Host and network verification** ensures public routability and domain safety

## Stage 1: Basic Structure and Format Validation

The `validate_https_url` function in [`claude_obsidian/capture.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/capture.py) (lines 1082-1089) performs initial sanitization to eliminate malformed or oversized inputs.

The function rejects any URL that violates these constraints:

- **Length limit**: Must be a non-empty `str` ≤ 8192 bytes
- **Character restrictions**: No whitespace, Unicode control characters, or backslashes (`\`)

These checks prevent buffer overflow scenarios and path traversal attempts before parsing begins.

## Stage 2: Scheme and Credential Sanitization

After structural validation, the pipeline enforces protocol restrictions and scans for embedded secrets.

### HTTPS-Only Enforcement

The validator checks the URL scheme at lines 1099-1102 in [`claude_obsidian/capture.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/capture.py). Only the `https` scheme (case-insensitive) is permitted; `http`, `ftp`, `file`, and other protocols trigger a `URL_SCHEME_FORBIDDEN` error.

### Userinfo and Secret Detection

Credential leakage prevention relies on `url_credential_issue` defined in [`claude_obsidian/url_safety.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/url_safety.py) (lines 77-91). This function inspects parsed URLs for:

- **Userinfo components**: Any `username:password@` syntax is forbidden
- **Sensitive query parameters**: The validator compares query keys against `_SENSITIVE_QUERY_KEYS`, `_SENSITIVE_QUERY_PARTS`, and `_SENSITIVE_QUERY_SUFFIXES` constants

Matches trigger `URL_USERINFO_FORBIDDEN` or `URL_SECRET_FORBIDDEN` errors, blocking URLs containing `access_token`, `api_key`, `password`, or heuristic variants.

## Stage 3: Host and Network Safety Verification

The `_validate_public_host` helper (lines 55-78 in [`claude_obsidian/capture.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/capture.py)) implements the most complex security logic, ensuring captured content originates from publicly accessible infrastructure.

### IDNA Normalization

The `_normalize_host` function (lines 45-52) converts internationalized domain names to ASCII-compatible encoding using IDNA standards.

### Forbidden Host Patterns

The validator explicitly blocks:
- `localhost` and any domain ending with `.localhost`
- Private topology domains: `.local`, `.internal`, `.lan`, `.home`, `.test`, `.invalid`

### IP Address Constraints

- **Single-label hostnames** (no dot) are rejected unless they resolve to an IP address
- **IP addresses** must be globally routable (`address.is_global` must return `True`)
- **Port ranges**, if specified, must fall within 1-65535

### Fragment Prohibition

URLs containing fragment identifiers (`#section`) are rejected with `URL_FRAGMENT_FORBIDDEN`, ensuring capture targets refer to complete resources rather than subresource anchors.

## Canonical Output Format

Upon passing all validation stages, `validate_https_url` returns a canonicalized string constructed at lines 1120-1124:

- **Netloc**: Hostname (IPv6 wrapped in brackets) with port omitted if 443
- **Path**: Defaults to `/` if empty
- **Query string**: Preserved unchanged from the original

## Implementation Examples

```python
from claude_obsidian.capture import validate_https_url

# ✅ Valid URL

url = "https://example.com/path?query=foo"
print(validate_https_url(url))

# → "https://example.com/path?query=foo"

# ❌ Invalid scheme

validate_https_url("http://example.com")

# CaptureValidationError: URL_SCHEME_FORBIDDEN – only HTTPS capture URLs are accepted

# ❌ Userinfo present

validate_https_url("https://user:pass@example.com")

# CaptureValidationError: URL_USERINFO_FORBIDDEN – URL userinfo credentials are forbidden

# ❌ Sensitive query param

validate_https_url("https://example.com/download?access_token=abc")

# CaptureValidationError: URL_SECRET_FORBIDDEN – sensitive URL query parameter is forbidden: access_token

# ❌ Private host

validate_https_url("https://localhost/api")

# CaptureValidationError: URL_PRIVATE_HOST – local URL host is forbidden: localhost

# ❌ Non‑public IP

validate_https_url("https://192.168.1.10/data")

# CaptureValidationError: URL_PRIVATE_HOST – non‑public IP is forbidden: 192.168.1.10

# ❌ Fragment included

validate_https_url("https://example.com/page#section")

# CaptureValidationError: URL_FRAGMENT_FORBIDDEN – capture URLs must not contain fragments

```

## Summary

- **Structural limits**: URLs must be ≤8192 bytes with no whitespace, control characters, or backslashes
- **Protocol restriction**: Only HTTPS scheme is accepted (case-insensitive)
- **Credential blocking**: Userinfo and sensitive query parameters (defined in [`url_safety.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/url_safety.py) constants) are forbidden
- **Network isolation**: Private IPs, localhost, and reserved domains (`.local`, `.internal`, etc.) are rejected
- **Output guarantee**: Valid URLs return as canonical HTTPS strings without fragments or default ports

## Frequently Asked Questions

### Why does claude-obsidian reject URLs with fragments?

The validator prohibits fragment identifiers (`#`) because they refer to subresource locations within a document rather than the document itself. As implemented in [`claude_obsidian/capture.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/capture.py), this ensures the capture system archives complete resources rather than anchored sections, preventing content inconsistency when fragments resolve to dynamic content.

### How does the validator detect sensitive query parameters?

Detection occurs in [`claude_obsidian/url_safety.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/url_safety.py) through the `url_credential_issue` function, which compares query keys against known secret patterns stored in `_SENSITIVE_QUERY_KEYS`, `_SENSITIVE_QUERY_PARTS`, and `_SENSITIVE_QUERY_SUFFIXES` constants. Any match triggers a `URL_SECRET_FORBIDDEN` error before network requests are initiated.

### What constitutes a "public" host in the validation context?

A public host must resolve to a globally routable IP address where `address.is_global` returns `True`. The `_validate_public_host` helper (lines 55-78) explicitly rejects localhost, RFC 1918 private addresses, and reserved TLDs like `.local` or `.internal`, ensuring captured content originates from internet-facing infrastructure.

### Can I override the HTTPS-only restriction for internal testing?

No. The scheme check at lines 1099-1102 in [`claude_obsidian/capture.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/capture.py) enforces HTTPS without configuration exceptions. This hardcoded restriction prevents accidental credential leakage over unencrypted transport and ensures the canonicalization logic (which assumes TLS port 443 defaults) functions correctly across all deployments.