# How Marin EndpointsService Enables Service Discovery and Lazy Resolution

> Learn how Marin's EndpointsService offers service discovery and lazy resolution. It resolves abstract URIs to HTTP URLs on demand, reducing network overhead during configuration parsing.

- Repository: [The Marin Project/marin](https://github.com/marin-community/marin)
- Tags: internals
- Published: 2026-08-28

---

**The Marin EndpointsService enables service discovery and lazy resolution by maintaining a pluggable registry of scheme resolvers that convert abstract URIs (such as `gcp://` or `k8s://`) into concrete HTTP URLs only when `resolve_endpoint_uri` is called, eliminating network overhead during configuration parsing.**

The Marin distributed system framework decouples service configuration from physical infrastructure using a flexible endpoints system. Unlike traditional service discovery that resolves addresses during startup, the **Marin EndpointsService** (`iris.cluster.endpoints`) implements lazy resolution to keep cluster initialization fast and avoid unnecessary external network calls.

## Core Architecture of the Marin EndpointsService

The endpoints system centers on a lightweight resolution layer that abstracts multiple backend providers behind a uniform API. This design allows cluster configurations to declare logical service locations using scheme-tagged URIs while deferring the actual lookup until runtime.

### The Scheme Resolver Registry

At the heart of the service lies a private registry mapping scheme names to resolver callables. In [`lib/iris/src/iris/cluster/endpoints.py`](https://github.com/marin-community/marin/blob/main/lib/iris/src/iris/cluster/endpoints.py), the `_REGISTRY` dictionary stores these mappings, populated via the public `register_scheme` function:

```python
def register_scheme(scheme: str, handler: SchemeResolver) -> None:
    """Register ``handler`` as the resolver for ``scheme://...`` URIs."""
    _REGISTRY[scheme] = handler          # → https://github.com/marin-community/marin/blob/main/lib/iris/src/iris/cluster/endpoints.py#L57-L60

```

Built-in resolvers for Google Cloud Platform (`gcp://`) and Kubernetes (`k8s://`) are registered automatically at module import time on lines 94‑95. This Registry pattern makes the system **scheme-agnostic** and allows new discovery mechanisms to be added without modifying core controller code.

### URI Resolution API

The primary entry point for all resolution requests is `resolve_endpoint_uri` (line 62), which accepts a scheme-tagged URI and metadata dictionary, then returns a concrete `http(s)://` URL:

```python
from iris.cluster.endpoints import resolve_endpoint_uri

uri = "gcp://my-controller-vm"
metadata = {"zone": "us-central1-a", "port": "10001"}
concrete_url = resolve_endpoint_uri(uri, metadata)

# → "http://10.128.0.5:10001"

```

## How Lazy Resolution Works in Marin

**Lazy resolution** means that resolver functions are invoked only when a component explicitly requests the concrete address. During configuration parsing, no network calls occur—not even DNS lookups. This defers expensive operations like `gcloud` CLI executions or Kubernetes API queries until the actual moment the URL is needed.

The controller triggers this resolution via `_resolve_cluster_endpoints` in [`lib/iris/src/iris/cluster/controller/main.py`](https://github.com/marin-community/marin/blob/main/lib/iris/src/iris/cluster/controller/main.py) (lines 46‑61). This function iterates over `cluster_config.endpoints` and calls `resolve_endpoint_uri` for each entry, ensuring that startup remains fast while guaranteeing that components receive resolvable addresses before they attempt connections.

## Built-in Scheme Resolvers

Marin ships with three primary resolvers that handle the most common deployment scenarios.

### GCP Cloud VM Discovery

The `_resolve_gcp` function (lines 27‑55) handles `gcp://` URIs by constructing and executing a `gcloud` command to retrieve the instance's internal IP address. It parses the CLI output and constructs the final URL:

```python

# Example resolution flow for gcp://my-vm

# 1. Executes: gcloud compute instances describe my-vm --zone=us-central1-a

# 2. Extracts internal IP: 10.128.0.5

# 3. Returns: http://10.128.0.5:10001

```

### Kubernetes In-Cluster Resolution

For `k8s://` URIs, the `_resolve_k8s` resolver (lines 67‑91) performs local DNS construction without external API calls. It transforms the URI into the standard Kubernetes cluster DNS format:

```python

# k8s://service-name/namespace resolves to:

http://service-name.namespace.svc.cluster.local:port

```

This approach avoids requiring kubeconfig or API access from the controller, relying instead on the cluster's internal DNS infrastructure.

### Direct HTTP/HTTPS Passthrough

URIs using `http://` or `https://` schemes bypass resolution entirely. The `resolve_endpoint_uri` function returns these unchanged, allowing configurations to mix static URLs with dynamic discovery endpoints seamlessly.

## Extending Service Discovery with Custom Schemes

The registration API enables developers to inject custom discovery logic without forking the core framework. Any callable matching the `SchemeResolver` signature can handle new protocols:

```python
from iris.cluster.endpoints import register_scheme, SchemeResolver

def myproto_resolver(uri: str, meta: dict[str, str]) -> str:
    # Custom logic to resolve the endpoint

    host = uri.split("://", 1)[1]
    return f"https://myproxy.example.com/{host}"

register_scheme("myproto", myproto_resolver)

# Usage

print(resolve_endpoint_uri("myproto://serviceA", {}))

# → "https://myproxy.example.com/serviceA"

```

Once registered, the custom resolver participates fully in lazy resolution alongside built-in providers.

## Integration with the Cluster Controller

The controller coordinates endpoint discovery during cluster initialization. In [`lib/iris/src/iris/cluster/controller/main.py`](https://github.com/marin-community/marin/blob/main/lib/iris/src/iris/cluster/controller/main.py), the `_resolve_cluster_endpoints` function (lines 46‑61) consumes the configuration and populates a dictionary of resolved addresses:

```python

# Inside iris.cluster.controller.main

endpoints = _resolve_cluster_endpoints(cluster_config)
log_server = endpoints.get("/system/log-server", "")

# `log_server` now holds the concrete URL or an empty string if not configured.

```

This integration point demonstrates how Marin separates configuration schema from runtime resolution, allowing the same configuration to resolve differently across development, staging, and production environments without code changes.

## Summary

- **Marin EndpointsService** provides scheme-agnostic service discovery through a pluggable resolver registry stored in [`lib/iris/src/iris/cluster/endpoints.py`](https://github.com/marin-community/marin/blob/main/lib/iris/src/iris/cluster/endpoints.py).
- **Lazy resolution** defers all network operations until `resolve_endpoint_uri` is explicitly called, keeping startup performance consistent regardless of backend latency.
- **Built-in resolvers** handle Google Cloud VMs (`gcp://`), Kubernetes services (`k8s://`), and static HTTP/HTTPS URLs without requiring external configuration.
- **Extensible registration** via `register_scheme` allows custom protocols to integrate seamlessly with the existing resolution pipeline.
- **Controller integration** in [`lib/iris/src/iris/cluster/controller/main.py`](https://github.com/marin-community/marin/blob/main/lib/iris/src/iris/cluster/controller/main.py) orchestrates bulk resolution while maintaining the lazy evaluation pattern.

## Frequently Asked Questions

### What is lazy resolution in Marin?

Lazy resolution is a design pattern in the Marin EndpointsService where network-intensive operations—such as querying Google Cloud APIs or performing DNS lookups—are deferred until a component actually requests the concrete service URL. This ensures that configuration parsing and system startup remain fast and do not block on external dependencies.

### How do I add a custom scheme resolver?

Import `register_scheme` from `iris.cluster.endpoints` and pass your resolver function along with the scheme name. Your function must accept a URI string and metadata dictionary and return a concrete HTTP or HTTPS URL. Once registered, any configuration using your scheme (e.g., `custom://service`) will automatically route to your resolver when `resolve_endpoint_uri` is called.

### When does Marin actually resolve gcp:// and k8s:// endpoints?

Resolution occurs at runtime when the controller or a specific component invokes `resolve_endpoint_uri`. For the cluster controller, this happens inside `_resolve_cluster_endpoints` during initialization, but after the configuration has been loaded. No resolution occurs during the initial parsing of the YAML or JSON configuration files.

### What happens if an endpoint URI scheme is not registered?

If `resolve_endpoint_uri` encounters a scheme that does not exist in the `_REGISTRY`, the function will raise a KeyError or similar exception indicating that no resolver is available for that scheme. Valid configurations should only use schemes that are either built-in (`http`, `https`, `gcp`, `k8s`) or explicitly registered via `register_scheme` before resolution is attempted.