How CoSky Manages Ephemeral Service Instances with TTL: A Redis-Based Service Registry Deep Dive

CoSky manages ephemeral service instances by storing them in Redis with a TTL (time-to-live) field (ttlAt), automatically expiring keys when heartbeats stop, while permanent instances use a sentinel value of -1 (TTL_AT_FOREVER) to persist indefinitely.

CoSky is an open-source service discovery and configuration platform built on Redis. Understanding how CoSky manages ephemeral service instances with TTL is essential for building resilient microservices that automatically prune dead nodes. The implementation combines Kotlin data classes, Redis hash storage, and atomic Lua scripts to ensure consistent lifecycle management across the cluster.

Core Data Model: ServiceInstance and the TTL Field

Every service instance in CoSky is represented by the ServiceInstance data class. This model includes a ttlAt field that stores the UNIX epoch second when the instance should expire.

The TTL Sentinel Value

In cosky-discovery/src/main/kotlin/me/ahoo/cosky/discovery/ServiceInstance.kt at line 46, the code defines a sentinel value for instances that should never expire:

const val TTL_AT_FOREVER = -1L

When ttlAt == TTL_AT_FOREVER, the instance is considered permanent (non-ephemeral). For ephemeral instances, ttlAt holds a positive timestamp indicating when the instance becomes invalid if not renewed.

Ephemeral vs. Permanent Storage

The ServiceInstance class distinguishes between these two modes using the isEphemeral boolean property. This flag determines whether the registry applies a Redis EXPIRE command to the instance key during registration and renewal.

Redis Storage and Codec Implementation

The ServiceInstanceCodec utility handles the conversion between Kotlin objects and Redis hash fields. Located in cosky-discovery/src/main/kotlin/me/ahoo/cosky/discovery/ServiceInstanceCodec.kt, the codec defines specific keys for persistence.

Hash Field Mapping

At line 33, the codec declares the ephemeral flag key:

private const val EPHEMERAL = "ephemeral"

During encoding, the codec writes the ephemeral status and TTL value as string fields:

encodeMetadataKey(EPHEMERAL) to instance.isEphemeral.toString()
encodeMetadataKey(TTL_AT) to instance.ttlAt.toString()

During decoding, if the TTL field is missing, the codec defaults to TTL_AT_FOREVER, ensuring backward compatibility and safety for permanent instances.

Registration Flow: Setting the Initial TTL

When a service registers, RedisServiceRegistry.register executes the Lua script registry_register.lua to store the instance atomically.

The registry_register.lua Script

At line 41 of cosky-discovery/src/main/resources/registry_register.lua, the script stores the ephemeral flag within the Redis hash:

redis.call("hmset", instanceKey,
    "instanceId", instanceId,
    "serviceId", serviceId,
    "schema", schema,
    "host", host,
    "port", port,
    "weight", weight,
    "ephemeral", ephemeral,
    unpack(ARGV, 8, #ARGV))

For ephemeral instances where ephemeral == "true", the script immediately applies a Redis expiration:

redis.call("expire", instanceKey, ttl)

This ensures that the TTL is enforced by Redis itself, not just application logic.

Heartbeat Mechanism: Renewing the TTL

Ephemeral instances must periodically invoke the renew method to refresh their expiration time. This heartbeat pattern prevents Redis from automatically deleting the instance key.

Validation in RedisServiceRegistry

In cosky-discovery/src/main/kotlin/me/ahoo/cosky/discovery/redis/RedisServiceRegistry.kt at line 167, the registry validates the ephemeral status before allowing renewal:

if (!serviceInstance.isEphemeral) {
    log.warn("Renew - instanceId:[${serviceInstance.instanceId}] @ namespace:[$namespace] is not ephemeral, can not renew.")
    return false
}

If the instance is ephemeral, the registry executes registry_renew.lua, which recalculates the TTL from ttlAt and resets the Redis key's expiration time using the EXPIRE command.

Service Discovery: Reading TTL Values

Clients retrieve service instances through RedisServiceDiscovery.getInstances, which enriches the returned data with current TTL information.

Fetching Remaining TTL

At line 83 of cosky-discovery/src/main/kotlin/me/ahoo/cosky/discovery/redis/RedisServiceDiscovery.kt, the discovery service invokes discovery_get_instance_ttl.lua:

DiscoveryRedisScripts.SCRIPT_REGISTRY_GET_INSTANCE_TTL,

This script returns the remaining seconds of the Redis key's lifetime, which the discovery client uses to populate the ttlAt field on the returned ServiceInstance objects. Clients can then determine liveness:

val now = System.currentTimeMillis() / 1000
val isExpired = instance.ttlAt < now && instance.ttlAt != ServiceInstance.TTL_AT_FOREVER

Automatic Expiration and Cleanup

When an ephemeral instance fails to send heartbeats within the TTL window, Redis automatically deletes the hash key due to the EXPIRE command set during registration and renewal. The discovery layer no longer returns these instances in getInstances or choose calls, effectively removing dead nodes from the service pool without requiring explicit deregistration.

Permanent instances, marked with TTL_AT_FOREVER, never have an EXPIRE command applied, so they persist until explicitly deregistered through the API.

Practical Implementation Examples

Register an Ephemeral Instance

val instance = InstanceDto(
    instanceId = UUID.randomUUID().toString(),
    host = "10.0.1.5",
    port = 8080,
    weight = 1,
    isEphemeral = true,
    ttlAt = System.currentTimeMillis() / 1000 + 30, // 30 second TTL
    metadata = mapOf("version" to "1.0")
).asServiceInstance(serviceId)

serviceRegistry.register(namespace, instance)   // triggers registry_register.lua

Send a Heartbeat

// Called every 20 seconds (must be less than TTL)
serviceRegistry.renew(namespace, instanceId)   // runs registry_renew.lua

Discover Instances with TTL Information

val instances: List<ServiceInstance> = discoveryClient.getInstances(namespace, serviceId)
instances.forEach { inst ->
    println("Instance ${inst.instanceId} expires at ${inst.ttlAt}")
}

Filter Expired Instances Manually

val now = System.currentTimeMillis() / 1000
val alive = instances.filter { 
    it.ttlAt == ServiceInstance.TTL_AT_FOREVER || it.ttlAt > now 
}

Summary

• Ephemeral instances use a ttlAt timestamp and the ephemeral=true flag, triggering Redis EXPIRE commands during registration and renewal.
• Permanent instances set ttlAt = TTL_AT_FOREVER (-1) and never receive expiration commands, persisting until manual deletion.
• Heartbeat renewal is strictly enforced for ephemeral instances only; attempts to renew permanent instances log warnings and return false.
• Automatic cleanup occurs when Redis deletes expired keys, causing the discovery layer to immediately exclude dead instances.
• TTL discovery uses the discovery_get_instance_ttl.lua script to provide clients with real-time remaining lifetime data.

Frequently Asked Questions

What happens if a non-ephemeral instance tries to renew its TTL?

According to the source code in RedisServiceRegistry.kt (line 167), the system logs a warning message stating the instance is not ephemeral and cannot renew, then returns false. Permanent instances must be explicitly deregistered rather than relying on TTL expiration.

How does Redis actually delete expired ephemeral instances?

Redis handles deletion automatically through the native EXPIRE mechanism. When registry_register.lua or registry_renew.lua executes redis.call("expire", instanceKey, ttl), Redis starts a countdown. If no renewal occurs before the TTL elapses, Redis deletes the key entirely, removing the instance from subsequent discovery queries.

How does the discovery client know the remaining lifetime of an instance?

The RedisServiceDiscovery class executes discovery_get_instance_ttl.lua (referenced at line 83 of RedisServiceDiscovery.kt) to fetch the remaining seconds from Redis. This value is mapped back to the ttlAt field of the ServiceInstance object, allowing clients to calculate exact expiration times.

What distinguishes ephemeral instances from permanent ones in the data model?

The ServiceInstanceCodec.kt file (line 33) defines an EPHEMERAL hash key stored in Redis. Ephemeral instances have ephemeral="true" and a positive ttlAt value, while permanent instances have ephemeral="false" and ttlAt set to TTL_AT_FOREVER (-1) as defined in ServiceInstance.kt (line 46).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}