# How the SIP008 Online Configuration Update Mechanism Fetches and Merges Remote Servers

> Learn how SIP008 fetches and merges remote server configurations in Shadowsocks Windows. Discover its efficient update mechanism

- Repository: [shadowsocks/shadowsocks-windows](https://github.com/shadowsocks/shadowsocks-windows)
- Tags: internals
- Published: 2026-03-05

---

**The SIP008 online configuration update mechanism downloads remote server configurations via HTTP GET, recursively parses JSON to extract valid server objects, tags them with their source URL to enable source-based deduplication, and atomically replaces existing entries from the same source before persisting the merged server list.**

The SIP008 online configuration update mechanism in shadowsocks-windows enables dynamic synchronization of proxy servers from remote JSON endpoints. Implemented across three distinct architectural layers—ViewModel, Controller, and Resolver—this system handles everything from user interface interactions to low-level JSON parsing. The implementation ensures that remote server lists remain synchronized with their authoritative sources while preserving user preferences and local server entries.

## Three-Layer Architecture

The feature follows a strict separation of concerns across three layers:

- **UI / ViewModel Layer**: Collects URLs from users and initiates updates via Reactive commands. Key file: [`shadowsocks-csharp/ViewModels/OnlineConfigViewModel.cs`](https://github.com/shadowsocks/shadowsocks-windows/blob/main/shadowsocks-csharp/ViewModels/OnlineConfigViewModel.cs).
- **Controller Layer**: Orchestrates the download, parsing, merging, and persistence logic. Key file: [`shadowsocks-csharp/Controller/ShadowsocksController.cs`](https://github.com/shadowsocks/shadowsocks-windows/blob/main/shadowsocks-csharp/Controller/ShadowsocksController.cs).
- **Resolver Layer**: Performs HTTP requests and recursively extracts server objects from arbitrary JSON structures. Key file: [`shadowsocks-csharp/Controller/Service/OnlineConfigResolver.cs`](https://github.com/shadowsocks/shadowsocks-windows/blob/main/shadowsocks-csharp/Controller/Service/OnlineConfigResolver.cs).

## User-Initiated Update Flow

The ViewModel exposes three Reactive commands—`Update`, `UpdateAll`, and `Add`—bound directly to UI controls. When a user selects a source and clicks *Update*, the command executes:

```csharp
Update = ReactiveCommand.CreateFromTask(
    () => _controller.UpdateOnlineConfig(SelectedSource), 
    canUpdateCopyRemove
);

```

This command returns a boolean indicating success, which the ViewModel surfaces to the user via message boxes.

## Controller Orchestration

The `ShadowsocksController` class manages the core logic for fetching and merging configurations through three primary methods.

### UpdateOnlineConfig Method

The `UpdateOnlineConfig(string url)` method (lines 77-92) preserves user context while refreshing remote data:

1. Captures the currently selected server identifier.
2. Invokes `UpdateOnlineConfigInternal(url)` to perform the actual update.
3. Restores the user's server selection in the merged list via `_config.index`.
4. Persists the configuration to disk using `SaveConfig`.

### UpdateOnlineConfigInternal Method

The internal implementation (lines 65-75) handles the atomic replacement logic:

```csharp
var onlineServer = await OnlineConfigResolver.GetOnline(url);
// Remove existing entries from this source
_config.configs = _config.configs
    .Where(c => c.group != url)
    .Concat(onlineServer)
    .ToList();
Configuration.SortByOnlineConfig(_config.configs);

```

This method returns the count of newly added servers and ensures that stale entries from the same URL are completely removed before inserting fresh data.

### UpdateAllOnlineConfig Method

For bulk operations, `UpdateAllOnlineConfig()` (lines 94-114) iterates over all URLs stored in `_config.onlineConfigSource`, invoking `UpdateOnlineConfigInternal` for each source. The method aggregates failed URLs into a list, allowing the UI to display specific error information for problematic endpoints.

## Remote Fetch and JSON Parsing

The `OnlineConfigResolver` class handles the network layer and flexible JSON extraction.

### HTTP GET Execution

The `GetOnline(string url)` method (lines 14-23) obtains an `HttpClient` instance from the main controller and performs the request:

```csharp
string server_json = await httpClient.GetStringAsync(url);
var servers = server_json.GetServers();
foreach (var server in servers)
{
    server.group = url;  // Tag with source for deduplication
}
return servers;

```

### Recursive JSON Extraction

The `OnlineConfigResolverEx.GetServers(this string json)` extension method uses **Newtonsoft.Json** to parse arbitrarily nested JSON structures. The recursive search (`SearchJObject`, `SearchJArray`, `SearchJToken`) traverses the entire object graph, identifying any object containing the mandatory SIP008 fields:

- `server`
- `server_port`
- `password`
- `method`

Each matching object converts to a `Server` model via `obj.ToObject<Server>()`, enabling support for various provider-specific JSON schemas without rigid structure requirements.

## Server Merging Strategy

The SIP008 online configuration update mechanism employs a source-aware merging strategy to maintain data integrity.

### Deduplication by Source URL

Before adding freshly-fetched servers, the controller executes:

```csharp
.Where(c => c.group != url)

```

This LINQ filter removes all existing entries whose `group` property equals the URL being refreshed. Since the resolver tags every fetched server with its origin URL (`server.group = url`), the system can precisely identify and replace only the entries originating from that specific remote source, leaving manually configured servers untouched.

### Sorting and Persistence

After concatenation, the merged collection passes through `Configuration.SortByOnlineConfig`, which orders servers according to SIP008 specifications (typically by `remarks` or `group` fields). The sorted list becomes the authoritative server set, and `SaveConfig()` atomically writes the updated configuration to disk.

## Implementation Examples

### Triggering Updates from the UI

```csharp
// In OnlineConfigView.xaml.cs (auto-generated bindings)
// User selects a source and clicks "Update"
await viewModel.Update.Execute();

// Result is a bool: true = success, false = failure

```

### Programmatic Bulk Updates

For background synchronization on startup:

```csharp
// Program.cs – background task launched after controller starts
Task.Run(async () =>
{
    await Task.Delay(TimeSpan.FromSeconds(10));
    
    List<string> failed = await MainController.UpdateAllOnlineConfig();
    
    if (failed.Any())
    {
        Console.WriteLine("Failed to fetch: " + string.Join(", ", failed));
    }
});

```

### Resolver Usage in Isolation

For unit testing or custom integrations:

```csharp
var url = "https://example.com/ss.json";
var servers = await OnlineConfigResolver.GetOnline(url);

// Verify source tagging
foreach (var s in servers)
{
    Debug.Assert(s.group == url);
}

```

## Summary

- The SIP008 online configuration update mechanism operates through three layers: ViewModel for UI, Controller for orchestration, and Resolver for HTTP/JSON handling.
- `ShadowsocksController.UpdateOnlineConfigInternal` performs atomic replacement of servers by filtering existing entries using the source URL tag.
- `OnlineConfigResolver.GetOnline` uses `HttpClient` for transport and recursively searches JSON structures to extract valid SIP008 server objects.
- Each fetched server receives a `group` property containing its source URL, enabling precise deduplication during merge operations.
- The mechanism supports both single-source updates and bulk refreshes across multiple configured URLs, with comprehensive error aggregation for failed endpoints.

## Frequently Asked Questions

### What is SIP008 and why does it matter for shadowsocks-windows?

SIP008 is a standardized JSON format for distributing shadowsocks server configurations via HTTP endpoints. It matters because it allows users and organizations to centrally manage server lists—adding, removing, or rotating endpoints—without requiring manual configuration file edits on each client. The shadowsocks-windows implementation supports this standard while maintaining flexibility for non-standard JSON structures through recursive parsing.

### How does the mechanism handle malformed or non-standard JSON responses?

The `OnlineConfigResolverEx` class employs recursive token searching via `SearchJToken`, `SearchJObject`, and `SearchJArray` methods to traverse arbitrarily nested JSON. Rather than expecting a fixed schema, it identifies any object containing the four mandatory fields (`server`, `server_port`, `password`, `method`) and converts those to `Server` objects. This approach accommodates providers who may wrap server arrays in metadata objects or use different key nesting levels.

### What happens when a remote URL is temporarily unreachable?

If `HttpClient.GetStringAsync` throws an exception or the resolver fails to parse the response, the error propagates back to the calling method. For single updates (`UpdateOnlineConfig`), the failure surfaces to the UI as a boolean false result. For bulk updates (`UpdateAllOnlineConfig`), failed URLs accumulate in a `List<string>` that returns to the caller, allowing the application to continue processing other sources while logging specific failures.

### How does the system prevent duplicate servers when refreshing configurations?

The system prevents duplicates through source-based tagging. When fetching, each server receives its origin URL in the `group` property. During merging, `UpdateOnlineConfigInternal` filters the existing configuration with `.Where(c => c.group != url)`, effectively removing all previous entries from that specific source before concatenating the new list. This ensures complete replacement rather than accumulation, keeping the server list synchronized with the remote authoritative source.