How the SIP008 Online Configuration Update Mechanism Fetches and Merges Remote Servers

The SIP008 online configuration update mechanism downloads remote server configurations via HTTP GET, recursively parses JSON to extract valid server objects, tags them with their source URL to enable source-based deduplication, and atomically replaces existing entries from the same source before persisting the merged server list.

The SIP008 online configuration update mechanism in shadowsocks-windows enables dynamic synchronization of proxy servers from remote JSON endpoints. Implemented across three distinct architectural layers—ViewModel, Controller, and Resolver—this system handles everything from user interface interactions to low-level JSON parsing. The implementation ensures that remote server lists remain synchronized with their authoritative sources while preserving user preferences and local server entries.

Three-Layer Architecture

The feature follows a strict separation of concerns across three layers:

• UI / ViewModel Layer: Collects URLs from users and initiates updates via Reactive commands. Key file: shadowsocks-csharp/ViewModels/OnlineConfigViewModel.cs.
• Controller Layer: Orchestrates the download, parsing, merging, and persistence logic. Key file: shadowsocks-csharp/Controller/ShadowsocksController.cs.
• Resolver Layer: Performs HTTP requests and recursively extracts server objects from arbitrary JSON structures. Key file: shadowsocks-csharp/Controller/Service/OnlineConfigResolver.cs.

User-Initiated Update Flow

The ViewModel exposes three Reactive commands—Update, UpdateAll, and Add—bound directly to UI controls. When a user selects a source and clicks Update, the command executes:

Update = ReactiveCommand.CreateFromTask(
    () => _controller.UpdateOnlineConfig(SelectedSource), 
    canUpdateCopyRemove
);

This command returns a boolean indicating success, which the ViewModel surfaces to the user via message boxes.

Controller Orchestration

The ShadowsocksController class manages the core logic for fetching and merging configurations through three primary methods.

UpdateOnlineConfig Method

The UpdateOnlineConfig(string url) method (lines 77-92) preserves user context while refreshing remote data:

1. Captures the currently selected server identifier.
2. Invokes UpdateOnlineConfigInternal(url) to perform the actual update.
3. Restores the user's server selection in the merged list via _config.index.
4. Persists the configuration to disk using SaveConfig.

UpdateOnlineConfigInternal Method

The internal implementation (lines 65-75) handles the atomic replacement logic:

var onlineServer = await OnlineConfigResolver.GetOnline(url);
// Remove existing entries from this source
_config.configs = _config.configs
    .Where(c => c.group != url)
    .Concat(onlineServer)
    .ToList();
Configuration.SortByOnlineConfig(_config.configs);

This method returns the count of newly added servers and ensures that stale entries from the same URL are completely removed before inserting fresh data.

UpdateAllOnlineConfig Method

For bulk operations, UpdateAllOnlineConfig() (lines 94-114) iterates over all URLs stored in _config.onlineConfigSource, invoking UpdateOnlineConfigInternal for each source. The method aggregates failed URLs into a list, allowing the UI to display specific error information for problematic endpoints.

Remote Fetch and JSON Parsing

The OnlineConfigResolver class handles the network layer and flexible JSON extraction.

HTTP GET Execution

The GetOnline(string url) method (lines 14-23) obtains an HttpClient instance from the main controller and performs the request:

string server_json = await httpClient.GetStringAsync(url);
var servers = server_json.GetServers();
foreach (var server in servers)
{
    server.group = url;  // Tag with source for deduplication
}
return servers;

Recursive JSON Extraction

The OnlineConfigResolverEx.GetServers(this string json) extension method uses Newtonsoft.Json to parse arbitrarily nested JSON structures. The recursive search (SearchJObject, SearchJArray, SearchJToken) traverses the entire object graph, identifying any object containing the mandatory SIP008 fields:

• server
• server_port
• password
• method

Each matching object converts to a Server model via obj.ToObject<Server>(), enabling support for various provider-specific JSON schemas without rigid structure requirements.

Server Merging Strategy

The SIP008 online configuration update mechanism employs a source-aware merging strategy to maintain data integrity.

Deduplication by Source URL

Before adding freshly-fetched servers, the controller executes:

.Where(c => c.group != url)

This LINQ filter removes all existing entries whose group property equals the URL being refreshed. Since the resolver tags every fetched server with its origin URL (server.group = url), the system can precisely identify and replace only the entries originating from that specific remote source, leaving manually configured servers untouched.

Sorting and Persistence

After concatenation, the merged collection passes through Configuration.SortByOnlineConfig, which orders servers according to SIP008 specifications (typically by remarks or group fields). The sorted list becomes the authoritative server set, and SaveConfig() atomically writes the updated configuration to disk.

Implementation Examples

Triggering Updates from the UI

// In OnlineConfigView.xaml.cs (auto-generated bindings)
// User selects a source and clicks "Update"
await viewModel.Update.Execute();

// Result is a bool: true = success, false = failure

Programmatic Bulk Updates

For background synchronization on startup:

// Program.cs – background task launched after controller starts
Task.Run(async () =>
{
    await Task.Delay(TimeSpan.FromSeconds(10));
    
    List<string> failed = await MainController.UpdateAllOnlineConfig();
    
    if (failed.Any())
    {
        Console.WriteLine("Failed to fetch: " + string.Join(", ", failed));
    }
});

Resolver Usage in Isolation

For unit testing or custom integrations:

var url = "https://example.com/ss.json";
var servers = await OnlineConfigResolver.GetOnline(url);

// Verify source tagging
foreach (var s in servers)
{
    Debug.Assert(s.group == url);
}

Summary

• The SIP008 online configuration update mechanism operates through three layers: ViewModel for UI, Controller for orchestration, and Resolver for HTTP/JSON handling.
• ShadowsocksController.UpdateOnlineConfigInternal performs atomic replacement of servers by filtering existing entries using the source URL tag.
• OnlineConfigResolver.GetOnline uses HttpClient for transport and recursively searches JSON structures to extract valid SIP008 server objects.
• Each fetched server receives a group property containing its source URL, enabling precise deduplication during merge operations.
• The mechanism supports both single-source updates and bulk refreshes across multiple configured URLs, with comprehensive error aggregation for failed endpoints.

Frequently Asked Questions

What is SIP008 and why does it matter for shadowsocks-windows?

SIP008 is a standardized JSON format for distributing shadowsocks server configurations via HTTP endpoints. It matters because it allows users and organizations to centrally manage server lists—adding, removing, or rotating endpoints—without requiring manual configuration file edits on each client. The shadowsocks-windows implementation supports this standard while maintaining flexibility for non-standard JSON structures through recursive parsing.

How does the mechanism handle malformed or non-standard JSON responses?

The OnlineConfigResolverEx class employs recursive token searching via SearchJToken, SearchJObject, and SearchJArray methods to traverse arbitrarily nested JSON. Rather than expecting a fixed schema, it identifies any object containing the four mandatory fields (server, server_port, password, method) and converts those to Server objects. This approach accommodates providers who may wrap server arrays in metadata objects or use different key nesting levels.

What happens when a remote URL is temporarily unreachable?

If HttpClient.GetStringAsync throws an exception or the resolver fails to parse the response, the error propagates back to the calling method. For single updates (UpdateOnlineConfig), the failure surfaces to the UI as a boolean false result. For bulk updates (UpdateAllOnlineConfig), failed URLs accumulate in a List<string> that returns to the caller, allowing the application to continue processing other sources while logging specific failures.

How does the system prevent duplicate servers when refreshing configurations?

The system prevents duplicates through source-based tagging. When fetching, each server receives its origin URL in the group property. During merging, UpdateOnlineConfigInternal filters the existing configuration with .Where(c => c.group != url), effectively removing all previous entries from that specific source before concatenating the new list. This ensures complete replacement rather than accumulation, keeping the server list synchronized with the remote authoritative source.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}