# How to Handle Unicode and CJK Output Encoding on Windows with Console.OutputEncoding

> Learn how OfficeCLI ensures correct Unicode and CJK output encoding on Windows by managing Console.OutputEncoding. Get reliable character display in your terminal applications.

- Repository: [OfficeAI/OfficeCLI](https://github.com/iofficeai/OfficeCLI)
- Tags: how-to-guide
- Published: 2026-08-01

---

**OfficeCLI guarantees correct Unicode and CJK character display on Windows by temporarily switching `Console.OutputEncoding` to UTF-8 without BOM when writing to an interactive terminal, then automatically restoring the original code page on process exit.**

When building cross-platform CLI tools in .NET, Windows consoles frequently corrupt Chinese, Japanese, and Korean (CJK) characters by defaulting to legacy code pages like CP936 or CP932. The OfficeCLI repository solves this through a robust encoding management strategy implemented in [`src/officecli/Program.cs`](https://github.com/iOfficeAI/OfficeCLI/blob/main/src/officecli/Program.cs) that detects console attachment, switches to UTF-8, and guarantees cleanup.

## Detecting Interactive Console Output

Before modifying encoding, OfficeCLI determines whether the process has a real console attached or if output is being piped to a file or another process. In [`Program.cs`](https://github.com/iOfficeAI/OfficeCLI/blob/main/Program.cs), the code validates both standard output and standard error streams:

```csharp
if (!Console.IsOutputRedirected || !Console.IsErrorRedirected)
{
    // Console is interactive, safe to modify encoding
}

```

This check prevents encoding changes when running under redirection scenarios like `officecli ... > output.txt` or piping to `| grep pattern`, where the receiving process expects the original encoding.

## Storing the Original Windows Code Page

To avoid permanently altering the user's terminal environment—a common source of "mojibake" in subsequent commands—the implementation immediately caches the current encoding:

```csharp
var previous = Console.OutputEncoding;

```

This preserves the system default (often CP850, CP936, or CP1252 depending on Windows locale) so it can be restored after the CLI finishes execution.

## Switching to UTF-8 Without BOM

When the existing code page differs from UTF-8 (code page **65001**), OfficeCLI instantiates a `UTF8Encoding` with the BOM explicitly disabled to prevent byte-order mark artifacts in console output:

```csharp
var utf8NoBom = new System.Text.UTF8Encoding(encoderShouldEmitUTF8Identifier: false);

if (previous.CodePage != System.Text.Encoding.UTF8.CodePage)
{
    Console.OutputEncoding = utf8NoBom;
    // Subsequent Console.Write* calls emit UTF-8 bytes
}

```

Setting `encoderShouldEmitUTF8Identifier: false` ensures that the first character written does not spit out invisible bytes that might confuse terminal emulators or downstream parsers.

## Restoring Original Encoding on Process Exit

The most critical step prevents the console from remaining locked to UTF-8 after the application terminates. OfficeCLI attaches restoration logic to both normal exits and cancellation events:

```csharp
void Restore(object? sender, EventArgs e)
{
    try { Console.OutputEncoding = previous; } 
    catch { /* console already gone */ }
}

AppDomain.CurrentDomain.ProcessExit += Restore;
Console.CancelKeyPress += Restore;

```

This `Restore` handler catches all exceptions silently because the console buffer may already be detached if the user closes the terminal window aggressively.

## Handling Redirected Output Streams

When `Console.IsOutputRedirected` returns true, directly setting `Console.OutputEncoding` has no effect on the stream writer. Instead, OfficeCLI replaces `Console.Out` and `Console.Error` with `StreamWriter` instances using the same UTF-8 encoding. As demonstrated in [`ResidentServer.cs`](https://github.com/iOfficeAI/OfficeCLI/blob/main/ResidentServer.cs) (lines 219-229), this ensures redirected bytes reach their destination unchanged regardless of the system code page:

```csharp
// Pattern used in ResidentServer.cs for error stream management
var utf8Writer = new StreamWriter(Console.OpenStandardError(), utf8NoBom);
utf8Writer.AutoFlush = true;
Console.SetError(utf8Writer);

```

## Implementing Unicode Support in Your Own .NET CLI

You can adopt this pattern in any .NET console application by wrapping the encoding logic at the entry point:

```csharp
using System;
using System.Text;

class Program
{
    static void Main()
    {
        var utf8NoBom = new UTF8Encoding(encoderShouldEmitUTF8Identifier: false);
        
        if (!Console.IsOutputRedirected && !Console.IsErrorRedirected)
        {
            var original = Console.OutputEncoding;
            
            if (original.CodePage != Encoding.UTF8.CodePage)
            {
                Console.OutputEncoding = utf8NoBom;
                
                AppDomain.CurrentDomain.ProcessExit += (_, __) =>
                {
                    try { Console.OutputEncoding = original; } catch { }
                };
            }
        }

        Console.WriteLine("日本語テキスト and 中文内容 display correctly");
    }
}

```

## Summary

- **Detection matters**: Always check `Console.IsOutputRedirected` before modifying encoding to avoid breaking pipelines.
- **UTF-8 without BOM**: Use `new UTF8Encoding(encoderShouldEmitUTF8Identifier: false)` to prevent invisible prefix bytes.
- **Cleanup is mandatory**: Store the original `Console.OutputEncoding` and restore it via `ProcessExit` and `CancelKeyPress` handlers.
- **Redirects need wrappers**: Replace `Console.Out` with explicit `StreamWriter` instances when output is piped.
- **Source files**: The complete implementation lives in [`src/officecli/Program.cs`](https://github.com/iOfficeAI/OfficeCLI/blob/main/src/officecli/Program.cs) (lines 5-30) with additional stream handling in [`ResidentServer.cs`](https://github.com/iOfficeAI/OfficeCLI/blob/main/ResidentServer.cs) (lines 219-229) according to the iOfficeAI/OfficeCLI source code.

## Frequently Asked Questions

### Why does Windows show garbled text with CJK characters?

Windows consoles default to legacy code pages (CP936 for Chinese GBK, CP932 for Japanese Shift-JIS) that only support a limited glyph subset. When a .NET application outputs UTF-8 bytes but the console interprets them using these legacy code pages, multi-byte sequences split incorrectly producing "mojibake" gibberish.

### What is Windows code page 65001?

Code page **65001** is the Windows identifier for UTF-8. When you assign `Console.OutputEncoding` to a UTF-8 encoding instance, the Win32 console API switches to this code page, enabling proper rendering of Unicode characters including CJK glyphs and emoji.

### Why disable the UTF-8 BOM for console output?

The **Byte Order Mark (BOM)** is a three-byte prefix (`0xEF 0xBB 0xBF`) that signals UTF-8 encoding in files, but in console output it manifests as invisible characters or garbled symbols at the start of the first printed line. Disabling it with `encoderShouldEmitUTF8Identifier: false` ensures clean terminal output.

### How do I handle Unicode output when piping to another program?

When output is redirected (piped), use `new StreamWriter(Console.OpenStandardOutput(), utf8Encoding)` to wrap the raw stream. This bypasses the console code page entirely and writes UTF-8 bytes directly to the pipe, ensuring the receiving process receives uncorrupted Unicode data regardless of the terminal's current encoding.