How to Perform Regex Replace in Python with `re.sub` (CPython Implementation)

CPython implements regex replace through the re.sub function in Lib/re/__init__.py, which provides a Python wrapper around the high-performance C routine sre_sub for substituting patterns in strings.

Performing a regex replace is a fundamental text-processing operation in Python. In the CPython repository, this functionality is exposed through the re module's sub function, offering both simple string substitutions and advanced callable-based replacements. Understanding how regex replace works at the source level helps developers optimize pattern matching performance and leverage advanced features like dynamic replacement logic.

The re.sub Method Signature and Parameters

According to the CPython source in Lib/re/__init__.py, the re.sub function accepts the following parameters:

re.sub(pattern, repl, string, count=0, flags=0)

• pattern: The regular expression pattern to search for, provided as a string or a compiled Pattern object.
• repl: The replacement string, or a callable that receives a match object and returns the replacement text.
• string: The input text where substitutions will be performed.
• count: The maximum number of replacements to make; defaults to 0 for unlimited replacements.
• flags: Optional regex flags (such as re.IGNORECASE) that modify matching behavior.

When repl is a function, CPython invokes it for every match, enabling dynamic replacement logic based on the specific match content.

How Regex Replace Works Internally in CPython

The implementation follows a clear path from the Python API down to the C engine. In Lib/re/__init__.py, re.sub acts as a thin wrapper that handles argument validation and compiles string patterns into Pattern objects when necessary.

The heavy lifting occurs in the C routine sre_sub, located in Objects/unicodeobject.c. This function iterates through the input string, identifies matches using the compiled regex engine, and constructs the resulting string by concatenating unchanged text segments with their replacements. When processing callable replacements, sre_sub executes the Python function for each match object before appending the returned value to the output buffer.

Practical Code Examples for Regex Replace

The following examples demonstrate common regex replace patterns using CPython's re module.

Static String Replacement

Replace specific word boundaries with fixed text:

import re

text = "The rain in Spain falls mainly on the plain."
result = re.sub(r"\bain\b", "ANE", text)
print(result)

# Output: The rANE in SpANE falls mANEly on the plANE.

Limiting Replacements with the count Parameter

Restrict the number of substitutions to the first two occurrences:

result = re.sub(r"\bain\b", "ANE", text, count=2)
print(result)

# Output: The rANE in SpANE falls mainly on the plain.

Dynamic Replacement Using Callables

Transform matched text dynamically by passing a function that processes each match object:

def swap_case(m):
    return m.group(0).swapcase()

result = re.sub(r"\b\w+\b", swap_case, "Hello World")
print(result)

# Output: hELLO wORLD

Case-Insensitive Replacement with Flags

Perform case-insensitive matching using the flags parameter:

result = re.sub(r"python", "Py", "I love PYTHON and python.", flags=re.IGNORECASE)
print(result)

# Output: I love Py and Py.

Key Source Files in CPython

Understanding the regex replace implementation requires examining these critical files in the CPython repository:

• Lib/re/__init__.py: Contains the public Python API, including the sub and subn functions that validate inputs and delegate to the C engine.
• Lib/re/_constants.py: Defines internal constants and the PatternError exception used throughout the regex subsystem.
• Objects/unicodeobject.c: Implements the core sre_sub routine that performs the actual string construction and replacement logic at the C level.

Summary

• re.sub in Lib/re/__init__.py is the standard method for performing regex replace operations in CPython.
• The function accepts patterns, replacement strings or callables, and optional count and flags parameters.
• Internally, the C routine sre_sub in Objects/unicodeobject.c handles the performance-critical string manipulation.
• Using callable replacements enables dynamic, context-aware substitution logic for complex text processing tasks.

Frequently Asked Questions

What is the difference between re.sub and re.subn in CPython?

While re.sub returns only the modified string, re.subn (also defined in Lib/re/__init__.py) returns a tuple containing the new string and the number of substitutions actually performed. Both functions utilize the same underlying sre_sub C implementation.

Can I use a lambda function as the replacement in re.sub?

Yes, any callable that accepts a match object and returns a string works as the repl argument. The CPython implementation in Objects/unicodeobject.c invokes this callable for every match discovered during the scan, making lambdas ideal for concise dynamic replacements.

How does the count parameter affect performance in re.sub?

Setting count to a positive number allows sre_sub to stop scanning after the specified number of matches, potentially reducing processing time on large strings. When count=0 (the default), the engine scans the entire input string for all possible matches.

Where is the regex pattern compilation handled when using re.sub?

If you pass a string pattern rather than a compiled Pattern object, re.sub automatically compiles it using the internal compilation logic in Lib/re/__init__.py before invoking the C engine. For repeated operations on the same pattern, pre-compiling with re.compile avoids redundant compilation overhead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}