PreProcessors and PostProcessors in SymbolicAI: A Complete Guide to LLM Request Handling

PreProcessors and PostProcessors in SymbolicAI separate prompt construction from response parsing, allowing fine-grained control over LLM inputs and outputs through modular transformation pipelines.

The SymbolicAI (symai) package implements a dual-stage processing architecture that isolates prompt engineering from response handling. By leveraging PreProcessors and PostProcessors, developers can transform arguments before they reach the engine and clean or extract data from raw LLM outputs without writing ad-hoc string manipulation code for every call.

What Are PreProcessors in SymbolicAI?

PreProcessors handle the transformation of user inputs into the exact text format required by the LLM engine. Each PreProcessor receives the Argument object containing the prompt, context, and metadata, then returns a string (or None) that gets concatenated into the final prompt.

In symai/pre_processors.py, the base PreProcessor class defines the interface:

class PreProcessor:
    def __call__(self, argument):
        # Transform argument.prop.prompt and return a string

        pass

Built-in PreProcessors include:

• JsonPreProcessor – Adds [JSON_BEGIN] markers before JSON payloads to guide the model output format.
• EqualsPreProcessor – Formats binary equality tests as a == b => strings for logical comparisons.
• PromptPreProcessor – Appends $> markers for interactive prompt contexts.

What Are PostProcessors in SymbolicAI?

PostProcessors clean, extract, or deserialize the raw response returned by the LLM engine. Each PostProcessor receives the engine's output string along with the original Argument object, enabling context-aware transformations.

The base class in symai/post_processors.py implements:

class PostProcessor:
    def __call__(self, response, argument):
        # Process response string and return transformed value

        pass

Common PostProcessors include:

• StripPostProcessor – Removes surrounding whitespace and quotation marks from raw text outputs.
• JsonTruncatePostProcessor – Extracts JSON objects delimited by [JSON_BEGIN] and [JSON_END] markers, handling markdown-wrapped responses.
• CodeExtractPostProcessor – Pulls fenced code blocks (e.g., python ... ) from markdown-formatted LLM outputs.

How the ProcessorPipeline Orchestrates PreProcessors and PostProcessors

The ProcessorPipeline class in symai/processor.py manages the execution order of processors. It automatically detects whether it is running in "pre" or "post" mode by inspecting the first item in the processor list:

if isinstance(self.processors[0], PreProcessor):
    # Build the prompt: concatenate all pre-processor outputs

    result = ""
    for proc in self.processors:
        result += proc(argument) or ""
    return result
else:
    # Post-process the engine output: chain transformations

    result = response
    for proc in self.processors:
        result = proc(result, argument)
    return result

This architecture allows both processor types to be optional. A Function expression can specify pre_processors, post_processors, both, or neither, providing flexible control over the LLM interaction lifecycle.

Practical Examples of Using PreProcessors and PostProcessors

Creating a Custom PreProcessor

You can inject system instructions by subclassing PreProcessor and modifying the prompt before it reaches the engine:

from symai.pre_processors import PreProcessor
from symai.components import Function

class SystemInstructionPreProcessor(PreProcessor):
    def __call__(self, argument):
        instruction = "You are a helpful assistant. "
        return instruction + str(argument.prop.prompt)

my_func = Function(
    prompt="Summarize the following text:",
    pre_processors=[SystemInstructionPreProcessor()],
    post_processors=[StripPostProcessor()],
)

result = my_func("Artificial intelligence is transforming industries.")

Extracting JSON with PostProcessors

When the LLM returns markdown-wrapped JSON, use JsonTruncatePostProcessor to extract the clean object:

from symai.components import Function
from symai.post_processors import JsonTruncatePostProcessor

json_func = Function(
    prompt="Return the data as JSON.",
    post_processors=[JsonTruncatePostProcessor()],
)

raw = json_func("Give me the capital of France.")

# raw => {"capital": "Paris"}

Chaining Multiple Processors

The ProcessorPipeline supports chaining multiple transformations for complex parsing workflows:

from symai.processor import ProcessorPipeline
from symai.post_processors import StripPostProcessor, CodeExtractPostProcessor

pipeline = ProcessorPipeline([
    StripPostProcessor(),
    CodeExtractPostProcessor()
])

raw_output = "`def hello():\n    return 'hi'`"
code = pipeline(raw_output, None)

# Result: def hello():\n    return 'hi'

Summary

• PreProcessors in SymbolicAI transform the Argument object into formatted prompt strings before sending requests to the LLM engine, living in symai/pre_processors.py.
• PostProcessors clean and extract data from raw engine responses, converting them into usable Python objects, defined in symai/post_processors.py.
• The ProcessorPipeline in symai/processor.py automatically detects processor type by checking isinstance(self.processors[0], PreProcessor) and executes the appropriate concatenation or transformation chain.
• Both processor types are optional and can be combined in Function expressions to create modular, reusable LLM interaction patterns without ad-hoc string manipulation.

Frequently Asked Questions

What is the difference between PreProcessors and PostProcessors in SymbolicAI?

PreProcessors handle the request phase, receiving the Argument object and returning strings that build the final prompt sent to the LLM. PostProcessors handle the response phase, receiving the raw engine output and the original Argument, then returning cleaned or deserialized data. They operate in symai/pre_processors.py and symai/post_processors.py respectively, and are orchestrated by ProcessorPipeline in symai/processor.py.

How do I create a custom PreProcessor in SymbolicAI?

Subclass PreProcessor from symai/pre_processors.py and implement __call__(self, argument). This method receives the Argument object containing argument.prop.prompt and should return a string to be concatenated into the final prompt. Pass an instance of your custom class to the pre_processors parameter when initializing a Function or Expression.

Can I use multiple PreProcessors and PostProcessors together?

Yes. The ProcessorPipeline accepts lists of processors. For pre-processing, it concatenates the output of each PreProcessor in order. For post-processing, it chains the output of each PostProcessor sequentially, passing the result of one as input to the next. This allows complex transformations like stripping whitespace then extracting JSON, or adding multiple formatting markers to a prompt.

Where are the built-in processors defined in the SymbolicAI repository?

Built-in PreProcessors are defined in symai/pre_processors.py, including JsonPreProcessor, EqualsPreProcessor, and PromptPreProcessor. Built-in PostProcessors live in symai/post_processors.py, including StripPostProcessor, JsonTruncatePostProcessor, and CodeExtractPostProcessor. The orchestration logic resides in symai/processor.py within the ProcessorPipeline class.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}