How to Export TensorFlow Models for Serving: Complete Guide to SavedModel and Frozen Graph Export

Exporting TensorFlow models for serving creates a stand-alone inference artifact (SavedModel, frozen graph, or checkpoint files) that can be deployed to TensorFlow Serving, TensorFlow Lite, or other inference runtimes.

The tensorflow/models repository provides two distinct export pipelines depending on your use case: the Object Detection API for TF 1.x/2.x models and the Official Vision exporter for modern TF 2.x architectures. Both workflows generate a saved_model/ directory containing the serialized computation graph and trained weights, along with signature definitions that specify input and output tensor shapes.

Understanding TensorFlow Model Export Pipelines

The repository implements two primary pathways to export TensorFlow models for serving:

• Object-Detection Pipeline (research/object_detection/exporter_main_v2.py): Supports legacy and current object detection models, exporting both frozen graphs (.pb files) and SavedModels.
• Official Vision Pipeline (official/vision/serving/export_saved_model.py): Modern exporter for classification, detection, segmentation, and video models using TF 2.x native APIs.

Both pipelines follow a common architecture: parse configuration, load checkpoints, build inference graphs with specific input signatures, and serialize to the SavedModel format.

Exporting Object Detection Models (TF 1.x/2.x)

Entry Point and Configuration

The Object Detection API uses research/object_detection/exporter_main_v2.py as its primary entry point. This script requires a pipeline configuration file (protobuf pipeline_pb2.TrainEvalPipelineConfig) that defines the model architecture, loss, optimizer, and post-processing parameters.

Key parameters include:

• --input_type: Specifies the serving signature (image_tensor, encoded_image_string_tensor, or tf_example)
• --trained_checkpoint_dir: Path to the checkpoint directory containing ckpt-* files
• --output_directory: Destination for the exported artifacts

Export Command and Parameters

Execute the export from the repository root:

INPUT_TYPE=image_tensor
PIPELINE_CONFIG_PATH=research/object_detection/configs/ssd_mobilenet_v2_fpnlite_640x640_coco.config
TRAINED_CHECKPOINT_PREFIX=training/checkpoint/ckpt-10000
EXPORT_DIR=exported/ssd_mobilenet_v2

python research/object_detection/exporter_main_v2.py \
  --input_type=${INPUT_TYPE} \
  --pipeline_config_path=${PIPELINE_CONFIG_PATH} \
  --trained_checkpoint_dir=${TRAINED_CHECKPOINT_PREFIX} \
  --output_directory=${EXPORT_DIR}

The exporter_lib_v2.export_inference_graph function handles the underlying graph construction, checkpoint restoration, and serialization.

Output Structure

The export directory contains:

• saved_model/: TensorFlow SavedModel directory ready for serving
• frozen_inference_graph.pb: Frozen graph format for legacy deployments
• checkpoint/: Checkpoint files of the exported model
• pipeline.config: Copy of the configuration file for reproducibility

Exporting Official Vision Models (TF 2.x)

Using the export_saved_model CLI

For modern vision architectures (RetinaNet, Mask R-CNN, image classification, segmentation), use official/vision/serving/export_saved_model.py. This script accepts an experiment configuration (YAML/JSON) and exports a SavedModel with optional TPU compatibility.

Required arguments:

• --experiment: Experiment name registered in the factory (e.g., retinanet_resnetfpn_coco)
• --checkpoint_path: Path to the trained checkpoint
• --export_dir: Output directory for the SavedModel

Example execution:

EXPERIMENT=retinanet_resnetfpn_coco
CHECKPOINT_PATH=training/checkpoint/ckpt-20000
EXPORT_DIR=exported/retinanet

export_saved_model \
  --experiment=${EXPERIMENT} \
  --checkpoint_path=${CHECKPOINT_PATH} \
  --export_dir=${EXPORT_DIR} \
  --batch_size=2 \
  --input_image_size=640,640 \
  --input_type=image_tensor

ExportModule Architecture

The exporter instantiates task-specific ExportModule subclasses:

• ClassificationModule: For image classification models
• DetectionModule: For object detection models
• SegmentationModule: For semantic segmentation models

These modules wrap the model's inference_* functions and expose them as serving signatures. The export_saved_model_lib.export_inference_graph function (lines 98-126) orchestrates the module creation and serialization.

Loading and Serving Exported Models

Once exported, load the SavedModel using TensorFlow's standard APIs:

import tensorflow as tf

# Path to the SavedModel produced by either exporter

saved_model_dir = "/tmp/exported/ssd_mobilenet_v2/saved_model"

# Load the model

model = tf.saved_model.load(saved_model_dir)

# Get the default serving signature (depends on input_type used during export)

infer = model.signatures["serving_default"]

# Prepare a single image tensor (uint8, shape [1, H, W, 3])

image = tf.io.decode_image(tf.io.read_file("sample.jpg"))
image = tf.expand_dims(image, axis=0)

# Run inference

output = infer(image)

print("Detections:", output["detection_boxes"], output["detection_scores"])

The signature key ("serving_default" or a custom key if --function_keys was used) must match the input_type selected during export.

Key Implementation Details

Signature Generation

Input signatures are generated in official/vision/serving/export_utils.py. The get_image_input_signatures function creates different TensorFlow SignatureDefs based on the export configuration:

• image_tensor: [batch, None, None, 3] uint8 tensor for raw image arrays
• image_bytes/encoded_image_string_tensor: 1-D string tensor for JPEG/PNG encoded images
• tf_example: 1-D string tensor for serialized TFExample protobufs
• tflite: Fixed [1, H, W, 3] float32 tensor for mobile deployment

TPU Compatibility

When exporting with --add_tpu_function_alias, the exporter adds a tpu_candidate alias pointing to the appropriate inference function (lines 49-64 in export_saved_model.py). This allows the SavedModel to run on TPU accelerators without modification.

Model Flops and Parameters Logging

If --log_model_flops_and_params is set, the exporter runs a dummy forward pass to compute FLOPs and writes model_flops.txt and model_params.txt (lines 82-117). This helps with deployment capacity planning.

Summary

• TensorFlow models for serving are exported as SavedModel directories containing frozen graphs, checkpoints, and signature definitions compatible with TensorFlow Serving.
• Object Detection API uses research/object_detection/exporter_main_v2.py to export TF 1.x/2.x detection models with support for image_tensor, encoded_image_string_tensor, and tf_example input types.
• Official Vision exporter (official/vision/serving/export_saved_model.py) handles modern TF 2.x architectures (classification, detection, segmentation) using task-specific ExportModule classes.
• Signature generation varies by input type: raw tensors for image_tensor, string tensors for encoded images or TFExamples, and fixed-size float tensors for TFLite.
• Post-export loading uses tf.saved_model.load() with the appropriate signature key (typically "serving_default") to run inference in Python or deploy to TensorFlow Serving.

Frequently Asked Questions

What is the difference between SavedModel and frozen graph export?

The SavedModel format (directory containing saved_model.pb) is the modern standard for TensorFlow serving, supporting multiple signatures, asset management, and version compatibility with both TensorFlow 1.x and 2.x. The frozen graph (frozen_inference_graph.pb) is a legacy single-file format where variables are converted to constants, suitable for older deployment targets or embedded systems that cannot handle the directory structure. The Object Detection exporter generates both formats by default, while the Official Vision exporter focuses on SavedModel.

Which input_type should I use for TensorFlow Serving?

Choose image_tensor if your client sends raw image arrays (uint8 tensors), which is common for in-process Python serving. Use encoded_image_string_tensor (or image_bytes in the vision exporter) if clients send JPEG or PNG encoded strings via REST or gRPC, as this moves image decoding into the TensorFlow graph. Select tf_example if your data pipeline already serializes examples as TFRecord format. The vision exporter also supports tflite for mobile deployment, which requires fixed input sizes.

How do I export a model for TPU inference?

When using the Official Vision exporter, add the --add_tpu_function_alias flag to export_saved_model. This creates a tpu_candidate alias in the SavedModel signatures that points to the TPU-compatible inference function, allowing the model to run on Cloud TPUs or TPU-enabled GKE clusters without graph modification. For Object Detection models, ensure you use the TF 2.x exporter (exporter_main_v2.py) with a TPU-compatible pipeline config, though TPU support varies by specific model architecture.

Can I export TensorFlow 1.x models using the official vision exporter?

No. The official/vision/serving/export_saved_model.py exporter is designed exclusively for TensorFlow 2.x models using the Keras API and tf.Module architecture. If you have a legacy TensorFlow 1.x model (such as older Object Detection models trained with the TF 1.x API), use research/object_detection/exporter_main_v2.py instead, which maintains backward compatibility with TF 1.x checkpoint formats while exporting to the TF 2.x SavedModel format.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}