# How to Implement Pose Estimation Using ST-GCN or MoveNet in Ailia Models

> Implement pose estimation with ST-GCN for action recognition or MoveNet for real-time keypoint detection using ailia-models Python scripts. Get automatic model download, preprocessing, and visualization.

- Repository: [axinc-ai/ailia-models](https://github.com/axinc-ai/ailia-models)
- Tags: how-to-guide
- Published: 2026-02-26

---

**You can implement pose estimation using ST-GCN for action recognition from skeleton sequences or MoveNet for real-time 2D keypoint detection by running the respective Python scripts in the ailia-models repository, which handle model downloading, preprocessing, and visualization automatically.**

The ailia-models repository by axinc-ai provides production-ready implementations for two distinct pose estimation approaches. Whether you need to classify human actions from video sequences using spatial-temporal graph convolutional networks or detect body keypoints in real-time using single-shot detectors, the repository offers complete pipelines with automatic model management and hardware acceleration via the Ailia SDK.

## Understanding the Two Pose Estimation Pipelines

### ST-GCN for Action Recognition

**ST-GCN** (Spatial-Temporal Graph Convolutional Network) tracks 2D skeletons over time and classifies actions from the resulting joint sequences. This pipeline combines a pose estimator backend with a graph neural network to recognize human activities like walking, dancing, or sports movements from video input.

### MoveNet for Real-Time Keypoint Detection

**MoveNet** is a single-shot 2D keypoint detector that identifies 17 body parts in still images or video streams. Available in **Thunder** (256×256 input, higher accuracy) and **Lightning** (192×192 input, faster inference) variants, MoveNet provides frame-by-frame pose estimation without temporal tracking.

## Implementing ST-GCN Pose Estimation

### Architecture Overview

The ST-GCN pipeline in [`action_recognition/st_gcn/st_gcn.py`](https://github.com/axinc-ai/ailia-models/blob/main/action_recognition/st_gcn/st_gcn.py) processes video through several distinct stages:

1. **Input handling** – Video frames are read using OpenCV.
2. **2D joint detection** – The system loads a pose estimator via `ailia.PoseEstimator`, supporting backends including OpenPose, PyOpenPose, or LW-Human-Pose (**[st_gcn.py lines 70-78]**).
3. **Coordinate normalization** – Detected keypoints undergo `pose_postprocess` to normalize coordinates to the range [0, 1] (**[st_gcn.py lines 104-108]**).
4. **Temporal tracking** – The `naive_pose_tracker` function in [`st_gcn_util.py`](https://github.com/axinc-ai/ailia-models/blob/main/st_gcn_util.py) stitches frames into a fixed-length skeleton tensor with shape **3×T×V×M** (channels × time × vertices × persons).
5. **Graph convolution** – The ST-GCN ONNX model (`st_gcn.onnx`) runs via `ailia.Net` (**[st_gcn.py lines 62-64]**), producing action classification logits.
6. **Label generation** – The `postprocess` function (**[st_gcn.py lines 111-138]**) converts raw outputs into voting labels and per-frame label sequences using the `KINETICS_LABEL` mapping from [`st_gcn_labels.py`](https://github.com/axinc-ai/ailia-models/blob/main/st_gcn_labels.py).
7. **Visualization** – The `stgcn_visualize` function in [`st_gcn_util.py`](https://github.com/axinc-ai/ailia-models/blob/main/st_gcn_util.py) renders skeletons, heatmaps, and action labels onto the output video.

### Key Files and Functions

| File | Role |
|------|------|
| [`action_recognition/st_gcn/st_gcn.py`](https://github.com/axinc-ai/ailia-models/blob/main/action_recognition/st_gcn/st_gcn.py) | CLI driver containing `recognize_offline()` and `recognize_realtime()` functions, argument parsing, and model initialization. |
| [`action_recognition/st_gcn/st_gcn_util.py`](https://github.com/axinc-ai/ailia-models/blob/main/action_recognition/st_gcn/st_gcn_util.py) | Contains `naive_pose_tracker` for temporal skeleton assembly and `stgcn_visualize` for rendering results. |
| [`action_recognition/st_gcn/st_gcn_labels.py`](https://github.com/axinc-ai/ailia-models/blob/main/action_recognition/st_gcn/st_gcn_labels.py) | Defines `KINETICS_LABEL` dictionary mapping class indices to human-readable action names from the Kinetics-400 dataset. |
| `pose_estimation/openpose` or `pose_estimation/lw_human_pose` | Pose estimation backends used by ST-GCN for 2D joint detection. |

### Running ST-GCN

To run ST-GCN action recognition on a video file using the default OpenPose backend:

```bash
python3 action_recognition/st_gcn/st_gcn.py \
    --video skateboarding.mp4 \
    --arch openpose

```

Available architecture options for the pose estimator include `openpose`, `pyopenpose`, and `lw_human_pose`.

To process a video offline and save the rendered output instead of displaying it in a window:

```bash
python3 action_recognition/st_gcn/st_gcn.py \
    --input skateboarding.mp4 \
    --savepath result.mp4

```

**Important CLI arguments:**

| Flag | Description |
|------|-------------|
| `--fps` | Target frames per second for realtime processing mode. |
| `--arch` | Pose estimation backend selection (`openpose`, `pyopenpose`, `lw_human_pose`). |
| `--img-save` | Save individual frames as PNG files instead of video output. |
| `--env_id` | Ailia runtime environment selector for GPU/CPU acceleration. |

## Implementing MoveNet Pose Estimation

### Architecture Overview

The MoveNet implementation in [`pose_estimation/movenet/movenet.py`](https://github.com/axinc-ai/ailia-models/blob/main/pose_estimation/movenet/movenet.py) follows a streamlined single-shot detection pipeline:

1. **Model acquisition** – The `check_and_download_models` function automatically fetches `movenet_thunder.onnx` or `movenet_lightning.onnx` along with their prototxt files from Google Cloud Storage (**[movenet.py lines 88-90]**).
2. **Input preprocessing** – The `crop_and_padding` function in [`movenet_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/movenet_utils.py) pads input images to square aspect ratio, resizes them to the model's expected resolution (256×256 for **Thunder**, 192×192 for **Lightning**), and returns normalized tensors.
3. **Inference** – The ONNX model executes via `ailia.Net` or `onnxruntime`, outputting a heatmap tensor of shape **(1, 1, 17, 3)** representing 17 body keypoints with x, y coordinates and confidence scores.
4. **Coordinate restoration** – Post-processing converts normalized coordinates back to the original image space, accounting for padding offsets applied during preprocessing.
5. **Visualization** – The `draw_prediction_on_image` function in [`movenet_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/movenet_utils.py) (**[lines 71-115]**) overlays circles and skeletal lines for all 17 keypoints onto the original image.

For video processing, MoveNet maintains a **crop region** that follows the person across frames using `init_crop_region` and `determine_crop_region`. This tracking reduces jitter and allows consistent input sizing without processing the full frame each time.

### Key Files and Functions

| File | Role |
|------|------|
| [`pose_estimation/movenet/movenet.py`](https://github.com/axinc-ai/ailia-models/blob/main/pose_estimation/movenet/movenet.py) | CLI driver handling argument parsing, model initialization, and main inference loops for images and video. |
| [`pose_estimation/movenet/movenet_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/pose_estimation/movenet/movenet_utils.py) | Contains `crop_and_padding`, `crop_and_resize` for preprocessing, and `draw_prediction_on_image` for visualization. |
| [`util/model_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/util/model_utils.py) | Shared helper functions for automatic model downloading and verification. |
| [`util/image_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/util/image_utils.py) | Wrapper functions for image loading used across both pipelines. |

### Running MoveNet

To detect pose keypoints in a single image using the default Thunder model:

```bash
python3 pose_estimation/movenet/movenet.py \
    --input input.jpg \
    --savepath result.png

```

To process a video using the lightweight Lightning variant for faster inference:

```bash
python3 pose_estimation/movenet/movenet.py \
    --video input.mp4 \
    --model_variant lightning \
    --savepath output.mp4

```

**Essential CLI arguments:**

| Flag | Description |
|------|-------------|
| `-i/--input` | Path to input image file(s). Multiple files can be specified. |
| `-v/--video` | Path to video file or `0` to use webcam input. |
| `-m/--model_variant` | Model selection: `thunder` (default, 256×256) or `lightning` (192×192). |
| `-o/--onnx` | Force usage of `onnxruntime` instead of the Ailia SDK. |
| `--benchmark` | Run warm-up iterations and report average inference time. |

## Comparing ST-GCN and MoveNet

While both pipelines perform pose estimation, they serve different use cases:

- **ST-GCN** requires a **temporal sequence** and classifies **actions** (e.g., "walking," "waving") using graph convolutions on skeleton data. It depends on an external pose estimator (OpenPose or LW-Human-Pose) to generate the initial 2D joints.

- **MoveNet** performs **single-frame keypoint detection** optimized for **real-time applications**. It outputs 17 body part locations directly without requiring a separate pose estimator, making it ideal for immediate coordinate extraction or low-latency tracking.

Choose ST-GCN when you need to understand *what action* a person is performing over time, and MoveNet when you need *where* body parts are located in individual frames.

## Summary

- **ST-GCN** combines 2D pose estimation with temporal tracking via `naive_pose_tracker` to create skeleton tensors (3×T×V×M) for action classification using graph convolutional networks.
- **MoveNet** provides single-shot 2D keypoint detection for 17 body parts through two model variants: **Thunder** (256×256, higher accuracy) and **Lightning** (192×192, faster inference).
- Both implementations in the ailia-models repository handle automatic model downloading via `check_and_download_models`, support CPU and GPU acceleration through the Ailia SDK, and include OpenCV-based visualization utilities.
- ST-GCN requires specifying a pose estimation backend (`--arch openpose` or `lw_human_pose`), while MoveNet operates as a standalone detector without external pose dependencies.

## Frequently Asked Questions

### What is the difference between ST-GCN and MoveNet in the ailia-models repository?

ST-GCN is an **action recognition** pipeline that takes sequences of 2D skeletons over time and classifies the performed action using graph convolutions, while MoveNet is a **pose estimation** model that detects 17 body keypoints in individual images or video frames without temporal analysis. ST-GCN requires a separate pose estimator backend to generate skeleton data, whereas MoveNet performs detection directly.

### How do I choose between the Thunder and Lightning variants of MoveNet?

Choose **Thunder** (256×256 input resolution) when accuracy is critical and computational resources are available, as it provides more precise keypoint localization. Choose **Lightning** (192×192 input resolution) for applications requiring higher frame rates or running on edge devices with limited processing power, as it offers significantly faster inference with slightly reduced accuracy.

### What pose estimation backends can I use with ST-GCN?

The ST-GCN implementation in [`action_recognition/st_gcn/st_gcn.py`](https://github.com/axinc-ai/ailia-models/blob/main/action_recognition/st_gcn/st_gcn.py) supports three pose estimation backends specified via the `--arch` argument: `openpose` (standard OpenPose implementation), `pyopenpose` (Python OpenPose bindings), and `lw_human_pose` (lightweight human pose estimator). These backends generate the 2D joint coordinates required by the `naive_pose_tracker` to construct the skeleton tensor for action classification.

### Do I need to manually download the ONNX models before running the scripts?

No, both ST-GCN and MoveNet implementations automatically handle model downloading through the `check_and_download_models` function found in [`util/model_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/util/model_utils.py). On first run, the scripts download the required ONNX files and prototxt configurations from Google Cloud Storage to a local `models` directory, then load them using `ailia.Net` for inference.