# How to Implement Style Transfer and Neural Rendering with the AILIA-Models Repository

> Implement style transfer and neural rendering easily with AILIA-Models ONNX. Run ready-to-use models via CLI or embed them in Python using the AILIA SDK.

- Repository: [axinc-ai/ailia-models](https://github.com/axinc-ai/ailia-models)
- Tags: tutorial
- Published: 2026-02-26

---

**The AILIA-Models repository provides ready-to-run ONNX implementations of AdaIN-based style transfer and NeRF/TripoSR-based neural rendering that you can execute via CLI or embed in Python applications using the AILIA SDK.**

The **axinc-ai/ailia-models** repository ships a comprehensive collection of computer-vision demos built on the **AILIA SDK**, a lightweight wrapper around ONNX and TensorRT runtimes. To implement style transfer and neural rendering, you will leverage pre-trained ONNX models for **Adaptive Instance Normalization (AdaIN)** artistic stylization and **Neural Radiance Fields (NeRF)** or **TripoSR** for 3D view synthesis and mesh extraction.

## Implementing Style Transfer with Adaptive Instance Normalization

The style transfer pipeline in [`style_transfer/adain/adain.py`](https://github.com/axinc-ai/ailia-models/blob/main/style_transfer/adain/adain.py) implements the AdaIN algorithm to blend the content of one image with the artistic style of another using VGG-based feature extraction and alpha-controlled blending.

### Architecture and Core Components

The implementation relies on three core components working sequentially:

- **VGG-19 Encoder** – The `adain-vgg.onnx` model extracts 512-channel feature maps from both content and style images.
- **AdaIN Layer** – Defined in [`style_transfer/adain/adain_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/style_transfer/adain/adain_utils.py), the `adaptive_instance_normalization` function aligns per-channel means and standard deviations of content features to match style features.
- **Decoder Network** – The `adain-decoder.onnx` model reconstructs the final RGB image from the normalized features.

The complete forward pass lives in the `style_transfer` function within [`adain.py`](https://github.com/axinc-ai/ailia-models/blob/main/adain.py):

```python
def style_transfer(vgg, decoder, content, style, alpha=args.alpha):
    # 1. Encode content and style

    content_f = vgg.predict(content.astype(np.float32))
    style_f   = vgg.predict(style)
    # 2. Adaptive Instance Normalization

    feat = adain_utils.adaptive_instance_normalization(content_f, style_f)
    # 3. Alpha blending controls stylization strength

    feat = feat * alpha + content_f * (1 - alpha)
    # 4. Decode to image space

    return decoder.predict(feat)

```

Images are pre-processed using `image_utils.load_image` to 512×512 resolution with `normalize_type='255'`, converting to AILIA's NCHW tensor layout. The helper `model_utils.check_and_download_models` automatically pulls ONNX weights from Google Cloud Storage when missing.

### Running AdaIN Style Transfer

Execute stylization via CLI:

```bash
python style_transfer/adain/adain.py \
    -i path/to/content.jpg \
    -t path/to/style.jpg \
    -a 0.8

```

For programmatic use, load the VGG and decoder networks via the AILIA SDK and call the utility functions directly:

```python
import ailia
import cv2
import numpy as np
from style_transfer.adain import adain_utils, adain

# Initialize models

vgg = ailia.Net('adain-vgg.onnx.prototxt', 'adain-vgg.onnx')
decoder = ailia.Net('adain-decoder.onnx.prototxt', 'adain-decoder.onnx')

def stylize(content_path, style_path, alpha=0.5):
    # Pre-process to 512x512, normalize to [0,255]

    content = adain.load_image(content_path, (512, 512), normalize_type='255', gen_input_ailia=True)
    style = adain.load_image(style_path, (512, 512), normalize_type='255', gen_input_ailia=True)
    
    # Extract features and blend

    content_f = vgg.predict(content.astype(np.float32))
    style_f = vgg.predict(style)
    feat = adain_utils.adaptive_instance_normalization(content_f, style_f)
    feat = feat * alpha + content_f * (1 - alpha)
    output = decoder.predict(feat)
    
    # Convert back to BGR for OpenCV

    img = cv2.cvtColor(output[0].transpose(1, 2, 0), cv2.COLOR_RGB2BGR)
    return np.clip(img * 255 + 0.5, 0, 255).astype(np.uint8)

result = stylize('photo.jpg', 'painting.jpg', alpha=0.7)
cv2.imwrite('output.png', result)

```

## Implementing Neural Rendering with NeRF and TripoSR

The neural rendering implementations in `neural_rendering/nerf/` and `neural_rendering/tripo_sr/` provide two distinct approaches: continuous volumetric view synthesis and explicit 3D mesh extraction from single images.

### NeRF Volume Rendering

The NeRF implementation in [`neural_rendering/nerf/nerf.py`](https://github.com/axinc-ai/ailia-models/blob/main/neural_rendering/nerf/nerf.py) renders novel views by querying an MLP that predicts RGB color and volume density for 3D coordinates.

**Key implementation details:**

- **MLP Network** – `nerf.opt.onnx` predicts **rgb + density** for any 3D point and view direction.
- **Positional Encoding** – The `get_embedder` function in [`utils_nerf.py`](https://github.com/axinc-ai/ailia-models/blob/main/utils_nerf.py) applies sinusoidal embeddings to input coordinates, enabling high-frequency detail capture.
- **Ray Marching** – The `render` function in [`utils_nerf.py`](https://github.com/axinc-ai/ailia-models/blob/main/utils_nerf.py) casts camera rays, samples `N_samples` points along each ray (configurable via `--N_samples`), and integrates colors using the volume rendering equation in `raw2outputs`.

Run NeRF rendering from the command line:

```bash
python neural_rendering/nerf/nerf.py \
    --datadir ./data/nerf_llff_data/ \
    --angle 0 \
    --render_factor 4 \
    -i dummy_input.png \
    -o rendered.png

```

### TripoSR 3D Mesh Reconstruction

TripoSR generates textured 3D meshes from single RGB images using a two-stage neural field architecture implemented in [`neural_rendering/tripo_sr/tripo_sr.py`](https://github.com/axinc-ai/ailia-models/blob/main/neural_rendering/tripo_sr/tripo_sr.py).

**Pipeline stages:**

1. **Encoder** – `TripoSR.onnx` predicts a latent scene code from a foreground-masked 512×512 image.
2. **Neural Field Decoder** – `TripoSR_decoder.onnx` queries density and color at arbitrary 3D points.
3. **Mesh Extraction** – The `TSR` wrapper class provides `extract_mesh(scene_codes, resolution)` which runs marching-cubes on the density field.

Pre-processing includes optional background removal via `remove_background` and foreground resizing via `resize_foreground` from [`util.py`](https://github.com/axinc-ai/ailia-models/blob/main/util.py).

Execute mesh generation:

```bash
python neural_rendering/tripo_sr/tripo_sr.py \
    -i portrait.png \
    -o portrait.obj \
    --mc-resolution 256 \
    --foreground-ratio 0.85

```

For Python integration:

```python
import cv2
import numpy as np
import ailia
from neural_rendering.tripo_sr.util import remove_background, resize_foreground, TSR

# Load encoder and decoder

net_enc = ailia.Net('TripoSR.onnx.prototxt', 'TripoSR.onnx')
net_dec = ailia.Net('TripoSR_decoder.onnx.prototxt', 'TripoSR_decoder.onnx')

# Initialize TSR wrapper with rendering parameters

model = TSR(net_dec, radius=0.87, feature_reduction='concat',
            density_activation='exp', density_bias=-1.0,
            num_samples_per_ray=128)
model.renderer.set_chunk_size(8192)

def mesh_from_image(img_path, resolution=256):
    img = cv2.imread(img_path, cv2.IMREAD_UNCHANGED)
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    
    # Pre-processing

    img = remove_background(img)
    img = resize_foreground(img, 0.85) / 255.0
    img = np.stack([[cv2.resize(img, (512, 512), cv2.INTER_LINEAR)]], axis=0)
    
    # Encode and extract

    scene_code = np.array(net_enc.run(img))[0]
    mesh = model.extract_mesh(scene_code, resolution=resolution)[0]
    return mesh

mesh = mesh_from_image('input.png')
mesh.export('output.obj')

```

## Summary

- **Style Transfer** in AILIA-Models uses **AdaIN** in [`style_transfer/adain/adain.py`](https://github.com/axinc-ai/ailia-models/blob/main/style_transfer/adain/adain.py) to align VGG-19 feature statistics between content and style images, with alpha blending controlling the stylization strength.
- **Neural Rendering** offers two modes: **NeRF** (`neural_rendering/nerf/`) for volumetric view synthesis via ray marching and positional encoding, and **TripoSR** (`neural_rendering/tripo_sr/`) for explicit mesh extraction using encoder-decoder neural fields with marching-cubes.
- All models are automatically downloaded via `model_utils.check_and_download_models` and run through the unified AILIA SDK interface, supporting both CLI workflows and programmatic Python integration.

## Frequently Asked Questions

### What is the difference between AdaIN and other style transfer methods in AILIA-Models?

The **AdaIN** implementation in `style_transfer/adain/` performs real-time arbitrary style transfer by aligning feature statistics in VGG-19 space, requiring no training for new styles. Alternative implementations like `animeganv2` or `beauty_gan` use dedicated GAN architectures trained for specific aesthetic outputs, while `pix2pixHD` targets high-resolution image-to-image translation with paired training data.

### How does NeRF rendering performance scale with resolution?

The NeRF renderer in [`neural_rendering/nerf/utils_nerf.py`](https://github.com/axinc-ai/ailia-models/blob/main/neural_rendering/nerf/utils_nerf.py) casts rays per pixel, so rendering time scales quadratically with image dimensions. Use the `--render_factor` flag to downsample rays for fast previews (e.g., factor 4 reduces resolution by 4×), or adjust `--N_samples` to trade quality for speed by reducing point sampling along each ray.

### Can TripoSR generate meshes from multiple input images?

The current TripoSR implementation in [`tripo_sr.py`](https://github.com/axinc-ai/ailia-models/blob/main/tripo_sr.py) processes single images through the `TripoSR.onnx` encoder to produce scene codes. While the architecture supports batched inference, multi-view fusion would require custom modifications to aggregate multiple scene codes before calling `extract_mesh` in the `TSR` class.

### Do I need GPU acceleration to run these models?

While the AILIA SDK supports CPU inference via ONNX Runtime, both style transfer and neural rendering involve heavy matrix operations (512-channel convolutions for AdaIN, millions of MLP queries for NeRF). GPU acceleration is strongly recommended for interactive performance, especially when extracting high-resolution meshes with `--mc-resolution` above 256 in TripoSR.