How ncnn Handles Multiple Inputs and Outputs: A Deep Dive into `input_indexes()` and `output_indexes()`

ncnn identifies network inputs and outputs by analyzing blob connectivity during model loading, exposing the resulting indexes through input_indexes() and output_indexes() vectors that map to the Extractor API.

Tencent's ncnn library treats neural networks as directed graphs of blobs (tensors) and layers, enabling support for arbitrary numbers of inputs and outputs. When you load a model, ncnn automatically discovers which blobs serve as entry points and which are terminal outputs, storing these relationships in internal index vectors accessible through the public API.

Understanding ncnn's Network Representation

At its core, ncnn represents every neural network as a bipartite graph where data flows from layer to layer through blobs. Each blob acts as an edge connecting a producer layer to one or more consumer layers. This design allows ncnn to support complex architectures with multiple input streams (such as multi-modal models) and multiple output heads (such as detection or multi-task networks).

The Net class encapsulates this graph structure, while the private NetPrivate implementation maintains the actual storage for layer and blob metadata.

How ncnn Identifies Input and Output Blobs

During the model loading phase, ncnn executes NetPrivate::update_input_output_indexes() to classify blobs based on their connectivity patterns.

Input Blob Detection

Input blobs are identified by scanning for layers of type LayerType::Input. For every Input layer encountered, ncnn appends the indexes of its top blobs (outputs) to the input_blob_indexes vector. This occurs in src/net.cpp at lines 861-874, where the code iterates through the layer list and extracts the top blob indices from each input layer.

Output Blob Detection

Output blobs are terminal tensors that have a producer (some layer generates them) but no consumers (no subsequent layer reads them). The detection logic in src/net.cpp lines 880-884 checks each blob for the condition producer != -1 && consumer == -1. When satisfied, the blob index is appended to output_blob_indexes.

The Internal Implementation of input_indexes() and output_indexes()

The public API exposes these discovered indexes through two const accessor methods defined in src/net.cpp at lines 2090-2097:

const std::vector<int>& Net::input_indexes() const
{
    return d->input_blob_indexes;
}

const std::vector<int>& Net::output_indexes() const
{
    return d->output_blob_indexes;
}

These methods return references to the internal vectors stored in the NetPrivate instance (d). The vectors contain blob indexes (0-based integers) that correspond to positions in the global blob list maintained by the network.

You can find the corresponding declarations in src/net.h at lines 134-138, which define the interface contract for these accessors.

Practical Usage: Working with Multiple Inputs and Outputs in ncnn

The index vectors bridge the Net definition and the Extractor execution context. When running inference, you use these indexes to bind input data and retrieve results.

C++ API Example

#include <ncnn/net.h>

int main()
{
    ncnn::Net net;
    net.load_param("multimodel.param");
    net.load_model("multimodel.bin");

    // Retrieve the discovered indexes
    const std::vector<int>& in_idx = net.input_indexes();
    const std::vector<int>& out_idx = net.output_indexes();

    // Create extractor
    ncnn::Extractor ex = net.create_extractor();

    // Feed multiple inputs using their blob indexes
    ncnn::Mat input0 = ncnn::Mat::from_pixels_resize(image0.data, ncnn::Mat::PIXEL_BGR, 
                                                      image0.w, image0.h, 224, 224);
    ncnn::Mat input1 = ncnn::Mat::from_pixels_resize(image1.data, ncnn::Mat::PIXEL_BGR, 
                                                      image1.w, image1.h, 224, 224);
    
    ex.input(in_idx[0], input0);
    ex.input(in_idx[1], input1);

    // Extract multiple outputs
    ncnn::Mat output0, output1;
    ex.extract(out_idx[0], output0);
    if (out_idx.size() > 1) {
        ex.extract(out_idx[1], output1);
    }

    return 0;
}

C API Example

The C interface exposes these indexes through ncnn_net_get_input_index and ncnn_net_get_output_index:

ncnn_net_t net = ncnn_net_create();
ncnn_net_load_param(net, "model.param");
ncnn_net_load_model(net, "model.bin");

int input_count = ncnn_net_get_input_index_count(net);
int output_count = ncnn_net_get_output_index_count(net);

ncnn_extractor_t ex = ncnn_extractor_create(net);

// Feed all inputs
for (int i = 0; i < input_count; ++i) {
    int idx = ncnn_net_get_input_index(net, i);
    ncnn_extractor_input_index(ex, idx, input_mats[i]);
}

// Extract all outputs
for (int i = 0; i < output_count; ++i) {
    int idx = ncnn_net_get_output_index(net, i);
    ncnn_extractor_extract_index(ex, idx, &output_mats[i]);
}

Python Bindings

The Python API mirrors the C++ interface through net.input_indexes() and net.output_indexes():

import ncnn

net = ncnn.Net()
net.load_param('model.param')
net.load_model('model.bin')

print(f"Input indexes: {net.input_indexes()}")    # e.g., [0, 1]

print(f"Output indexes: {net.output_indexes()}")  # e.g., [7, 8]

ex = net.create_extractor()
ex.input(net.input_indexes()[0], mat0)
ex.input(net.input_indexes()[1], mat1)

out0 = ex.extract(net.output_indexes()[0])
out1 = ex.extract(net.output_indexes()[1])

Summary

• ncnn treats networks as directed graphs where blobs are tensors and layers are operations.
• Input blobs are identified from Input layer top blobs during update_input_output_indexes() in src/net.cpp.
• Output blobs are terminal tensors with a producer but no consumer, detected at lines 880-884 of src/net.cpp.
• The input_indexes() and output_indexes() methods expose blob indexes as std::vector<int> references.
• These indexes enable the Extractor API to bind data to specific inputs and retrieve specific outputs in C++, C, and Python.

Frequently Asked Questions

How do I determine the number of inputs and outputs in an ncnn model?

Call net.input_indexes() or net.input_indexes() and check the size() of the returned vector. Each element represents one input or output blob. In the C API, use ncnn_net_get_input_index_count() and ncnn_net_get_output_index_count().

What is the difference between blob indexes and layer indexes in ncnn?

Blob indexes identify tensors (data) in the network's blob list, while layer indexes identify computation nodes in the layer list. The input_indexes() and output_indexes() methods return blob indexes because you bind data to tensors, not to layers. Layer indexes are used internally for graph traversal and optimization.

Can input and output indexes change between model versions?

Yes, the specific integer values in input_indexes() and output_indexes() depend on the order in which blobs are defined in the .param file. If you add, remove, or reorder layers between model versions, the blob indexes may shift. Always query the vectors at runtime rather than hardcoding index values.

How does the Python API expose input_indexes() and output_indexes()?

The Python bindings in python/src/main.cpp map the C++ methods directly to Python properties. Calling net.input_indexes() returns a Python list of integers corresponding to the C++ std::vector<int>. Similarly, net.output_indexes() provides the output blob indexes, allowing you to use them with extractor.input() and extractor.extract().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}