onnxruntime
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
Discover how ONNX Runtime TrainingSession manages checkpoint state and training progress with SetStateTensors and SaveCheckpoint APIs for seamless pause and resume.
ONNX Runtime Model Partitioning Strategy Across Execution Providers: A Deep DiveExplore ONNX Runtime's model partitioning strategy. Learn how it intelligently assigns model subgraphs to optimal execution providers for efficient hardware utilization.
How to Implement Custom Operators in ONNX Runtime Using the Custom Operator APILearn to implement custom operators in ONNX Runtime using ORT custom operator API. Define metadata, implement compute logic, and register your domain for seamless integration.
Optimizing ONNX Runtime Inference Latency: 8 Best Practices for Sub-Millisecond ServingOptimize ONNX Runtime inference latency with 8 best practices for sub millisecond serving. Learn graph optimizations, IO binding, hardware acceleration, and threading for faster AI.
How to Resolve Kernel Registration Conflicts with Multiple Execution Providers in ONNX RuntimeResolve ONNX Runtime kernel registration conflicts with multiple execution providers. Learn how to ensure unique types, avoid version overlaps, and register providers correctly.
C API vs C++ API for ONNX Runtime: Key Differences and When to Use EachExplore ONNX Runtime C API vs C++ API differences. Understand manual memory management in C API and RAII benefits in the C++ API for efficient model deployment.
How Mixed Precision Training Works in ONNX Runtime: Architecture and ImplementationDiscover how ONNX Runtime implements mixed precision training. Learn about graph transformations, FP16/BF16 casts, stability graphs, and loss scaling for efficient training.
ONNX Runtime Kernel Registry Architecture: Provider-Aware Operator ResolutionExplore the ONNX Runtime kernel registry architecture. Understand provider-aware operator resolution with KernelRegistry, KernelRegistryManager, and KernelDefBuilder for efficient graph execution.
How to Handle Operator Compatibility When Migrating Between ONNX Runtime VersionsLearn to manage operator compatibility when migrating ONNX Runtime versions. Discover how ONNX Runtime ensures backward compatibility and controls opset validation for smooth transitions.
ONNX Runtime Threading Model: Thread-Pool-Based Parallel Execution ExplainedDiscover ONNX Runtime's thread-pool-based threading model for parallel execution. Learn how it leverages Eigen or OpenMP for efficient operator parallelization.
How GraphTransformerMgr Applies Optimizations in Stages in ONNX RuntimeDiscover how GraphTransformerMgr optimizes ONNX Runtime graphs in three stages: registration, multi-pass execution, and state inspection. Learn about its advanced pipeline.
ONNX Runtime Profiling Tools: A Complete Guide to Performance AnalysisMaster ONNX Runtime performance analysis with our complete guide. Learn to use profiling tools for detailed session, operator, and execution provider timing data to optimize your models.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →