# Soup | Alpamys Makazhan | Knowledge Base | Instagit

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

GitHub Stars: 2k

Repository: https://github.com/MakazhanAlpamys/Soup

---

## Articles

### [How to Contribute to the Soup CLI Project: A Complete Guide for Developers](/MakazhanAlpamys/Soup/how-to-contribute-to-the-soup-cli-project)

Learn how to contribute to the Soup CLI project. This guide covers forking, setting up your environment, running tests, linting with ruff, and submitting pull requests for developers.

- Tags: how-to-guide
- Published: 2026-09-06

### [What Is the Autopilot Feature in Soup CLI? A Complete Guide to Zero-Configuration Fine‑Tuning](/MakazhanAlpamys/Soup/what-is-the-autopilot-feature-in-soup-cli)

Discover the Autopilot feature in Soup CLI. This zero-configuration assistant automatically fine-tunes your models by analyzing data, models, and hardware for optimal hyperparameters. No manual tuning needed.

- Tags: deep-dive
- Published: 2026-09-06

### [How to Use the Soup CLI Model Registry for Lineage Tracking](/MakazhanAlpamys/Soup/how-to-use-the-model-registry-in-soup-cli-for-lineage-tracking)

Learn to use the Soup CLI model registry for robust lineage tracking. Discover how to record training runs, configurations, artifacts, and link derived models with `soup registry` subcommands.

- Tags: how-to-guide
- Published: 2026-09-06

### [What Is the Purpose of the `cans` Directory in Soup CLI: A Complete Guide to Soup Cans](/MakazhanAlpamys/Soup/what-is-the-purpose-of-the-cans-directory-in-soup-cli)

Discover the purpose of the cans directory in Soup CLI. Learn how Soup Cans packages capture, transport, and replay training iterations for reproducible results.

- Tags: how-to-guide
- Published: 2026-09-06

### [How Soup CLI Tracks Experiments and Metrics: A Deep Dive into the SQLite-Based ExperimentTracker](/MakazhanAlpamys/Soup/how-does-soup-cli-track-experiments-and-metrics)

Discover how Soup CLI tracks experiments and metrics with its SQLite-based ExperimentTracker. Learn about run metadata, training metrics, and cost estimates stored locally.

- Tags: deep-dive
- Published: 2026-09-06

### [How to Specify Multi-Dataset Interleaving in Soup Configuration](/MakazhanAlpamys/Soup/how-to-specify-multi-dataset-interleaving-in-soup-configuration)

Learn how to specify multi-dataset interleaving in Soup configuration. Easily combine datasets using concat under over or probs strategies for efficient data loading.

- Tags: how-to-guide
- Published: 2026-09-06

### [What Data Formats Does Soup CLI Support for Fine-Tuning?](/MakazhanAlpamys/Soup/what-data-formats-does-soup-cli-support-for-fine-tuning)

Soup CLI supports 15+ data formats for fine-tuning including Alpaca, ShareGPT, LLaVA, and DPO. Easily ingest JSONL, CSV, Parquet, and text files for efficient model training.

- Tags: api-reference
- Published: 2026-09-06

### [How to Use the MLX Backend for Apple Silicon Training with Soup](/MakazhanAlpamys/Soup/how-to-use-the-mlx-backend-for-apple-silicon-training-with-soup)

Unlock Apple Silicon training with Soup's MLX backend. Easily fine-tune models by configuring your soup.yaml and installing optional dependencies. Start optimizing your ML workflows today.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Use the Unsloth Backend for Faster Training in Soup](/MakazhanAlpamys/Soup/how-to-use-the-unsloth-backend-for-faster-training-in-soup)

Accelerate Soup training with Unsloth backend. Easily configure your soup.yaml and install unsloth for 4-bit quantization and 2x faster GPU training.

- Tags: how-to-guide
- Published: 2026-09-06

### [Soup CLI Quantization Methods: Complete Guide to GPTQ, AWQ, HQQ, and More](/MakazhanAlpamys/Soup/what-quantization-methods-are-supported-by-soup-cli)

Explore Soup CLI's ten quantization methods including GPTQ, AWQ, HQQ, and more. Optimize your models efficiently with this complete guide to supported quantization techniques.

- Tags: how-to-guide
- Published: 2026-09-06

### [How Soup CLI Handles Automatic Batch Sizing: A Technical Deep Dive](/MakazhanAlpamys/Soup/how-does-soup-cli-handle-automatic-batch-sizing)

Learn how Soup CLI automatically sizes batches using static formulas or GPU memory probes. Discover the best method for your hardware configuration.

- Tags: deep-dive
- Published: 2026-09-06

### [Soup CLI LoRA Configurations: Complete Guide to Fine-Tuning with Low-Rank Adaptation](/MakazhanAlpamys/Soup/what-are-the-different-lora-configurations-supported-by-soup-cli)

Explore Soup CLI LoRA configurations including DoRA, VeRA, and OLoRA. Fine-tune models efficiently with this complete guide to Low-Rank Adaptation.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Perform Direct Preference Optimization (DPO) with the Soup CLI](/MakazhanAlpamys/Soup/how-to-perform-direct-preference-optimization-dpo-with-soup-cli)

Learn how to perform Direct Preference Optimization DPO with the Soup CLI. Streamline LoRA, quantization, and iterative training using a simple YAML config.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Perform Supervised Fine-Tuning (SFT) with Soup CLI: A Complete Guide](/MakazhanAlpamys/Soup/how-to-perform-supervised-fine-tuning-sft-with-soup-cli)

Master Supervised Fine-Tuning SFT with the Soup CLI. Follow our guide to easily train models using streaming data, LoRA, and checkpointing.

- Tags: how-to-guide
- Published: 2026-09-06

### [How Soup CLI Integrates with HuggingFace Transformers, PEFT, and TRL](/MakazhanAlpamys/Soup/how-does-soup-cli-integrate-with-huggingface-transformers-peft-and-trl)

Discover how Soup CLI seamlessly integrates HuggingFace Transformers, PEFT, and TRL. Explore its layered architecture for efficient model loading, dynamic LoRA configuration, and TRL trainer compatibility.

- Tags: how-to-guide
- Published: 2026-09-06

### [Benefits of Using Rich Console Output in Soup: 7 Architectural Advantages for Modern CLI Development](/MakazhanAlpamys/Soup/what-are-the-benefits-of-using-rich-console-output-in-soup)

Discover the 7 architectural advantages of Soup's Rich Console Output for modern CLI development. Enhance user experience with color, structure, and progress indicators.

- Tags: architecture
- Published: 2026-09-06

### [How Soup CLI's Lazy-Import Architecture Improves Startup Performance and Memory Usage](/MakazhanAlpamys/Soup/how-does-soup-cli-lazy-import-architecture-improve-performance)

Soup CLI's lazy import architecture slashes startup time and memory usage by deferring heavy dependencies like PyTorch until needed. Optimize your CLI performance now.

- Tags: internals
- Published: 2026-09-06

### [How to Use the `soup version` Command in the Soup CLI](/MakazhanAlpamys/Soup/how-to-use-the-soup-version-command)

Learn how to use the soup version command to check your Soup CLI package version. This simple command displays the current version through the Rich console interface.

- Tags: how-to-guide
- Published: 2026-09-06

### [What Is the Role of Pydantic in Soup CLI's Configuration?](/MakazhanAlpamys/Soup/what-is-the-role-of-pydantic-in-soup-cli-configuration)

Discover how Pydantic enhances Soup CLI's configuration with type safety and validation for soup.yaml files, ensuring robust and error-free execution.

- Tags: internals
- Published: 2026-09-06

### [How to Configure LLM Fine-Tuning Using YAML Files in Soup: A Complete Guide](/MakazhanAlpamys/Soup/how-to-configure-llm-fine-tuning-using-yaml-files-in-soup)

Learn to configure LLM fine-tuning with Soup's YAML file. This guide explains parsing YAML into Pydantic models for robust, validated workflows. Get started now!

- Tags: how-to-guide
- Published: 2026-09-06

### [Supported LLM Fine-Tuning Tasks in Soup CLI: A Complete Guide](/MakazhanAlpamys/Soup/what-are-the-supported-llm-fine-tuning-tasks-in-soup-cli)

Explore 12+ LLM fine-tuning tasks in Soup CLI including SFT DPO ORPO PPO KTO GRPO unlearning text-to-speech distillation and MoE routing. Master your models with our comprehensive guide.

- Tags: how-to-guide
- Published: 2026-09-06

### [Soup CLI GPU Detection and Utilization: A Deep Dive into the Multi-Backend Architecture](/MakazhanAlpamys/Soup/how-does-soup-cli-handle-gpu-detection-and-utilization)

Explore Soup CLI's multi-backend architecture for GPU detection and utilization. It unifies device info across CUDA, MPS, and MLX for optimized distributed training and memory allocation.

- Tags: deep-dive
- Published: 2026-09-06

### [Soup CLI System Requirements: Python Version, GPU, and VRAM Specifications](/MakazhanAlpamys/Soup/what-are-the-system-requirements-for-soup-cli)

Discover Soup CLI system requirements including Python 3.10+ GPU VRAM needs for fine-tuning or CPU inference. Optimize your AI workflow now.

- Tags: getting-started
- Published: 2026-09-06

### [How to Install the Soup CLI for LLM Fine-Tuning](/MakazhanAlpamys/Soup/how-to-install-soup-cli-for-llm-fine-tuning)

Install the Soup CLI for LLM fine-tuning using pip or pipx. Learn how to add optional dependencies for a streamlined setup and begin training your models efficiently.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Troubleshoot VRAM Issues with Soup Doctor: A Complete Guide](/MakazhanAlpamys/Soup/troubleshoot-vram-issues-soup-doctor)

Troubleshoot VRAM issues with Soup Doctor. Learn how this tool predicts GPU memory needs and prevents out-of-memory crashes for smoother training.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Implement Model Steering with Soup Steer: CLI and Python API Guide](/MakazhanAlpamys/Soup/implement-model-steering-soup-steer)

Learn to implement model steering with Soup Steer using its CLI and Python API. This guide covers building artifacts, loading them, and installing forward hooks for CAA, ITI, and RepE.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Use A/B Testing to Compare Model Variants with the Soup Framework](/MakazhanAlpamys/Soup/how-to-use-ab-testing-compare-model-variants)

Learn to use A/B testing for model variants with the Soup framework. Implement mSPRT for early stopping and guaranteed error control to optimize your models.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Set Up Compliance Templates (HIPAA, SOC 2, EU AI Act) with Soup](/MakazhanAlpamys/Soup/set-up-compliance-templates-hipaa-soc2-eu-ai-act)

Easily set up HIPAA, SOC 2, and EU AI Act compliance templates with Soup. Generate pre-configured workflows and execute regulatory commands for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Use Soup's Autopilot Feature for Zero-Config Training](/MakazhanAlpamys/Soup/how-to-use-autopilot-zero-config-training)

Effortlessly train models with Soup's autopilot. Discover how this zero-config feature analyzes your setup to generate a complete training configuration automatically.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Configure Multi‑GPU Training with DeepSpeed or FSDP in Soup](/MakazhanAlpamys/Soup/configure-multi-gpu-training-deepspeed-fsdp)

Learn how to configure multi-GPU training with DeepSpeed or FSDP in Soup. Effortlessly scale your models using simple flags for automatic config and GPU count adaptation.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Use Soup's TUI for Interactive Training Monitoring: A Complete Guide](/MakazhanAlpamys/Soup/how-to-use-tui-interactive-training-monitoring)

Monitor your AI training runs interactively with Soup's TUI. Enjoy real-time metrics, auto-refresh, and keyboard navigation for a seamless experience.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Create Synthetic Training Data with Soup Data Forge: A Complete Guide](/MakazhanAlpamys/Soup/create-synthetic-training-data-soup-data-forge)

Learn to create synthetic training data with Soup Data Forge. This guide details its multi-stage pipeline for high-quality data generation, preserving provenance.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Implement Data Quality Scoring with `soup data score`](/MakazhanAlpamys/Soup/implement-data-quality-scoring-soup-data-score)

Implement data quality scoring with soup data score. Evaluate PII toxicity language distribution educational value and benchmark contamination in JSONL datasets without dependencies.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Use Soup’s MCP Server for Programmatic Model Interactions](/MakazhanAlpamys/Soup/how-to-use-mcp-server-programmatic-model-interactions)

Learn to use Soup's MCP server for programmatic model interactions. Start the server with `soup mcp serve` to drive Soup using any MCP-compatible client.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Export Models to GGUF Format for Ollama Deployment with Soup](/MakazhanAlpamys/Soup/export-models-gguf-ollama-deployment)

Easily export models to GGUF format for Ollama deployment using the soup export CLI. Convert Hugging Face checkpoints and automatically register them with Ollama for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Enable Layer Streaming for Preference Losses (DPO, GRPO) in Soup](/MakazhanAlpamys/Soup/enable-layer-streaming-preference-losses-dpo-grpo)

Learn to enable layer streaming for preference losses like DPO and GRPO in Soup v0.72.4. Fine-tune larger models by streaming decoder layers from CPU RAM or disk.

- Tags: how-to-guide
- Published: 2026-08-16

### [How LoftQ Works for Quantization-Aware LoRA Initialization in Soup LoRA Fine-Tuning](/MakazhanAlpamys/Soup/how-loftq-works-quantization-aware-lora)

Discover how LoftQ initializes LoRA adapters and quantizes base model weights for Soup LoRA fine-tuning. Learn about this PEFT-backed technique for efficient model optimization.

- Tags: deep-dive
- Published: 2026-08-16

### [How to Configure `rank_pattern` and `alpha_pattern` for MoE Models in Soup](/MakazhanAlpamys/Soup/configure-rank_pattern-alpha_pattern-moe-models)

Learn to configure rank_pattern and alpha_pattern for MoE models in Soup. Optimize fine-tuning by adjusting LoRA ranks for expert and attention layers.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Use PiSSA (`pissa init`) for Faster Early Convergence in Soup](/MakazhanAlpamys/Soup/how-to-use-pissa-init-faster-convergence)

Accelerate training convergence with PiSSA pissa init. Initialize adapter weights from SVD for faster early epochs in your Soup project. Learn how to implement this powerful technique.

- Tags: how-to-guide
- Published: 2026-08-16

### [LoRA vs DoRA vs VeRA vs OLoRA: How to Choose the Right PEFT Method](/MakazhanAlpamys/Soup/lora-dora-vera-olora-comparison)

Explore LoRA DoRA VeRA and OLoRA parameter-efficient fine-tuning PEFT methods. Understand their differences and choose the best adapter technique for your next project.

- Tags: comparison
- Published: 2026-08-16

### [How to Set Up Eval-Gated Training with the SHIP Verdict in Soup](/MakazhanAlpamys/Soup/how-to-set-up-eval-gated-training-ship-verdict)

Learn how to set up eval gated training with the SHIP verdict in Soup. Automatically stop fine tuning when benchmark regressions are detected to save resources and improve model quality.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Configure Reward Functions Using Soup Reward Synth: A Complete Guide](/MakazhanAlpamys/Soup/how-to-configure-reward-functions-soup-reward-synth)

Configure reward functions using Soup Reward Synth. Automate reward verifier generation via a four-stage pipeline: detection, induction, rendering, and calibration. Learn more now!

- Tags: how-to-guide
- Published: 2026-08-16

### [How the `--noise-floor` Option in `soup ship` Handles GPU Non-Determinism](/MakazhanAlpamys/Soup/soup-ship-noise-floor-gpu-non-determinism)

Learn how soup ships --noise-floor option addresses GPU non-determinism by re-executing models and aggregating results into a statistical noise floor for accurate variability quantification.

- Tags: internals
- Published: 2026-08-16

### [How to Use the Release Gate (`soup ship`) for Model Quality Validation](/MakazhanAlpamys/Soup/how-to-use-release-gate-soup-ship-validate-model-quality)

Validate model quality for production using the soup ship release gate. Ensure task improvement and prevent catastrophic forgetting with this essential tool.

- Tags: how-to-guide
- Published: 2026-08-16

### [How DPO Works with Layer Streaming and Its Memory Trade‑Offs in Soup](/MakazhanAlpamys/Soup/dpo-layer-streaming-memory-tradeoffs)

Discover how DPO works with layer streaming in Soup, optimizing memory for large preference models on consumer GPUs. Learn about the trade-offs and efficient training methods.

- Tags: deep-dive
- Published: 2026-08-16

### [`stream_source: auto` vs `RAM` vs `disk` in Soup: Storage Mode Explained](/MakazhanAlpamys/Soup/stream_source-auto-vs-ram-vs-disk)

Understand stream_source: auto, RAM, and disk in Soup. Learn how Soup materializes datasets for faster training or greater scalability based on available memory.

- Tags: deep-dive
- Published: 2026-08-16

### [How to Configure Stream Layers for Optimal VRAM Usage in Soup](/MakazhanAlpamys/Soup/how-to-configure-stream_layers-for-optimal-vram-usage)

Optimize VRAM usage in Soup by configuring stream layers. Learn how to enable layer streaming to load transformer weights on demand and save GPU memory.

- Tags: how-to-guide
- Published: 2026-08-16

### [How Layer Streaming Enables Fine‑Tuning an 8B Model on a 4 GB GPU: Soup's Memory‑Efficient Training Architecture](/MakazhanAlpamys/Soup/how-does-layer-streaming-enable-fine-tuning-8b-model-4gb-gpu)

Discover how layer streaming slashes GPU memory needs from 30 GB to 4 GB, enabling 8B model fine-tuning on low-spec hardware. Learn about Soup's efficient training.

- Tags: deep-dive
- Published: 2026-08-16