# How to Use PiSSA (`pissa init`) for Faster Early Convergence in Soup

> Accelerate training convergence with PiSSA pissa init. Initialize adapter weights from SVD for faster early epochs in your Soup project. Learn how to implement this powerful technique.

- Repository: [Alpamys Makazhan/Soup](https://github.com/MakazhanAlpamys/Soup)
- Tags: how-to-guide
- Published: 2026-08-16

---

**Set `init_strategy="pissa"` in your LoRA configuration to initialize adapter weights from the SVD of pretrained weights, accelerating convergence in early training epochs.**

PiSSA (Pi Singular‑Value‑Decomposition Adaptation) is a strategic initialization method for LoRA adapters that can dramatically reduce time‑to‑convergence. In the Soup fine‑tuning framework, enabling PiSSA requires a single configuration change that triggers an SVD‑based initialization of the **A** and **B** weight matrices. This article explains the implementation details, compatibility constraints, and practical usage patterns based on the Soup source code.

## What PiSSA Initialization Does

PiSSA replaces the default random initialization of LoRA matrices with weights derived from the singular‑value decomposition of the underlying pretrained layer. This creates a better‑aligned starting point that preserves more of the model's existing capabilities while the adapter learns task‑specific adjustments.

In [`src/soup_cli/config/schema.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/config/schema.py), the `init_strategy` field accepts four supported values (lines 110‑118):

- `"random"` — default Gaussian initialization
- `"pissa"` — SVD‑based initialization for faster convergence
- `"olora"` — orthogonal initialization variant
- `"loftq"` — LoftQ quantized initialization

When `"pissa"` is selected, Soup passes `init_lora_weights="pissa"` to PEFT via the builder utility in [`src/soup_cli/utils/peft_builder.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/utils/peft_builder.py) (lines 59‑63).

## Configuration Methods

### YAML Configuration

Define PiSSA in your [`soup.yaml`](https://github.com/MakazhanAlpamys/Soup/blob/main/soup.yaml) file:

```yaml
training:
  lora:
    r: 8
    init_strategy: pissa
    use_rslora: true

```

The `use_rslora: true` flag remains compatible with PiSSA and applies rank‑stabilized scaling.

### Programmatic Configuration

Import and configure `LoraConfig` directly:

```python
from soup_cli.config.schema import LoraConfig

cfg = LoraConfig(
    r=8,
    init_strategy="pissa",
    use_rslora=True,
)

```

To verify the configuration propagates correctly, inspect the built PEFT spec:

```python
from soup_cli.utils.peft_builder import build_peft_config

spec = build_peft_config(
    lora_cfg=cfg,
    target_modules="q_proj, v_proj",
    task_type="CAUSAL_LM",
)

# Confirm PiSSA is active

assert spec["init_kwargs"]["init_lora_weights"] == "pissa"

```

### Command Line Usage

Enable PiSSA via CLI flags:

```bash
soup train \
  --lora.r 8 \
  --lora.init_strategy pissa \
  --lora.use_rslora true

```

## Critical Compatibility Constraints

PiSSA cannot be combined with certain advanced LoRA variants. The validation logic in [`src/soup_cli/config/schema.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/config/schema.py) (lines 777‑784) enforces these restrictions:

| Combination | Status | Error Source |
|-------------|--------|--------------|
| PiSSA + DoRA (`use_dora=True`) | **Blocked** | Pydantic validator raises `ValueError` |
| PiSSA + VeRA (`use_vera=True`) | **Blocked** | Pydantic validator raises `ValueError` |
| PiSSA + RSLoRA (`use_rslora=True`) | **Allowed** | No conflict detected |

If you attempt an invalid combination, the configuration validator rejects it before training begins. This prevents silent failures and wasted compute.

## When to Use PiSSA

PiSSA initialization excels in scenarios where:

- **Rapid experimentation cycles** matter — faster early convergence reduces iteration time
- **Standard LoRA or RSLoRA** is sufficient — you don't require DoRA's weight‑decomposition or VeRA's parameter efficiency
- **Compute for initial SVD** is available — a one‑time SVD computation runs on the first forward pass

The trade‑off is minimal: one additional SVD operation at initialization versus potentially 10‑30% faster convergence in early epochs, based on typical PEFT training patterns.

## Source Code Reference

Three files govern PiSSA behavior in Soup:

- [`src/soup_cli/config/schema.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/config/schema.py) — schema definition and validation rules
- [`src/soup_cli/utils/peft_builder.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/utils/peft_builder.py) — PEFT configuration assembly
- [`tests/test_pissa_init.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/tests/test_pissa_init.py) — unit tests verifying correct propagation

Test coverage in [`tests/test_v07200.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/tests/test_v07200.py) confirms that invalid PiSSA combinations trigger descriptive validation errors.

## Summary

- **Set `init_strategy="pissa"`** in `LoraConfig` to enable SVD‑based initialization
- **Works with RSLoRA** but **conflicts with DoRA and VeRA**
- **Propagates through `build_peft_config`** in [`peft_builder.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/peft_builder.py) to PEFT's `init_lora_weights`
- **Costs one SVD pass** at initialization for faster early‑training progress

## Frequently Asked Questions

### Can I use PiSSA with DoRA for better performance?

No. The Soup validator explicitly blocks this combination in [`schema.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/schema.py) (lines 777‑784). DoRA and PiSSA modify weight initialization in incompatible ways. You must choose one or the other.

### Does PiSSA increase memory usage?

Marginally. The SVD computation requires temporary memory proportional to the weight matrix dimensions, but this occurs once at initialization and does not affect training‑time memory.

### How does PiSSA compare to LoftQ initialization?

Both use structured initialization, but **PiSSA** applies SVD to full‑precision weights (faster convergence), while **LoftQ** focuses on quantization‑aware initialization. For non‑quantized training, PiSSA typically shows stronger early‑epoch gains.

### Where is the PiSSA logic actually implemented?

Soup delegates to PEFT's native PiSSA implementation. The Soup layer handles configuration validation in [`schema.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/schema.py) and parameter passing through [`peft_builder.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/peft_builder.py). The SVD computation itself occurs inside PEFT when `init_lora_weights="pissa"` is received.