# How `get_batches` Splits Files Exceeding Token Budgets into Smaller Chunks in NVIDIA SkillSpector

> Discover how SkillSpector's get_batches method intelligently splits large files into smaller chunks, ensuring they fit within token budgets using line-based segmentation.

- Repository: [NVIDIA Corporation/SkillSpector](https://github.com/NVIDIA/SkillSpector)
- Tags: internals
- Published: 2026-06-19

---

**SkillSpector's `get_batches` method calculates the remaining token budget after accounting for prompt overhead, then uses `chunk_file_by_lines` to carve oversized files into overlapping line-based chunks that each fit within the model's input limits.**

The NVIDIA SkillSpector repository provides LLM-powered code analysis capabilities that must handle arbitrarily large source files. When a file exceeds the model's token limit, the `LLMAnalyzerBase.get_batches` method in [`src/skillspector/llm_analyzer_base.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_analyzer_base.py) implements a sophisticated budgeting and splitting strategy to ensure no LLM request is truncated while preserving line-accurate context.

## Calculate Static Prompt Overhead

Before processing file content, `get_batches` first determines how many tokens the analyzer's base prompt consumes. This **static overhead** is calculated using the `estimate_tokens` utility, which divides character count by `CHARS_PER_TOKEN` (defined as 4).

```python
base_overhead = estimate_tokens(self.base_prompt)   # lines 265-266

```

This value represents tokens permanently reserved for instructions, few-shot examples, or system prompts that accompany every request.

## Determine the Usable Content Budget

With the overhead established, the method calculates the actual space available for file content. It retrieves the maximum input tokens from `get_max_input_tokens`, then subtracts both the base overhead and any additional overhead specific to analyzer subclasses. The code guarantees a minimum of **1,024 tokens** for content to prevent unusable micro-batches.

```python
content_budget = max(self._input_budget - base_overhead - extra, 1024)   # lines 85-86

```

This `content_budget` becomes the hard limit for each individual file or chunk processed by the system.

## Whole File vs. Chunked Processing

The method evaluates each file against the calculated budget before deciding whether to batch it whole or split it.

**For files that fit:** If `estimate_tokens(content) ≤ content_budget`, `get_batches` creates a single `Batch` containing the entire file content, line numbers, and any associated findings.

```python
batches.append(Batch(file_path=path, content=content, findings=file_findings))   # lines 90-95

```

**For oversized files:** When content exceeds the budget, the method delegates to `chunk_file_by_lines` to perform intelligent splitting.

## Line-Based Chunking with Context Preservation

The `chunk_file_by_lines` function (implemented at lines 129-169) processes large files line-by-line, accumulating token counts until the `chunk_budget` is reached. Critically, it maintains **50 lines of overlap** between consecutive chunks using the `CHUNK_OVERLAP_LINES` constant, ensuring that context around chunk boundaries is not lost during analysis.

The function returns a list of tuples containing the chunk text and its precise line range:

```python
for chunk_text, s_line, e_line in chunk_file_by_lines(content, chunk_budget):   # lines 298-300

```

This overlap prevents security vulnerabilities or bugs that span chunk boundaries from being missed due to arbitrary file splitting.

## Constructing Batches with Attached Findings

Each generated chunk becomes its own `Batch` object. The `Batch` dataclass (defined at lines 102-111) tracks whether it represents a partial file via the `is_chunk` property and stores `start_line` and `end_line` for accurate reporting.

```python
Batch(
    file_path=path,
    content=chunk_text,
    start_line=s_line,
    end_line=e_line,
    findings=chunk_findings,
)

```

The method uses `findings_in_range` to attach only relevant pre-existing findings to each chunk, ensuring that diagnostic context travels with the specific lines it references.

## Practical Implementation Example

The following example demonstrates how `get_batches` automatically handles mixed file sizes:

```python
from skillspector.llm_analyzer_base import LLMAnalyzerBase

# Initialize analyzer with GPT-4

my_analyzer = LLMAnalyzerBase(
    base_prompt="Find security issues in the following Python file.",
    model="gpt-4"
)

# Cache containing both small and oversized files

file_cache = {
    "big_script.py": open("big_script.py").read(),
    "small.py": "print('hello')",
}

# Generate batches respecting token budgets

batches = my_analyzer.get_batches(
    file_paths=["big_script.py", "small.py"],
    file_cache=file_cache,
)

for b in batches:
    print(f"Batch for {b.file_path} (lines {b.start_line}-{b.end_line or 'end'}), "
          f"{len(b.content)} chars")

```

The output produces a single batch for [`small.py`](https://github.com/NVIDIA/SkillSpector/blob/main/small.py) and multiple overlapping batches for [`big_script.py`](https://github.com/NVIDIA/SkillSpector/blob/main/big_script.py), each conforming to the token budget calculated from the GPT-4 context window.

## Summary

- **`get_batches`** orchestrates token budgeting by first calculating prompt overhead, then reserving a minimum of 1,024 tokens for actual content.
- **`chunk_file_by_lines`** splits oversized files into line-aligned chunks with 50-line overlaps to preserve boundary context.
- Each chunk becomes a **Batch** object tracking its source file and line range, with relevant findings attached via `findings_in_range`.
- The implementation in [`src/skillspector/llm_analyzer_base.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_analyzer_base.py) ensures no LLM request exceeds model limits while maintaining line-accurate analysis capabilities.

## Frequently Asked Questions

### How does SkillSpector calculate token counts for budgeting?

SkillSpector uses a simple heuristic in `estimate_tokens` that divides character count by `CHARS_PER_TOKEN` (set to 4). This approximation runs quickly without requiring the full tokenizer of the target LLM, making it suitable for rapid batching decisions in [`src/skillspector/llm_analyzer_base.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_analyzer_base.py).

### Why does the chunking algorithm use 50 lines of overlap?

The `CHUNK_OVERLAP_LINES` constant (set to 50) ensures that security vulnerabilities or code constructs spanning chunk boundaries are not split into contexts that lose semantic meaning. This overlap provides sufficient surrounding context for the LLM to understand relationships that cross the artificial chunk divider.

### What happens if a file is smaller than the content budget?

For files where `estimate_tokens(content)` is less than or equal to `content_budget`, `get_batches` creates a single `Batch` containing the entire file. The method skips chunking entirely and attaches all file-specific findings to this single batch, optimizing for fewer LLM calls when possible.

### Where are the `Batch` objects defined in the source code?

The `Batch` dataclass is defined at lines 102-111 in [`src/skillspector/llm_analyzer_base.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_analyzer_base.py). It includes fields for `file_path`, `content`, `start_line`, `end_line`, and `findings`, along with convenience properties like `is_chunk` and `file_label` that downstream analyzers use for reporting.