How to Fine-Tune the Text-to-CAD Model: A Complete Guide to Custom AI CAD Generation

Fine-tuning the text-to-cad model involves training an OpenAI model on a dataset of natural-language prompts mapped to CLI commands, then updating the skill configuration to use the fine-tuned model ID instead of the default GPT model.

The earthtojake/text-to-cad repository provides a lightweight Python library called cadgen that powers CAD generation through a clean separation between the LLM interface and the CAD kernel. Because the heavy geometry processing is handled by the cadgen engine, you can fine-tune the text-to-cad model to produce domain-specific CLI commands without recompiling Open CASCADE or modifying the core generation logic.

Understanding the Text-to-CAD Architecture

Before fine-tuning, it is essential to understand how the system decouples language understanding from geometry processing.

The architecture follows a strict three-step pipeline:

1. Prompt → LLM – The skill reads user input and calls the OpenAI chat completion API. The request is built in packages/cadgen/src/cadgen/_internal/generation.py, which injects the model ID from the skill configuration.

2. LLM → cadgen CLI – The model responds with a CLI command such as cadgen step export --shape cylinder --diameter 10. The parser in packages/cadgen/src/cadgen/cli.py validates and dispatches this command.

3. CLI → Geometry – The CLI invokes internal generators like packages/cadgen/src/cadgen/step_export.py (for STEP/GLB files) or packages/cadgen/src/cadgen/urdf.py (for robot descriptions) to produce the final output.

Fine-tuning only affects step one. By training the LLM to emit better CLI commands for your specific domain, you improve translation accuracy without touching the OCCT-based kernels in cadgen.

Preparing Your Training Dataset

Create a JSONL file where each line contains a prompt (natural language) and completion (exact CLI string) pair. The completion must match the syntax expected by packages/cadgen/src/cadgen/cli.py.

import json
import pathlib

samples = [
    {
        "prompt": "Create a 10 mm diameter, 20 mm tall cylinder",
        "completion": "cadgen step export --shape cylinder --diameter 10 --height 20"
    },
    {
        "prompt": "Make a 5 mm radius sphere",
        "completion": "cadgen step export --shape sphere --radius 5"
    },
    {
        "prompt": "Generate a URDF for a 1kg box robot",
        "completion": "cadgen urdf export --shape box --mass 1 --name box_bot"
    }
]

train_path = pathlib.Path("cad_finetune_data.jsonl")
train_path.write_text("\n".join(json.dumps(s) for s in samples))

Ensure your completions use valid cadgen subcommands and flags. Invalid syntax will cause runtime errors in cadgen/step_export.py or cadgen/urdf.py when the skill executes the generated command.

Uploading and Fine-Tuning with OpenAI

Use the OpenAI Python SDK to upload your dataset and create a fine-tuned job. Set your API key via environment variables and monitor the job until it succeeds.

import openai
import os

openai.api_key = os.getenv("OPENAI_API_KEY")

# Upload the training file

file_resp = openai.File.create(
    file=open("cad_finetune_data.jsonl", "rb"),
    purpose="fine-tune"
)
file_id = file_resp.id
print(f"Uploaded file ID: {file_id}")

# Create the fine-tuning job

fine_tune = openai.FineTune.create(
    training_file=file_id,
    model="gpt-3.5-turbo",
    n_epochs=4,
    batch_size=4
)
print(f"Job ID: {fine_tune.id}")

Monitor progress using openai.FineTune.list_events(fine_tune.id) until the status returns "succeeded". Retrieve the fine-tuned model ID (format: ft-<random_id>) for the next step.

ft_model = openai.FineTune.retrieve(fine_tune.id).fine_tuned_model
print(f"Fine-tuned model: {ft_model}")

Integrating the Fine-Tuned Model

Point the skill to your custom model by editing the OpenAI configuration file. Each skill stores its LLM settings in skills/<skill>/agents/openai.yaml (e.g., skills/cad/agents/openai.yaml).

import yaml
import pathlib

config_path = pathlib.Path("skills/cad/agents/openai.yaml")
cfg = yaml.safe_load(config_path.read_text())

# Update to your fine-tuned model ID

cfg["model"] = ft_model  # e.g., "ft-g5c5d1e2..."

config_path.write_text(yaml.safe_dump(cfg))

The packages/cadgen/src/cadgen/_internal/generation.py module reads this configuration at runtime to set the model parameter in OpenAI API requests. No other code changes are required—the cadgen CLI parser (cli.py) and geometry exporters (step_export.py, urdf.py) remain unchanged.

Key Files and Their Roles

File	Purpose for Fine-Tuning
packages/cadgen/src/cadgen/_internal/generation.py	Constructs the OpenAI API request; consumes the model field from openai.yaml.
skills/<skill>/agents/openai.yaml	Configuration file where you specify the fine-tuned model ID.
packages/cadgen/src/cadgen/cli.py	Parses CLI strings produced by your fine-tuned model; validate completions against this syntax.
packages/cadgen/src/cadgen/step_export.py	Handles STEP/STL/GLB generation; target of cadgen step export commands.
packages/cadgen/src/cadgen/urdf.py	Generates robot descriptions; target of cadgen urdf export commands.
tests/python/skills/*/test_skill_structure.py	Validates that agents/openai.yaml exists and is valid YAML.

Summary

• Fine-tuning changes only the LLM layer – The cadgen CAD kernel (step_export.py, urdf.py) and CLI parser (cli.py) require no modifications.
• Use prompt-completion pairs – Train on natural language mapped to exact cadgen CLI commands in JSONL format.
• Upload via OpenAI SDK – Create a fine-tuned job using openai.FineTune.create with a base model like gpt-3.5-turbo.
• Update skill configuration – Set the model field in skills/<skill>/agents/openai.yaml to your fine-tuned model ID.
• Zero rebuild required – The architecture's separation between LLM interface and CAD engine means no recompilation of OCCT or native binaries.

Frequently Asked Questions

Do I need to recompile the CAD engine after fine-tuning?

No. The text-to-cad architecture deliberately separates the LLM interface from the geometry kernel. Fine-tuning only changes the model ID in skills/<skill>/agents/openai.yaml. The cadgen library (step_export.py, urdf.py) processes geometry using pre-compiled OCCT bindings regardless of which LLM generated the CLI command.

What base model should I use for fine-tuning the text-to-cad model?

Use gpt-3.5-turbo or gpt-4o-mini as your base model when calling openai.FineTune.create. These models provide the best balance of command-following accuracy and cost efficiency for structured CLI generation tasks. Avoid base models without strong instruction-following capabilities, as they may produce invalid syntax for cadgen/cli.py.

How many training examples are required for effective fine-tuning?

Start with 50-100 high-quality examples covering your specific domain vocabulary (e.g., aerospace fittings, robotic joints). The text-to-cad model learns the mapping between your terminology and cadgen CLI flags quickly because the output space is constrained to valid command syntax. Add more examples if the model hallucinates parameters not supported by cadgen step export or cadgen urdf export.

Can I fine-tune for specific output formats like URDF or SDF?

Yes. Include completions targeting cadgen urdf export or cadgen sdf export in your training JSONL. The packages/cadgen/src/cadgen/urdf.py module handles URDF generation, while SDF support follows the same pattern. Your fine-tuned model can learn to select the correct generator based on the prompt context (e.g., "Create a robot" → URDF, "Create a simulation world" → SDF).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}