# CLI Skills for Placement Testing and Phase Quizzes in AI Engineering From Scratch

> Master CLI skills for AI Engineering From Scratch placement tests and phase quizzes. Explore essential command line tools designed to evaluate your AI engineering knowledge.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-17

---

**The rohitg00/ai-engineering-from-scratch repository provides four command-line utilities—[`install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/install_skills.py), [`lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/lesson_run.py), [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py), and [`check_readme_counts.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/check_readme_counts.py)—that let you install curriculum artefacts, execute placement tests, validate quiz integrity, and synchronize the lesson catalogue.**

The AI Engineering From Scratch curriculum ships with a dedicated CLI toolchain designed specifically for placement testing and phase quizzes. These command-line utilities, located in the `scripts/` directory, provide the only authorized method for accessing and executing the **skill artefacts** stored under `phases/**/outputs/`. By leveraging these tools, educators and learners can programmatically install test prompts, run individual lessons with their associated quizzes, and audit the curriculum for structural integrity before any assessment begins.

## Installing and Managing Skill Artefacts with [`install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/install_skills.py)

The [`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py) script serves as the primary CLI tool for copying curriculum artefacts—skills, prompts, and agents—from the repository into a target directory. This is the essential first step for preparing placement test environments, as it extracts the specific markdown files used as quiz inputs.

The script accepts several critical flags for filtering artefacts:

- `--type {skill,prompt,agent,all}` – Selects the artefact kind; placement tests specifically require **skill** artefacts
- `--phase N` – Limits installation to a single phase (e.g., `3` for Phase 03 quizzes)
- `--tag TAG` – Filters by metadata tags; quizzes are typically tagged `quiz`
- `--layout {flat,by-phase,skills}` – Controls the directory structure of the output
- `--dry-run` – Previews actions without writing files
- `--force` – Overwrites existing files without prompting

When executed, the script generates a [`manifest.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/manifest.json) file that records every installed artefact and its source path. This manifest is crucial for CI pipelines that execute automated placement tests, as it provides a verifiable record of which quiz questions were deployed.

```bash

# Install only Phase 03 quiz artefacts into a sandbox directory

python3 scripts/install_skills.py ./placement_test_dir \
    --type skill \
    --phase 3 \
    --tag quiz \
    --layout by-phase

```

## Running Lessons and Quizzes with [`lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/lesson_run.py)

The [`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py) utility executes individual lessons from the command line, including their demo code and associated quizzes. This script replicates the exact interface used by placement-test runners to present questions and capture learner responses.

To execute a lesson, pass the phase slug and lesson slug in the format `<phase-slug>/<lesson-slug>`:

```bash

# Run Phase 03, Lesson 05 (includes both demo code and quiz)

python3 scripts/lesson_run.py 03-advanced-ml/05-regularization

```

The script supports flags to control execution flow:

- `--no-tests` – Skips the unit-test suite and demo code, displaying only the quiz prompts
- `--quiet` – Suppresses non-essential output for cleaner CI integration

When running a lesson containing a quiz, the script prints quiz prompts to stdout. This output stream is what placement-test harnesses capture to present questions and record answers, making [`lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/lesson_run.py) the core engine for interactive assessment.

## Validating Curriculum Integrity with [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py)

Before launching any placement testing session, run [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) to verify that every lesson complies with the curriculum contract. This script checks for the presence and syntactic validity of [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) files across all phases.

Unlike the other utilities, [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) requires no command-line flags:

```bash

# Verify all quiz definitions are well-formed

python3 scripts/audit_lessons.py

```

The audit catches malformed quiz artefacts before they reach the CLI test runner, preventing runtime errors during placement assessments. According to the repository's [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) policy, this validation step is mandatory for maintaining the integrity of the CLI-first curriculum access pattern.

## Synchronizing the Quiz Catalogue with [`check_readme_counts.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/check_readme_counts.py)

The [`scripts/check_readme_counts.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/check_readme_counts.py) utility ensures the repository README accurately reflects the current number of lessons and available quizzes. This script supports the CI pipeline that publishes the quiz catalogue used by placement test systems.

- `--fix` – Automatically corrects count mismatches in the README

While this tool does not directly execute tests, it maintains the public-facing documentation that lists available placement tests and phase quizzes, ensuring learners always see the correct curriculum scope.

## Step-by-Step Workflow for Placement Testing

The four CLI tools combine into a complete workflow for managing curriculum assessments:

1. **Discover** the skill-files under `phases/**/outputs/skill-*.md` that implement placement-test questions
2. **Install** the required artefacts using [`install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/install_skills.py) with appropriate `--type` and `--phase` filters
3. **Run** the specific lesson or quiz with [`lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/lesson_run.py) to display prompts and capture answers
4. **Audit** the lesson definitions with [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) to catch malformed [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) before test execution

This workflow ensures that all CLI skills for placement testing and phase quizzes are validated, installed, and executed through the authorized command-line interface defined in [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md).

## Code Examples

Here are the essential commands for working with the placement testing CLI:

```bash

# 1. Install skill artefacts for Phase 03 quizzes only

python3 scripts/install_skills.py ./placement_test_dir \
    --type skill \
    --phase 3 \
    --tag quiz \
    --layout by-phase

# 2. Run a specific lesson and display its quiz

python3 scripts/lesson_run.py 03-advanced-ml/05-regularization

# 3. Run lesson in quiz-only mode (skip demo code)

python3 scripts/lesson_run.py 03-advanced-ml/05-regularization --no-tests

# 4. Validate all quiz files before testing

python3 scripts/audit_lessons.py

# 5. Fix README lesson counts for the quiz catalogue

python3 scripts/check_readme_counts.py --fix

```

## Summary

- **[`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py)** copies skill artefacts from `phases/**/outputs/` to a target directory using filters like `--type skill`, `--phase`, and `--tag quiz`, generating a [`manifest.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/manifest.json) for CI tracking.
- **[`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py)** executes lessons and quizzes via the `<phase-slug>/<lesson-slug>` syntax, supporting `--no-tests` for quiz-only output.
- **[`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)** validates the integrity of [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) files across the curriculum without requiring additional flags.
- **[`scripts/check_readme_counts.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/check_readme_counts.py)** synchronizes the public quiz catalogue with the `--fix` option for automated documentation updates.
- The complete CLI workflow follows four steps: discover artefacts, install with filters, run lessons with [`lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/lesson_run.py), and audit with [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py).
- All tools operate according to the [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) policy, which mandates CLI utilities as the only authorized access method for curriculum artefacts.

## Frequently Asked Questions

### How do I extract only the quiz questions for a specific phase?

Run `python3 scripts/install_skills.py <target_dir> --type skill --phase <N> --tag quiz --layout by-phase`. This installs only the markdown skill artefacts tagged as quizzes from the specified phase into your target directory, organized by phase.

### Can I run a quiz without executing the lesson's demo code?

Yes. Pass the `--no-tests` flag to [`scripts/lesson_run.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/lesson_run.py) when executing the lesson. This skips the unit-test suite and demo code, printing only the quiz prompts to stdout for immediate interaction or capture by a test harness.

### What ensures the quiz files are valid before I run them?

The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) utility validates every [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file in the curriculum for syntactic correctness and structural compliance. Run this script before any placement testing session to eliminate malformed artefacts that could cause runtime failures.

### Where are the actual quiz questions stored in the repository?

The quiz content lives in markdown files under `phases/**/outputs/skill-*.md`. These files contain the test prompts and metadata used by the placement testing framework. The [`install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/install_skills.py) script is the only authorized method for extracting these files from their phase-organized locations.