# How AI-Powered Guides and Courses Are Generated Using the AI SDK in Developer Roadmap

> Learn how AI-powered guides and courses are generated with the AI SDK in Developer Roadmap. Discover how Vercel AI SDK and custom parsers create content from backend endpoints.

- Repository: [Kamran Ahmed/developer-roadmap](https://github.com/kamranahmedse/developer-roadmap)
- Tags: how-to-guide
- Published: 2026-02-24

---

**The Developer Roadmap repository generates AI-powered guides and courses by combining the Vercel AI SDK's `DefaultChatTransport` with custom streaming parsers that incrementally render content from backend endpoints.**

The `kamranahmedse/developer-roadmap` project implements a reactive generation pipeline that transforms user queries into structured learning materials. By leveraging the `ai` npm package alongside custom transport layers and stream processors, the application delivers real-time AI-generated content without blocking the UI.

## Core Architecture

The generation system relies on three integrated components: the AI SDK transport layer, helper utilities that orchestrate API calls, and streaming parsers that transform raw bytes into renderable content.

### AI SDK Transport Layer

The application initializes **chat transports** using the AI SDK's `DefaultChatTransport` class to handle streaming HTTP requests. In [`src/lib/ai.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/lib/ai.ts), two transport instances are configured:

- **`chatRoadmapTransport`** – handles roadmap-specific AI interactions
- **`topicDetailAiChatTransport`** – manages topic-level chat streams

These transports abstract the HTTP request logic and provide typed message handling through the **`UIMessage`** interface, ensuring type safety across the streaming boundary.

### Helper Utilities for Guide and Course Generation

The orchestration layer resides in dedicated helper files that POST to backend endpoints and manage callback lifecycles:

1. **`generateGuide`** in [`src/helper/generate-ai-guide.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/helper/generate-ai-guide.ts) sends requests to `/v1-generate-ai-guide` and returns a `ReadableStream` for guide content
2. **`generateCourse`** in [`src/helper/generate-ai-course.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/helper/generate-ai-course.ts) communicates with `/v1-generate-ai-course` to stream markdown-based course structures

Both helpers accept callback functions (`onMessage`, `onStream`, `onCourseChange`) that execute as chunks arrive, enabling progressive UI updates.

### Streaming Parsers

The SDK does not parse stream data automatically. Instead, the repository provides thin wrapper functions:

- **`readChatStream`** ([`src/lib/chat.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/lib/chat.ts)) – processes guide streams by splitting lines on `:` separators and interpreting prefixes (`0` for messages, `d` for details)
- **`readStream`** ([`src/lib/ai.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/lib/ai.ts)) – handles course streams by accumulating chunks until newlines, then triggering `onStream` callbacks

These parsers convert raw byte streams into structured data or markdown strings that components can render immediately.

## Generating AI Guides: End-to-End Flow

When a user requests an AI guide, the system executes a six-step streaming pipeline:

1. **UI Trigger** – [`GenerateAIGuide.tsx`](https://github.com/kamranahmedse/developer-roadmap/blob/main/GenerateAIGuide.tsx) invokes `generateGuide({ term, onGuideChange, onHtmlChange })`
2. **API Request** – The helper POSTs the search term to `/v1-generate-ai-guide` with fetch credentials
3. **Stream Processing** – `readChatStream` reads the response body using `getReader()` and `TextDecoder`
4. **Prefix Parsing** – Each line splits on `:`, where prefix `0` indicates message content and `d` indicates metadata details
5. **Progressive Rendering** – The parser concatenates JSON payloads and invokes `onMessage` with accumulated text, which converts markdown to HTML via `markdownToHtmlWithHighlighting`
6. **Completion** – `onMessageEnd` fires to finalize the UI and invalidate the AI-limit React Query cache

The stream format follows a strict protocol defined by `CHAT_RESPONSE_PREFIX` constants, allowing the frontend to distinguish between content chunks and control messages.

### Guide Generation Code Example

```typescript
import { generateGuide } from '../../helper/generate-ai-guide';
import { markdownToHtmlWithHighlighting } from '../../lib/markdown';

async function streamAIGuide(term: string) {
  let accumulatedContent = '';
  
  await generateGuide({
    term,
    onGuideChange: (content) => {
      accumulatedContent = content;
      console.log('Raw markdown:', content);
    },
    onHtmlChange: (html) => {
      // Render HTML immediately as chunks arrive
      document.getElementById('guide-container')!.innerHTML = html;
    },
    onLoadingChange: (isLoading) => {
      // Toggle loading spinner
    },
    onError: (error) => {
      console.error('Guide generation failed:', error);
    },
  });
}

```

The `onGuideChange` callback receives incremental markdown updates, while `onHtmlChange` provides pre-processed HTML with syntax highlighting applied.

## Generating AI Courses: End-to-End Flow

Course generation follows a similar pattern but handles structured data extraction from markdown streams:

1. **Initiation** – [`GenerateAICourse.tsx`](https://github.com/kamranahmedse/developer-roadmap/blob/main/GenerateAICourse.tsx) calls `generateCourse({ term, onCourseChange })`
2. **API Communication** – The helper submits POST data to `/v1-generate-ai-course`
3. **Marker Extraction** – The stream contains embedded markers (`@COURSEID:xyz@`, `@COURSESLUG:abc@`) that `generateCourse` extracts to update the browser URL via `window.history.replaceState`
4. **Markdown Accumulation** – `readStream` buffers chunks until newline characters, then calls `onStream` with complete lines
5. **Structure Parsing** – Raw markdown (minus markers) passes to `generateAiCourseStructure`, which parses headers (`## Module Name`) and list items (`- Lesson Name`) into a strongly-typed `AiCourse` object

6. **State Updates** – `onCourseChange` receives both the structured `AiCourse` object and the raw markdown string, allowing the UI to render a live preview while maintaining the structured data for navigation

### Course Generation Code Example

```typescript
import { generateCourse } from '../../helper/generate-ai-course';
import type { AiCourse } from '../../lib/ai';

async function streamAICourse(term: string) {
  await generateCourse({
    term,
    onCourseChange: (structuredCourse: AiCourse, rawMarkdown: string) => {
      // structuredCourse contains { title, modules: [{ title, lessons: [...] }] }
      console.log('Course title:', structuredCourse.title);
      
      // rawMarkdown preserves the original stream for display
      renderCoursePreview(structuredCourse);
    },
    onLoadingChange: (loading) => {
      // Update loading state in UI
    },
  });
}

```

The `AiCourse` type enforces a specific schema with `title`, `modules`, and nested `lessons` arrays, enabling the frontend to build interactive course navigation trees from streaming markdown.

## Low-Level Stream Processing Implementation

The parsing logic differs between guides and courses based on their data formats.

### Guide Stream Parsing (`readChatStream`)

Located in [`src/lib/chat.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/lib/chat.ts), this function handles the `Uint8Array` stream from the AI SDK:

```typescript
export async function readChatStream(
  stream: ReadableStream<Uint8Array>,
  { onMessage, onMessageEnd, onDetails }: ChatStreamCallbacks
) {
  const reader = stream.getReader();
  const decoder = new TextDecoder('utf-8');
  let result = '';

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    
    const text = decoder.decode(value, { stream: true });
    const lines = text.split('\n');
    
    for (const line of lines) {
      if (!line) continue;
      const [prefix, payload] = line.split(':');
      
      if (prefix === '0') { // CHAT_RESPONSE_PREFIX.message
        result += JSON.parse(payload);
        await onMessage?.(result);
      } else if (prefix === 'd') { // CHAT_RESPONSE_PREFIX.details
        await onDetails?.(JSON.parse(payload));
      }
    }
  }
  
  await onMessageEnd?.(result);
  reader.releaseLock();
}

```

This parser specifically handles the `0:` (content) and `d:` (details) protocol used by the guide generation backend.

### Course Stream Parsing (`readStream`)

The course implementation in [`src/lib/ai.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/lib/ai.ts) uses a simpler line-based approach:

```typescript
export async function readStream(
  stream: ReadableStream<Uint8Array>,
  { onStream }: { onStream: (text: string) => Promise<void> | void }
) {
  const reader = stream.getReader();
  const decoder = new TextDecoder('utf-8');
  let buffer = '';

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    
    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split('\n');
    buffer = lines.pop() || ''; // Keep incomplete line in buffer
    
    for (const line of lines) {
      if (line) await onStream(line);
    }
  }
  
  if (buffer) await onStream(buffer); // Flush remaining content
  reader.releaseLock();
}

```

After stream completion, `generateAiCourseStructure` processes the accumulated markdown using regex patterns to extract the course title, module headers, and lesson lists into the final `AiCourse` object.

## Summary

- **The Developer Roadmap** uses the Vercel AI SDK's `DefaultChatTransport` to establish streaming connections with backend generation endpoints.
- **Guide generation** relies on `readChatStream` in [`src/lib/chat.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/lib/chat.ts) to parse prefixed line protocols (`0:` for content, `d:` for details) and progressively render markdown via [`generate-ai-guide.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/generate-ai-guide.ts).
- **Course generation** employs `readStream` in [`src/lib/ai.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/lib/ai.ts) to process markdown streams, extract metadata markers for URL updates, and convert content into structured `AiCourse` objects using `generateAiCourseStructure`.
- **Both flows** implement reactive callbacks (`onMessage`, `onStream`, `onCourseChange`) that update the UI incrementally without waiting for complete document generation.

## Frequently Asked Questions

### What AI SDK version does the Developer Roadmap use?

The repository uses the `ai` npm package (Vercel AI SDK) which provides the `DefaultChatTransport` class and `UIMessage` types. The specific version is defined in the project's [`package.json`](https://github.com/kamranahmedse/developer-roadmap/blob/main/package.json) dependencies, utilizing the SDK's core streaming primitives without the React hooks layer for these particular features.

### How does the streaming parser handle incomplete chunks?

Both `readChatStream` and `readStream` implement buffering logic to handle partial UTF-8 byte sequences. They accumulate chunks in a buffer, split on newline characters, and process complete lines while preserving incomplete data for the next iteration. This ensures that multi-byte characters split across stream chunks are correctly decoded using `TextDecoder` with the `stream: true` option.

### Can the AI course generation handle custom formatting?

The `generateAiCourseStructure` function in [`src/lib/ai.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/lib/ai.ts) expects markdown following a specific convention: the course title as an H1, modules as H2 headers (`## Module Name`), and lessons as bullet points (`- Lesson Name`) or list items. While the stream can contain additional markdown, the parser specifically looks for these hierarchical elements to construct the `AiCourse` object with proper nesting of modules and lessons.

### What happens when the AI generation stream encounters an error?

The helper functions in [`src/helper/generate-ai-guide.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/helper/generate-ai-guide.ts) and [`src/helper/generate-ai-course.ts`](https://github.com/kamranahmedse/developer-roadmap/blob/main/src/helper/generate-ai-course.ts) wrap stream operations in try-catch blocks. If the fetch request fails or the stream encounters a parsing error, the `onError` callback fires with the error details, allowing the UI component to display error states. Additionally, the `reader.releaseLock()` method ensures the stream reader is properly cleaned up even when errors occur during processing.