How AI-Powered Guides and Courses Are Generated Using the AI SDK in Developer Roadmap

The Developer Roadmap repository generates AI-powered guides and courses by combining the Vercel AI SDK's DefaultChatTransport with custom streaming parsers that incrementally render content from backend endpoints.

The kamranahmedse/developer-roadmap project implements a reactive generation pipeline that transforms user queries into structured learning materials. By leveraging the ai npm package alongside custom transport layers and stream processors, the application delivers real-time AI-generated content without blocking the UI.

Core Architecture

The generation system relies on three integrated components: the AI SDK transport layer, helper utilities that orchestrate API calls, and streaming parsers that transform raw bytes into renderable content.

AI SDK Transport Layer

The application initializes chat transports using the AI SDK's DefaultChatTransport class to handle streaming HTTP requests. In src/lib/ai.ts, two transport instances are configured:

• chatRoadmapTransport – handles roadmap-specific AI interactions
• topicDetailAiChatTransport – manages topic-level chat streams

These transports abstract the HTTP request logic and provide typed message handling through the UIMessage interface, ensuring type safety across the streaming boundary.

Helper Utilities for Guide and Course Generation

The orchestration layer resides in dedicated helper files that POST to backend endpoints and manage callback lifecycles:

1. generateGuide in src/helper/generate-ai-guide.ts sends requests to /v1-generate-ai-guide and returns a ReadableStream for guide content
2. generateCourse in src/helper/generate-ai-course.ts communicates with /v1-generate-ai-course to stream markdown-based course structures

Both helpers accept callback functions (onMessage, onStream, onCourseChange) that execute as chunks arrive, enabling progressive UI updates.

Streaming Parsers

The SDK does not parse stream data automatically. Instead, the repository provides thin wrapper functions:

• readChatStream (src/lib/chat.ts) – processes guide streams by splitting lines on : separators and interpreting prefixes (0 for messages, d for details)
• readStream (src/lib/ai.ts) – handles course streams by accumulating chunks until newlines, then triggering onStream callbacks

These parsers convert raw byte streams into structured data or markdown strings that components can render immediately.

Generating AI Guides: End-to-End Flow

When a user requests an AI guide, the system executes a six-step streaming pipeline:

1. UI Trigger – GenerateAIGuide.tsx invokes generateGuide({ term, onGuideChange, onHtmlChange })
2. API Request – The helper POSTs the search term to /v1-generate-ai-guide with fetch credentials
3. Stream Processing – readChatStream reads the response body using getReader() and TextDecoder
4. Prefix Parsing – Each line splits on :, where prefix 0 indicates message content and d indicates metadata details
5. Progressive Rendering – The parser concatenates JSON payloads and invokes onMessage with accumulated text, which converts markdown to HTML via markdownToHtmlWithHighlighting
6. Completion – onMessageEnd fires to finalize the UI and invalidate the AI-limit React Query cache

The stream format follows a strict protocol defined by CHAT_RESPONSE_PREFIX constants, allowing the frontend to distinguish between content chunks and control messages.

Guide Generation Code Example

import { generateGuide } from '../../helper/generate-ai-guide';
import { markdownToHtmlWithHighlighting } from '../../lib/markdown';

async function streamAIGuide(term: string) {
  let accumulatedContent = '';
  
  await generateGuide({
    term,
    onGuideChange: (content) => {
      accumulatedContent = content;
      console.log('Raw markdown:', content);
    },
    onHtmlChange: (html) => {
      // Render HTML immediately as chunks arrive
      document.getElementById('guide-container')!.innerHTML = html;
    },
    onLoadingChange: (isLoading) => {
      // Toggle loading spinner
    },
    onError: (error) => {
      console.error('Guide generation failed:', error);
    },
  });
}

The onGuideChange callback receives incremental markdown updates, while onHtmlChange provides pre-processed HTML with syntax highlighting applied.

Generating AI Courses: End-to-End Flow

Course generation follows a similar pattern but handles structured data extraction from markdown streams:

1. Initiation – GenerateAICourse.tsx calls generateCourse({ term, onCourseChange })

2. API Communication – The helper submits POST data to /v1-generate-ai-course

3. Marker Extraction – The stream contains embedded markers (@COURSEID:xyz@, @COURSESLUG:abc@) that generateCourse extracts to update the browser URL via window.history.replaceState

4. Markdown Accumulation – readStream buffers chunks until newline characters, then calls onStream with complete lines

5. Structure Parsing – Raw markdown (minus markers) passes to generateAiCourseStructure, which parses headers (## Module Name) and list items (- Lesson Name) into a strongly-typed AiCourse object

6. State Updates – onCourseChange receives both the structured AiCourse object and the raw markdown string, allowing the UI to render a live preview while maintaining the structured data for navigation

Course Generation Code Example

import { generateCourse } from '../../helper/generate-ai-course';
import type { AiCourse } from '../../lib/ai';

async function streamAICourse(term: string) {
  await generateCourse({
    term,
    onCourseChange: (structuredCourse: AiCourse, rawMarkdown: string) => {
      // structuredCourse contains { title, modules: [{ title, lessons: [...] }] }
      console.log('Course title:', structuredCourse.title);
      
      // rawMarkdown preserves the original stream for display
      renderCoursePreview(structuredCourse);
    },
    onLoadingChange: (loading) => {
      // Update loading state in UI
    },
  });
}

The AiCourse type enforces a specific schema with title, modules, and nested lessons arrays, enabling the frontend to build interactive course navigation trees from streaming markdown.

Low-Level Stream Processing Implementation

The parsing logic differs between guides and courses based on their data formats.

Guide Stream Parsing (readChatStream)

Located in src/lib/chat.ts, this function handles the Uint8Array stream from the AI SDK:

export async function readChatStream(
  stream: ReadableStream<Uint8Array>,
  { onMessage, onMessageEnd, onDetails }: ChatStreamCallbacks
) {
  const reader = stream.getReader();
  const decoder = new TextDecoder('utf-8');
  let result = '';

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    
    const text = decoder.decode(value, { stream: true });
    const lines = text.split('\n');
    
    for (const line of lines) {
      if (!line) continue;
      const [prefix, payload] = line.split(':');
      
      if (prefix === '0') { // CHAT_RESPONSE_PREFIX.message
        result += JSON.parse(payload);
        await onMessage?.(result);
      } else if (prefix === 'd') { // CHAT_RESPONSE_PREFIX.details
        await onDetails?.(JSON.parse(payload));
      }
    }
  }
  
  await onMessageEnd?.(result);
  reader.releaseLock();
}

This parser specifically handles the 0: (content) and d: (details) protocol used by the guide generation backend.

Course Stream Parsing (readStream)

The course implementation in src/lib/ai.ts uses a simpler line-based approach:

export async function readStream(
  stream: ReadableStream<Uint8Array>,
  { onStream }: { onStream: (text: string) => Promise<void> | void }
) {
  const reader = stream.getReader();
  const decoder = new TextDecoder('utf-8');
  let buffer = '';

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    
    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split('\n');
    buffer = lines.pop() || ''; // Keep incomplete line in buffer
    
    for (const line of lines) {
      if (line) await onStream(line);
    }
  }
  
  if (buffer) await onStream(buffer); // Flush remaining content
  reader.releaseLock();
}

After stream completion, generateAiCourseStructure processes the accumulated markdown using regex patterns to extract the course title, module headers, and lesson lists into the final AiCourse object.

Summary

• The Developer Roadmap uses the Vercel AI SDK's DefaultChatTransport to establish streaming connections with backend generation endpoints.
• Guide generation relies on readChatStream in src/lib/chat.ts to parse prefixed line protocols (0: for content, d: for details) and progressively render markdown via generate-ai-guide.ts.
• Course generation employs readStream in src/lib/ai.ts to process markdown streams, extract metadata markers for URL updates, and convert content into structured AiCourse objects using generateAiCourseStructure.
• Both flows implement reactive callbacks (onMessage, onStream, onCourseChange) that update the UI incrementally without waiting for complete document generation.

Frequently Asked Questions

What AI SDK version does the Developer Roadmap use?

The repository uses the ai npm package (Vercel AI SDK) which provides the DefaultChatTransport class and UIMessage types. The specific version is defined in the project's package.json dependencies, utilizing the SDK's core streaming primitives without the React hooks layer for these particular features.

How does the streaming parser handle incomplete chunks?

Both readChatStream and readStream implement buffering logic to handle partial UTF-8 byte sequences. They accumulate chunks in a buffer, split on newline characters, and process complete lines while preserving incomplete data for the next iteration. This ensures that multi-byte characters split across stream chunks are correctly decoded using TextDecoder with the stream: true option.

Can the AI course generation handle custom formatting?

The generateAiCourseStructure function in src/lib/ai.ts expects markdown following a specific convention: the course title as an H1, modules as H2 headers (## Module Name), and lessons as bullet points (- Lesson Name) or list items. While the stream can contain additional markdown, the parser specifically looks for these hierarchical elements to construct the AiCourse object with proper nesting of modules and lessons.

What happens when the AI generation stream encounters an error?

The helper functions in src/helper/generate-ai-guide.ts and src/helper/generate-ai-course.ts wrap stream operations in try-catch blocks. If the fetch request fails or the stream encounters a parsing error, the onError callback fires with the error details, allowing the UI component to display error states. Additionally, the reader.releaseLock() method ensures the stream reader is properly cleaned up even when errors occur during processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}