# How the Log Parser Extracts Stream Information in the IPTV-Org Repository

> Discover how the iptv-org log parser extracts stream information from generators log. Learn about file types paths and channel counts detailed in each JSON entry.

- Repository: [iptv-org/iptv](https://github.com/iptv-org/iptv)
- Tags: how-to-guide
- Published: 2026-02-25

---

**The LogParser class reads the `generators.log` file line-by-line, parses each JSON entry into a LogItem object, and returns a typed array containing stream metadata including file type, path, and channel count.**

The iptv-org/iptv repository manages thousands of M3U playlist files organized by categories, countries, languages, and regions. To track these generation artifacts, the build pipeline writes structured logs that the log parser extracts stream information from, enabling automated reporting and table generation. Understanding this parsing mechanism reveals how the repository maintains its comprehensive index of IPTV streams.

## Understanding the generators.log File Structure

Before parsing occurs, the generation workflow writes a plain-text log file at `logs/generators.log`. Each line represents a single artifact produced during the build process:

```json
{"type":"region","filepath":"regions/afr.m3u","count":2}
{"type":"category","filepath":"categories/entertainment.m3u","count":150}
{"type":"country","filepath":"countries/us.m3u","count":450}

```

The **LogItem** type definition in [`scripts/core/logParser.ts`](https://github.com/iptv-org/iptv/blob/main/scripts/core/logParser.ts) formalizes this structure:

```typescript
export type LogItem = {
  type: string
  filepath: string
  count: number
}

```

## Core Log Parser Implementation

### The LogParser Class

Located at [`scripts/core/logParser.ts`](https://github.com/iptv-org/iptv/blob/main/scripts/core/logParser.ts), the **LogParser** class provides a single method that transforms raw log content into typed objects. The implementation handles edge cases like empty content and blank lines:

```typescript
export class LogParser {
  parse(content: string): LogItem[] {
    if (!content) return []
    const lines = content.split('\n')
    return lines.map(line => (line ? JSON.parse(line) : null)).filter(l => l)
  }
}

```

The parsing logic follows a strict pipeline:
1. **Guard clause** returns empty array for falsy input
2. **Line splitting** breaks the file on newline characters
3. **JSON parsing** converts each non-empty line to an object
4. **Null filtering** removes empty entries resulting from blank lines

## How Downstream Components Use Parsed Stream Data

The parsed **LogItem** array powers multiple table generators across the repository. Each component filters the logs by type to extract relevant statistics.

### Region Statistics Generation

In [`scripts/tables/regionsTable.ts`](https://github.com/iptv-org/iptv/blob/main/scripts/tables/regionsTable.ts), the parser collects regional playlist data:

```typescript
const parser = new LogParser()
const logsStorage = new Storage(LOGS_DIR)
const generatorsLog = await logsStorage.load('generators.log')
const parsed = parser.parse(generatorsLog)
const logRegions = parsed.filter((logItem: LogItem) => logItem.type === 'region')

```

### Language and Category Aggregation

Similar patterns appear in [`scripts/tables/languagesTable.ts`](https://github.com/iptv-org/iptv/blob/main/scripts/tables/languagesTable.ts) and [`scripts/tables/categoriesTable.ts`](https://github.com/iptv-org/iptv/blob/main/scripts/tables/categoriesTable.ts):

```typescript
const parser = new LogParser()
const generatorsLog = await logsStorage.load('generators.log')
const parsed = parser.parse(generatorsLog)
const logLanguages = parsed.filter(l => l.type === 'language')

```

## Step-by-Step Stream Information Extraction Process

When the build pipeline needs to analyze generated playlists, it follows this exact sequence:

1. **Load raw content** using `Storage(LOGS_DIR).load('generators.log')` to read the entire file as a string
2. **Instantiate parser** by creating `new LogParser()` (the class maintains no internal state)
3. **Split into lines** via `content.split('\n')` to process each entry individually
4. **Parse JSON entries** using `JSON.parse(line)` for every non-empty line
5. **Filter valid objects** to remove null values from blank lines or parsing errors
6. **Return typed array** of `LogItem` objects ready for filtering and aggregation

## Practical Example: Querying Stream Statistics

The following script demonstrates how to extract comprehensive statistics from the log file using the **LogParser**:

```typescript
import { Storage } from '@freearhey/storage-js'
import { LogParser, LogItem } from '../core'
import { LOGS_DIR } from '../constants'

async function getStreamStats() {
  // Load the raw log content
  const logsStorage = new Storage(LOGS_DIR)
  const rawLog = await logsStorage.load('generators.log')

  // Parse into structured objects
  const parser = new LogParser()
  const items: LogItem[] = parser.parse(rawLog)

  // Aggregate total stream count per file type
  const stats = items.reduce((acc, cur) => {
    acc[cur.type] = (acc[cur.type] ?? 0) + cur.count
    return acc
  }, {} as Record<string, number>)

  console.log('Stream statistics by log type:', stats)
}

getStreamStats()

```

Executing this script produces aggregated output similar to:

```

Stream statistics by log type: {
  raw: 14,
  category: 31,
  language: 5,
  country: 4,
  region: 23,
  source: 8,
  index: 4
}

```

## Summary

- The **LogParser** class in [`scripts/core/logParser.ts`](https://github.com/iptv-org/iptv/blob/main/scripts/core/logParser.ts) converts raw `generators.log` text into typed **LogItem** objects
- Each log entry contains three fields: `type` (artifact category), `filepath` (relative path), and `count` (number of streams)
- The parser handles edge cases by filtering empty lines and returning an empty array for null input
- Downstream table generators in `scripts/tables/` use the parsed data to create markdown statistics for regions, languages, and categories
- The implementation relies on standard JavaScript methods: `split('\n')`, `JSON.parse()`, and `filter()`

## Frequently Asked Questions

### What file format does the generators.log use?

The `generators.log` file uses a **newline-delimited JSON** (NDJSON) format. Each line contains a single JSON object representing one generated artifact, with no outer array or commas between entries. This format allows the **LogParser** to process the file line-by-line using `split('\n')` and `JSON.parse()`.

### How does the LogParser handle malformed or empty lines?

The parser implements defensive programming to handle irregular input. When splitting content by newlines, it maps each line through a conditional: `line ? JSON.parse(line) : null`. Empty strings become `null` values, which are then removed by the final `.filter(l => l)` operation. If the input content is falsy, the method returns an empty array immediately.

### Can I use LogParser to analyze custom log files?

Yes, the **LogParser** is generic and stateless. As long as your log file contains newline-separated JSON objects with `type`, `filepath`, and `count` fields (or any structure you define), you can instantiate `new LogParser()` and call `parse(content)` to receive a typed array. The class has no dependencies on specific file paths or storage implementations.

### Where does the generators.log file get created?

The `generators.log` file is written during the playlist generation workflow, specifically in [`scripts/commands/playlist/generate.ts`](https://github.com/iptv-org/iptv/blob/main/scripts/commands/playlist/generate.ts). As the command generates raw playlists, category files, regional collections, and other artifacts, it appends a JSON line to the log for each file created. The **Storage** class from `@freearhey/storage-js` handles the file I/O operations.