Supported Content Extraction Formats in Hister: CLI and HTTP API Guide

Hister supports four content extraction formats—text (default), JSON, CSV, and RSS—available via the CLI --format flag or HTTP API query parameters, with RSS restricted to web endpoints only.

Hister is an open-source search engine written in Go that provides flexible content extraction capabilities for both command-line and programmatic workflows. Understanding the supported content extraction formats in Hister is essential for integrating search results into applications, data pipelines, or feed readers. The repository at asciimoo/hister implements these formats across the core command implementation and HTTP server files.

Overview of Supported Output Formats

Hister exposes four distinct output formats through its search interface. Each format serves specific use cases ranging from human-readable debugging to machine-readable data exchange.

Text Format (Default)

The default text format provides plain-text representation of search results, rendering each document as a readable snippet suitable for terminal viewing. This format activates automatically when omitting the --format parameter and requires no additional configuration.

JSON Format

JSON output delivers a structured API response containing search metadata (search_duration, total_results) alongside an array of document objects with fields including title, url, text, and metadata. This format is ideal for application integration and automated parsing.

CSV Format

CSV output generates comma-separated values suitable for spreadsheet applications or data analysis pipelines. The format supports customizable field selection through the --csv-fields option, allowing users to specify which document attributes appear in the output.

RSS Format

RSS 2.0 feed generation is available exclusively through the HTTP API at /search?format=rss, making search results consumable by feed readers and RSS aggregators. This format is not exposed through the CLI interface.

CLI Implementation and Format Selection

The command-line interface defines the global --format (or -f) flag in cmd/root.go, which the search command implementation in cmd/search.go processes to dispatch the appropriate output renderer.

CLI usage examples:

# Plain-text (default)

hister search "golang concurrency"

# JSON output

hister search "golang concurrency" --format json

# CSV output, selecting only title and URL fields

hister search "golang concurrency" --format csv --csv-fields title,url

The search command validates the format parameter against supported values and delegates rendering to format-specific handlers within cmd/search.go.

HTTP API Endpoint Handling

The web interface in server/endpoints.go processes the ?format= query parameter to serve JSON, CSV, and RSS responses. Unlike the CLI, the HTTP endpoint exclusively supports RSS generation, enabling feed reader integration.

HTTP endpoint examples:

GET /search?query=golang+concurrency&format=json   # JSON response

GET /search?query=golang+concurrency&format=csv    # CSV response

GET /search?query=golang+concurrency&format=rss    # RSS feed

The endpoint handlers construct appropriate content-type headers and serialization logic based on the requested format parameter.

Programmatic Access with the Go Client

The official Go client in client/search.go provides a type-safe interface for requesting specific content extraction formats programmatically. The SearchRequest struct accepts a Format field that forwards the desired output type to the server.

Programmatic usage example:

import (
    "github.com/asciimoo/hister/client"
    "context"
)

func main() {
    c := client.NewClient("http://localhost:8080")
    // JSON format
    resp, _ := c.Search(context.Background(),
        client.SearchRequest{
            Query:  "golang concurrency",
            Format: "json",
        })
    fmt.Println(string(resp.Body)) // raw JSON payload
}

Customizing CSV Field Selection

When using the CSV format, Hister allows granular control over output columns through the --csv-fields flag. This parameter accepts a comma-separated list of document attributes such as title, url, or text, determining which fields appear in the generated table.

Example:

hister search "docker containers" --format csv --csv-fields title,url

Summary

• Hister supports four content extraction formats: text, JSON, CSV, and RSS.
• The CLI accepts --format with values text, json, or csv, defined in cmd/root.go and implemented in cmd/search.go.
• RSS format is exclusively available via the HTTP API endpoint implemented in server/endpoints.go.
• CSV output supports customizable fields through the --csv-fields parameter.
• The Go client in client/search.go enables programmatic format selection via the SearchRequest struct.

Frequently Asked Questions

What content extraction formats does Hister support?

Hister supports four formats: plain text (default), JSON, CSV, and RSS 2.0. The CLI provides access to text, JSON, and CSV, while the HTTP API additionally supports RSS feeds as implemented in server/endpoints.go.

How do I export Hister search results to CSV?

Use the command hister search "query" --format csv and optionally specify fields with --csv-fields title,url,text. The implementation in cmd/search.go handles the serialization of document objects into comma-separated values based on the requested fields.

Can I generate RSS feeds from Hister search results?

Yes, but only through the HTTP API. Send a GET request to /search?query=your+search&format=rss as implemented in server/endpoints.go. The CLI does not support RSS output generation because the RSS formatter is excluded from cmd/search.go.

Which source files control format output in Hister?

The format flag definition resides in cmd/root.go, while cmd/search.go dispatches CLI format rendering. HTTP format handling occurs in server/endpoints.go, and the Go client implementation in client/search.go manages programmatic format requests through the Search method.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}