# How to Add a Document to Hister Using the API: A Complete Guide

> Learn to add documents to Hister using the API. This guide covers POST requests to /api/add and /api/batch with Go client or raw HTTP calls for efficient document management.

- Repository: [Adam Tauber/hister](https://github.com/asciimoo/hister)
- Tags: how-to-guide
- Published: 2026-09-01

---

**You can add documents to Hister via POST requests to `/api/add` for single documents or `/api/batch` for bulk operations, using either the Go client library or raw HTTP calls with JSON or form-encoded payloads.**

Hister is an open-source search engine that exposes a straightforward HTTP API for document ingestion. Whether you are building a crawler or integrating existing content sources, adding a document to Hister requires sending structured data to specific REST endpoints implemented in the Go-based server.

## API Endpoints for Document Ingestion

Hister provides two primary HTTP endpoints for inserting documents into the index:

- **`POST /api/add`** – Accepts a single document via `application/json` or `application/x-www-form-urlencoded` content types.
- **`POST /api/batch`** – Accepts multiple documents in a single request using a JSON body with an `"ops"` array, supporting a maximum of 100 operations per call.

Both endpoints enforce request size limits and validate payloads before indexing. The server-side logic resides in [`server/endpoints.go`](https://github.com/asciimoo/hister/blob/main/server/endpoints.go), where the `serveAdd` function (lines 731‑800) handles single-document requests and orchestrates the validation and storage pipeline.

## Adding a Single Document

### Using Raw HTTP Requests

For simple integrations, send a JSON payload directly to the `/api/add` endpoint:

```bash
curl -X POST http://localhost:8080/api/add \
     -H "Content-Type: application/json" \
     -d '{
           "url":"https://example.com/article",
           "title":"Example Article",
           "text":"The article body…"
         }'

```

Alternatively, use URL-encoded form data if your client does not support JSON:

```bash
curl -X POST http://localhost:8080/api/add \
     -H "Content-Type: application/x-www-form-urlencoded" \
     --data-urlencode "url=https://example.com/article" \
     --data-urlencode "title=Example Article" \
     --data-urlencode "text=The article body…"

```

### Using the Go Client Library

The recommended approach for Go projects uses the `client` package. The `AddDocumentJSON` method constructs the request and handles response parsing:

```go
import (
    "github.com/asciimoo/hister/client"
    "github.com/asciimoo/hister/server/document"
)

func addSingleDocument(c *client.Client) error {
    doc := &document.Document{
        URL:   "https://example.com/article",
        Title: "Example Article",
        Text:  "The article body…",
    }
    // POST /api/add
    return c.AddDocumentJSON(doc)
}

```

The client automatically manages connection pooling and error handling while targeting the same underlying server endpoint.

## Adding Multiple Documents in Bulk

For high-throughput scenarios, batching documents reduces network overhead. The `/api/batch` endpoint accepts an `"ops"` array where each element specifies an `"add"` operation with a nested document object.

### Batch HTTP Request Example

```bash
curl -X POST http://localhost:8080/api/batch \
     -H "Content-Type: application/json" \
     -d '{
           "ops": [
             {"op":"add","document":{"url":"https://example.com/first","title":"First","text":"First body"}},
             {"op":"add","document":{"url":"https://example.com/second","title":"Second","text":"Second body"}}
           ]
         }'

```

The server enforces a hard limit of 100 operations per request and validates the total request size before processing.

### Bulk Insertion with the Go Client

The `AddDocumentsJSON` function in [`client/document.go`](https://github.com/asciimoo/hister/blob/main/client/document.go) (lines 32‑73) abstracts batch construction and automatically handles the 100-item limit by splitting large slices into multiple API requests:

```go
func addMultipleDocuments(c *client.Client) error {
    docs := []*document.Document{
        {URL: "https://example.com/first", Title: "First", Text: "First body"},
        {URL: "https://example.com/second", Title: "Second", Text: "Second body"},
    }
    results, err := c.AddDocumentsJSON(docs)
    if err != nil {
        return err
    }
    for _, r := range results {
        if r.Status != http.StatusCreated {
            return fmt.Errorf("failed: %s (status %d)", r.Error, r.Status)
        }
    }
    return nil
}

```

This method returns per-document status codes, allowing you to identify specific failures within a batch.

## Validation and Response Codes

Before indexing, Hister validates every document using the `validateAddDocument` function in [`server/endpoints.go`](https://github.com/asciimoo/hister/blob/main/server/endpoints.go) (lines 803‑843). This validation ensures:

- The **URL** field is well-formed and present.
- For remote-file types, the URL uses an allowed scheme (HTTP/HTTPS).

After validation, the server invokes `c.Indexer.AddContext` to store the document. The API returns distinct HTTP status codes:

- **HTTP 201 Created** – The document was successfully indexed.
- **HTTP 406 Not Acceptable** – The document was rejected due to user-defined indexing rules (skip filters) or validation failures.

## Key Implementation Files

Understanding the source structure helps when debugging or extending the API:

- **[`server/endpoints.go`](https://github.com/asciimoo/hister/blob/main/server/endpoints.go)** – Contains `serveAdd` (single document handler) and `validateAddDocument` (validation logic).
- **[`client/document.go`](https://github.com/asciimoo/hister/blob/main/client/document.go)** – Implements `AddDocumentJSON` and `AddDocumentsJSON` for client-side batching.
- **[`server/document/document.go`](https://github.com/asciimoo/hister/blob/main/server/document/document.go)** – Defines the `Document` struct and processing helpers used by both server and client.

## Summary

- Hister exposes **POST `/api/add`** for single documents and **POST `/api/batch`** for bulk operations up to 100 items.
- The API accepts **`application/json`** or **`application/x-www-form-urlencoded`** payloads.
- The **Go client library** provides `AddDocumentJSON` and `AddDocumentsJSON` methods that handle batching and error checking automatically.
- Successful indexing returns **HTTP 201**, while skipped documents return **HTTP 406**.
- Validation occurs in `validateAddDocument` within [`server/endpoints.go`](https://github.com/asciimoo/hister/blob/main/server/endpoints.go), ensuring URL integrity before storage.

## Frequently Asked Questions

### What is the maximum number of documents per batch request?

The `/api/batch` endpoint enforces a maximum of **100 operations per request**. The Go client library automatically partitions larger slices into multiple compliant requests, but raw HTTP callers must implement this splitting logic manually.

### What HTTP status codes does the Hister API return when adding documents?

The API returns **HTTP 201 Created** when a document is successfully indexed and stored. It returns **HTTP 406 Not Acceptable** when the document passes validation but is skipped due to user-defined indexing rules or filters.

### Can I add documents using form-encoded data instead of JSON?

Yes. While JSON is recommended, the `/api/add` endpoint also accepts **`application/x-www-form-urlencoded`** content. Map the `url`, `title`, and `text` fields as form parameters when using this format.

### How does the Go client handle bulk document insertion?

The `AddDocumentsJSON` function in [`client/document.go`](https://github.com/asciimoo/hister/blob/main/client/document.go) accepts a slice of `*document.Document` structs, automatically chunks them into batches of 100 or fewer, and sends sequential POST requests to `/api/batch`. It returns a slice of results containing individual status codes and error messages for each document.