# What Is the File Indexing Manager in OpenCTI and How Does It Work?

> Discover OpenCTI's file indexing manager. Learn how this background module creates a searchable index of platform attachments, synchronizing metadata and content with access controls in real time.

- Repository: [OpenCTI Platform/opencti](https://github.com/opencti-platform/opencti)
- Tags: internals
- Published: 2026-02-19

---

**The file indexing manager in OpenCTI is a background module that maintains a searchable Elasticsearch or OpenSearch index of every platform attachment, synchronizing file metadata and content with entity-level access controls in real time.**

This specialized manager ensures that uploaded documents, reports, and artifacts remain discoverable through the platform’s search interface while respecting dynamic security markings. According to the OpenCTI-Platform/opencti source code, the feature is available exclusively in the Enterprise Edition and operates through a combination of periodic batch jobs and event-driven stream processing.

## Core Architecture and Responsibilities

The file indexing manager serves as the bridge between OpenCTI’s file storage layer and its search infrastructure. It extracts binary content, generates internal identifiers, and maintains strict consistency between file accessibility and parent entity restrictions.

### Periodic File Ingestion

At its core, the manager runs a **locked-resource job** every minute (configurable via `file_index_manager:interval`) to index newly imported files. The `fileIndexHandler` function in [`src/manager/fileIndexManager.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/manager/fileIndexManager.ts) acquires a distributed lock using `FILE_INDEX_MANAGER_KEY` to prevent duplicate execution across clustered API nodes.

Once locked, the handler verifies Enterprise Edition status through `isEnterpriseEditionFromSettings`, retrieves the last successful run timestamp via `getIndexFromDate`, and delegates to `indexImportedFiles`. This function processes files in **batches of 20**, streaming content with `BluePromise.map` at a concurrency of 5 to balance throughput with resource utilization.

### Real-Time Stream Processing

Beyond periodic scans, the manager listens to the platform’s event stream to handle immediate updates. The `handleStreamEvents` function monitors `UPDATE` events for STIX objects that own files. When the patch contains `granted_refs` or `object_marking_refs` changes, the manager triggers `elUpdateFilesWithEntityRestrictions` to propagate access control modifications to the search index without waiting for the next batch cycle.

### Enterprise Edition Requirements

The manager is **strictly an Enterprise-only feature**. The bootstrap logic in [`src/managers.js`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/managers.js) checks `ENABLED_FILE_INDEX_MANAGER` and `isAttachmentProcessorEnabled()` before invoking `fileIndexManager.start()`. Additionally, the runtime handlers verify `isEnterpriseEditionFromSettings` before executing indexing operations, ensuring community editions skip the overhead entirely.

## Configuration and Initialization

System administrators control the manager through settings defined in [`src/config/conf.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/config/conf.ts). Key configuration parameters include:

- **`file_index_manager:interval`** – Polling frequency in milliseconds (default: 60000 for 1 minute)
- **`file_index_manager:lock_key`** – Distributed lock identifier for the periodic job
- **`file_index_manager:stream_lock_key`** – Lock key for the stream processor
- **`ENABLED_FILE_INDEX_MANAGER`** – Boolean flag enabling the module

The initialization sequence in [`src/managers.js`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/managers.js) demonstrates the conditional startup:

```javascript
import fileIndexManager from './manager/fileIndexManager';

if (ENABLED_FILE_INDEX_MANAGER && isAttachmentProcessorEnabled()) {
  await fileIndexManager.start();
}

```

The `start` method creates two asynchronous intervals: one for the periodic indexing job and one for the stream handler, both managed through the `initFileIndexManager` lifecycle.

## Indexing Workflow and Code Implementation

### The fileIndexHandler Loop

The primary indexing logic resides in [`src/manager/fileIndexManager.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/manager/fileIndexManager.ts). The handler implements a defensive locking pattern to ensure singleton execution across distributed deployments:

```javascript
const fileIndexHandler = async () => {
  const settings = await getEntityFromCache(context, SYSTEM_USER, ENTITY_TYPE_SETTINGS);
  if (isEnterpriseEditionFromSettings(settings)) {
    const lock = await lockResources([FILE_INDEX_MANAGER_KEY], { retryCount: 0 });
    try {
      const managerConfiguration = await getManagerConfigurationFromCache(context, SYSTEM_USER, 'FILE_INDEX_MANAGER');
      if (managerConfiguration?.manager_running) {
        const indexFromDate = await getIndexFromDate(context);
        await indexImportedFiles(context, indexFromDate);
      }
    } finally {
      if (lock) await lock.unlock();
    }
  }
};

```

This pattern ensures that only one node processes files at any given time, while the `retryCount: 0` parameter causes immediate exit if another node holds the lock.

### Batch Processing and Content Extraction

The `indexImportedFiles` function (lines 69-78 in the manager file) queries for files created after the last indexed date, validates enabled MIME-type prefixes, and constructs documents containing:

- **File ID** and **generated internal OpenCTI identifier** (via `generateFileIndexId` from [`src/schema/identifier.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/schema/identifier.ts))
- **Owner entity** reference (if applicable)
- **File name** and **upload date**
- **Base64-encoded binary content**

These documents are sent to `elIndexFiles` in [`src/database/file-search.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/database/file-search.ts) for bulk insertion into the search cluster.

### Handling Entity Restriction Updates

When security markings change, the stream processor ensures index consistency:

```javascript
if (event.data.type === EVENT_TYPE_UPDATE) {
  const updateEvent = event.data;
  const isDataRestrictionsUpdate = updateEvent.context?.patch?.some(
    op => op.path?.includes('granted_refs') || op.path?.includes('object_marking_refs')
  );
  if (stixFiles?.length && isDataRestrictionsUpdate) {
    const entity = await internalLoadById(context, SYSTEM_USER, entityId, { type: entityType });
    await elUpdateFilesWithEntityRestrictions(entity);
  }
}

```

This code path prevents search results from returning files the user no longer has permission to access following entity updates.

## GraphQL API and Domain Functions

The platform exposes file indexing operations through GraphQL resolvers defined in [`src/resolvers/indexedFile.js`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/resolvers/indexedFile.js), with domain logic implemented in [`src/domain/indexedFile.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/domain/indexedFile.ts).

### Search and Metrics Queries

- **`indexedFilesMetrics`** – Returns statistics about the file index state
- **`indexedFiles`** – Paginated search results with filtering capabilities
- **`indexedFilesCount`** – Aggregate count matching specified criteria

### Resetting the Index

Administrators can completely rebuild the index using the `resetFileIndexing` mutation:

```graphql
mutation {
  resetFileIndexing
}

```

The resolver invokes `resetFileIndexing(context, user)` from the domain layer, which performs three critical operations: clearing the manager configuration state, deleting all indexed documents via `elDeleteAllFiles`, and publishing a user-action audit event. This function is located at lines 42-62 of [`src/domain/indexedFile.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/domain/indexedFile.ts).

### Graceful Shutdown

The `shutdown` method in `initFileIndexManager` clears active intervals and terminates the stream processor, ensuring no partial indexing operations corrupt the state during platform restarts.

## Summary

- The **file indexing manager in OpenCTI** maintains a real-time Elasticsearch/OpenSearch index of all platform attachments, enabling full-text search across uploaded content.
- It operates through **dual mechanisms**: a periodic batch job (default 1-minute intervals) processing 20 files per batch, and a **stream processor** reacting immediately to entity permission changes.
- The feature requires **Enterprise Edition** licensing and explicit configuration via `ENABLED_FILE_INDEX_MANAGER` in [`src/config/conf.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/config/conf.ts).
- Core implementation files include [`src/manager/fileIndexManager.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/manager/fileIndexManager.ts) for orchestration, [`src/domain/indexedFile.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/domain/indexedFile.ts) for GraphQL domain logic, and [`src/database/file-search.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/database/file-search.ts) for low-level Elasticsearch operations.
- **Distributed locking** via `FILE_INDEX_MANAGER_KEY` prevents race conditions in clustered deployments, while `BluePromise.map` concurrency controls resource utilization during bulk indexing.

## Frequently Asked Questions

### Does the file indexing manager work in OpenCTI Community Edition?

No. The file indexing manager is an **Enterprise-only feature** that requires a valid Enterprise Edition license. The platform explicitly checks `isEnterpriseEditionFromSettings` in `fileIndexHandler`, `fileIndexStreamHandler`, and the status endpoint before executing any indexing operations. Community Edition deployments will skip initialization entirely based on the `ENABLED_FILE_INDEX_MANAGER` flag check in [`src/managers.js`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/managers.js).

### How can administrators reset or rebuild the file index?

Administrators can trigger a complete index reset through the **GraphQL API** using the `resetFileIndexing` mutation. This operation, implemented in [`src/domain/indexedFile.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/domain/indexedFile.ts), clears the manager’s configuration cache, deletes all existing indexed documents via `elDeleteAllFiles`, and publishes an audit event. Alternatively, modifying the `file_index_manager:interval` configuration and restarting the platform allows for gradual re-indexing without data deletion.

### What happens to file indexing when entity permissions change?

When a STIX object owning files receives an update to its `granted_refs` (sharing) or `object_marking_refs` (classification) properties, the **stream processor** immediately detects the change through the `handleStreamEvents` function. It then invokes `elUpdateFilesWithEntityRestrictions` to update the indexed documents’ access control metadata in Elasticsearch, ensuring search results respect the latest security posture without requiring a full re-index.

### Which Elasticsearch operations does the manager perform?

The manager relies on three primary operations from [`src/database/file-search.ts`](https://github.com/OpenCTI-Platform/opencti/blob/main/src/database/file-search.ts): **`elIndexFiles`** for bulk insertion of new file documents, **`elUpdateFilesWithEntityRestrictions`** for patching access control metadata on existing entries, and **`elDeleteAllFiles`** for administrative index resets. These functions handle the translation between OpenCTI’s internal file identifiers (generated via `generateFileIndexId`) and the search cluster’s document structure.