# How to Integrate LogSentinelAI with Elasticsearch for SIEM

> Integrate LogSentinelAI with Elasticsearch for SIEM. Enrich logs with host metadata and index them for Kibana visualization. Receive real-time Telegram alerts for critical events.

- Repository: [JungJungIn/logsentinelai](https://github.com/call518/logsentinelai)
- Tags: how-to-guide
- Published: 2026-02-26

---

**LogSentinelAI integrates with Elasticsearch by enriching LLM-analyzed log chunks with host metadata and indexing them into configurable indices via the `send_to_elasticsearch_raw()` function, enabling SIEM visualization through Kibana while supporting real-time Telegram alerts for high-severity events.**

LogSentinelAI provides a modular pipeline for LLM-based log analysis that natively supports Elasticsearch integration for SIEM workflows. By configuring environment variables in the `call518/logsentinelai` repository and utilizing the built-in enrichment engine, security teams can automatically index structured threat intelligence into Elasticsearch indices for real-time dashboarding and alerting.

## Elasticsearch Configuration Environment

LogSentinelAI reads Elasticsearch connection parameters from environment variables defined in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py). The system exposes module globals including `ELASTICSEARCH_HOST`, `ELASTICSEARCH_USER`, `ELASTICSEARCH_PASSWORD`, and `ELASTICSEARCH_INDEX` that control where analyzed logs are stored.

Create a `.env` file or export variables directly:

```dotenv
ELASTICSEARCH_HOST=http://localhost:9200
ELASTICSEARCH_USER=elastic
ELASTICSEARCH_PASSWORD=changeme
ELASTICSEARCH_INDEX=logsentinelai-analysis

```

These values are loaded when `config.apply_config()` executes during package initialization. The configuration module validates that these globals are available before any indexing operations begin.

## Core Integration Architecture

### Elasticsearch Client Initialization

The [`src/logsentinelai/core/elasticsearch.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/elasticsearch.py) file manages the Elasticsearch connection through `get_elasticsearch_client()`. This function instantiates a reusable `Elasticsearch` object and verifies connectivity before returning the client to the calling code.

### Data Enrichment Pipeline

Before indexing, the `send_to_elasticsearch_raw()` function performs critical enrichment steps defined in [`src/logsentinelai/core/elasticsearch.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/elasticsearch.py). The pipeline:

- Generates unique document IDs using the format `log_type_YYYYMMDD_hhmmss_ffffff[_chunk_N]`
- Attaches host-level metadata via `get_host_metadata()`
- Adds precise timestamps and GeoIP data when available
- Validates event severity levels for alert triggers

### Centralized Routing

The [`src/logsentinelai/core/commons.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/commons.py) file provides a higher-level abstraction that forwards enriched data from various analyzers to the Elasticsearch indexing function. This ensures consistent data formatting regardless of the log source type.

## End-to-End Data Flow

The integration follows a five-stage pipeline orchestrated by [`src/logsentinelai/cli.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/cli.py):

1. **Log Ingestion** – An analyzer (such as [`analyzers/httpd_access.py`](https://github.com/call518/logsentinelai/blob/main/analyzers/httpd_access.py)) reads raw log chunks from the filesystem
2. **LLM Analysis** – The system sends chunks to the LLM via [`core/llm.py`](https://github.com/call518/logsentinelai/blob/main/core/llm.py) and receives structured JSON payloads identifying security events
3. **Enrichment** – `send_to_elasticsearch_raw()` attaches host metadata, timestamps, and document identifiers
4. **Alert Evaluation** – If `TELEGRAM_ENABLED` is true and events meet the `TELEGRAM_ALERT_LEVEL` threshold, the system dispatches real-time notifications
5. **Indexing** – The final JSON document is pushed to the configured Elasticsearch index via `client.index()`, returning `True` only when the response contains `"result": "created"` or `"updated"`

## Implementation Methods

### Method 1: CLI-Based Integration

The simplest approach uses the built-in command-line interface to process logs and automatically index results:

```bash
pip install -r requirements.txt
logsentinelai run --log-type httpd_access --log-path /var/log/apache/access.log

```

The CLI invokes `process_log_chunk()` from [`src/logsentinelai/core/commons.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/commons.py), which handles the complete enrichment and indexing workflow without requiring custom code.

### Method 2: Programmatic Python Integration

For custom SIEM pipelines, import the Elasticsearch functions directly:

```python
from logsentinelai.core.elasticsearch import send_to_elasticsearch_raw

analysis_result = {
    "events": [
        {"severity": "CRITICAL", "event_type": "SQLi", "description": "Possible SQL injection detected"},
        {"severity": "HIGH", "event_type": "PortScan", "description": "Multiple ports probed"}
    ],
    "summary": "2 security events detected"
}

success = send_to_elasticsearch_raw(
    data=analysis_result,
    log_type="httpd_access",
    chunk_id=1
)

print(f"Document indexed: {success}")

```

This approach allows integration with existing Python-based security orchestration tools while maintaining LogSentinelAI's enrichment capabilities.

### Method 3: Custom Analyzer Integration

When building custom log analyzers, leverage the commons helper to ensure consistent Elasticsearch formatting:

```python
from logsentinelai.core.commons import process_log_chunk

# Custom analyzer implementation

def my_custom_analyzer(log_path):
    # Analysis logic here

    results = analyze_logs(log_path)
    return results

# The commons module handles Elasticsearch routing automatically

```

## Enabling Real-Time Security Alerts

LogSentinelAI supports Telegram integration for immediate notification of high-severity events. Configure these additional environment variables alongside your Elasticsearch settings:

```dotenv
TELEGRAM_ENABLED=true
TELEGRAM_TOKEN=YOUR_BOT_TOKEN
TELEGRAM_CHAT_ID=YOUR_CHAT_ID
TELEGRAM_ALERT_LEVEL=HIGH

```

When `send_to_elasticsearch_raw()` processes events with severity levels matching or exceeding `TELEGRAM_ALERT_LEVEL`, it triggers `utils.telegram_alert.send_telegram_alert()` to dispatch formatted alerts containing event details and host metadata before completing the Elasticsearch indexing operation.

## Verifying the Integration

After running your first analysis, verify the integration by querying the target index:

```bash
curl -X GET "localhost:9200/logsentinelai-analysis/_search?pretty" \
  -H "Content-Type: application/json" \
  -d '{"query": {"match_all": {}}}'

```

Successful integration returns documents containing the enriched fields added by `send_to_elasticsearch_raw()`, including `host_metadata`, `timestamp`, and the unique `chunk_id` identifier.

## Summary

- **Configuration**: Set `ELASTICSEARCH_HOST`, `ELASTICSEARCH_USER`, `ELASTICSEARCH_PASSWORD`, and `ELASTICSEARCH_INDEX` environment variables in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py) before running analyses
- **Client Management**: The `get_elasticsearch_client()` function in [`src/logsentinelai/core/elasticsearch.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/elasticsearch.py) handles connection pooling and health checks
- **Enrichment**: `send_to_elasticsearch_raw()` automatically adds host metadata, timestamps, and unique document IDs using the format `log_type_YYYYMMDD_hhmmss_ffffff[_chunk_N]`
- **Routing**: [`src/logsentinelai/core/commons.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/commons.py) provides the central routing layer that connects analyzers to Elasticsearch storage
- **Alerts**: Optional Telegram notifications trigger when event severity meets the configured `TELEGRAM_ALERT_LEVEL` threshold

## Frequently Asked Questions

### How do I configure Elasticsearch authentication credentials?

LogSentinelAI uses environment variables defined in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py). Export `ELASTICSEARCH_USER` and `ELASTICSEARCH_PASSWORD` before starting the application, or define them in a `.env` file. The `get_elasticsearch_client()` function reads these globals to instantiate the authenticated client connection.

### What happens if Elasticsearch is unreachable during analysis?

The `send_to_elasticsearch_raw()` function returns `True` only when the Elasticsearch response contains `"result": "created"` or `"updated"`. If the connection fails or the cluster returns an error, the function returns `False`, allowing calling code in [`src/logsentinelai/core/commons.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/commons.py) to handle retries or logging failures appropriately.

### Can I customize the index name or document ID format?

Yes. The `ELASTICSEARCH_INDEX` environment variable controls the target index name. While the document ID format `log_type_YYYYMMDD_hhmmss_ffffff[_chunk_N]` is generated automatically within `send_to_elasticsearch_raw()`, you can modify the indexing logic in [`src/logsentinelai/core/elasticsearch.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/elasticsearch.py) if your SIEM requires alternative naming conventions.

### Is Telegram alerting required for Elasticsearch integration?

No. Telegram integration is optional. If `TELEGRAM_ENABLED` is not set to `true`, the system skips alert generation and proceeds directly to indexing. The alerting logic is embedded in `send_to_elasticsearch_raw()` but only executes when the configuration variables are present and event severity meets the `TELEGRAM_ALERT_LEVEL` threshold.