How to Build Custom Kibana Dashboards for LogSentinelAI Data

You can build custom Kibana dashboards for LogSentinelAI by extending the JSON data model in src/logsentinelai/core/elasticsearch.py, creating Lens visualizations from the logsentinelai-analysis-* index pattern, and exporting your dashboard objects as NDJSON for version control and automated deployment.

LogSentinelAI is an AI-driven log analysis engine that transforms raw log files into structured JSON events enriched with host metadata and GeoIP information. To build custom Kibana dashboards for LogSentinelAI data, you need to understand how the pipeline indexes documents into Elasticsearch and how to manipulate the saved objects that define your visualizations.

Understanding the LogSentinelAI Data Pipeline

Before creating visualizations, you must understand how LogSentinelAI structures and indexes data into Elasticsearch.

Log Ingestion and Analysis

The CLI analyzers (logsentinelai-httpd-access, logsentinelai-linux-system, etc.) read raw log files and feed them to a Large Language Model (LLM). The LLM produces a Pydantic-validated JSON payload containing parsed events, severity levels, and source IP addresses.

Elasticsearch Indexing and Enrichment

In src/logsentinelai/core/elasticsearch.py, the system constructs enriched JSON payloads before indexing:

# src/logsentinelai/core/elasticsearch.py (excerpt)

enriched_data = {
    **data,
    "@timestamp": datetime.datetime.utcnow().isoformat(),
    "@log_type": log_type,
    "@document_id": doc_id,
    **host_metadata,
}

The index name and connection parameters are defined in src/logsentinelai/core/config.py. Documents are pushed to the logsentinelai-analysis index, which Kibana accesses via the logsentinelai-analysis-* index pattern.

Setting Up the Default Kibana Dashboard

LogSentinelAI ships with pre-configured saved objects that provide a starting point for customization.

Importing Saved Objects

The repository includes two NDJSON files in the root directory:

• Kibana-9.0.3-Advanced-Settings.ndjson – Pre-configured UI settings (theme, default time range)
• Kibana-9.0.3-Dashboard-LogSentinelAI.ndjson – Index pattern, Lens visualizations, and the default dashboard

Import these via the Kibana UI (Stack Management → Saved Objects → Import) or using the API as shown in the INSTALL-and-USAGE.md guide.

Default Index Pattern

When imported, Kibana automatically creates the logsentinelai-analysis-* index pattern with @timestamp as the time field. This pattern powers all visualizations and must exist before building custom dashboards.

Building Custom Kibana Visualizations

To build custom Kibana dashboards for LogSentinelAI data, you can extend the data model, create new Lens visualizations, and manage the exported objects.

Extending the Data Model

Add custom fields to the Elasticsearch documents by modifying src/logsentinelai/core/elasticsearch.py:

# src/logsentinelai/core/elasticsearch.py (excerpt)

enriched_data = {
    **data,
    "@timestamp": datetime.datetime.utcnow().isoformat(),
    "@log_type": log_type,
    "@document_id": doc_id,
    **host_metadata,
    # ---- Custom field added by you ----

    "custom_department": "finance",          # <-- new attribute

}

After deploying your updated code, re-run a CLI analyzer (e.g., logsentinelai-linux-system). The new custom_department field will appear in the index and can be used in Kibana visualizations.

Creating Lens Visualizations

1. Open Kibana → Visualize → Create visualization → Lens.
2. Select the logsentinelai-analysis-* index pattern.
3. Drag events.severity to the Break down by bucket and events (or @log_type) to the Metric (e.g., Count).
4. Choose Bar or Donut chart, click Save as "Severity Breakdown – Custom".

Exporting and Importing Dashboard Objects

Export your custom dashboard to version control:

# From the repository root

# Export all saved objects (includes the new visualization)

curl -X GET "http://localhost:5601/api/saved_objects/_export" \
  -H "kbn-xsrf: true" \
  -u elastic:changeme \
  -o custom-dashboard.ndjson

Import into another environment:

curl -X POST "http://localhost:5601/api/saved_objects/_import" \
  -H "kbn-xsrf: true" \
  -F file=@custom-dashboard.ndjson \
  -u elastic:changeme

When replacing only the dashboard, edit custom-dashboard.ndjson to keep the existing index-pattern and advanced-settings objects and remove duplicates.

Automating Dashboard Deployment

Integrate dashboard updates into your CI/CD pipeline:

# .github/workflows/kibana-import.yml (excerpt)

- name: Import Kibana objects
  run: |
    curl -X POST "http://localhost:5601/api/saved_objects/_import" \
      -H "kbn-xsrf: true" \
      -F file=@Kibana-9.0.3-Dashboard-LogSentinelAI.ndjson \
      -u ${{ secrets.ELASTIC_USER }}:${{ secrets.ELASTIC_PASSWORD }}

Summary

• LogSentinelAI enriches raw logs into structured JSON and indexes them into the logsentinelai-analysis index via src/logsentinelai/core/elasticsearch.py.
• The repository provides pre-built NDJSON saved objects (Kibana-9.0.3-Dashboard-LogSentinelAI.ndjson) that create the index pattern and default visualizations.
• You can build custom Kibana dashboards for LogSentinelAI data by extending the Python data model, creating Lens visualizations from the logsentinelai-analysis-* index pattern, and managing objects via the Kibana Import/Export API.
• Version control your custom dashboards by exporting NDJSON files and automating deployment through CI/CD pipelines.

Frequently Asked Questions

How do I add custom fields to LogSentinelAI data for use in Kibana?

Modify src/logsentinelai/core/elasticsearch.py to include additional key-value pairs in the enriched_data dictionary before it is sent to Elasticsearch. After redeploying and re-running the CLI analyzer, the new fields will be available in the logsentinelai-analysis-* index pattern for visualization.

What is the default index pattern used by LogSentinelAI in Kibana?

The default index pattern is logsentinelai-analysis-*, which matches all indices created by the LogSentinelAI indexing process. This pattern is automatically created when you import the Kibana-9.0.3-Dashboard-LogSentinelAI.ndjson saved objects file, with @timestamp configured as the time field.

Can I automate the deployment of custom Kibana dashboards across environments?

Yes. Export your custom dashboards and visualizations as NDJSON files using the Kibana Saved Objects Export API (/api/saved_objects/_export). Store these files in version control, then use the Import API (/api/saved_objects/_import) in your CI/CD pipelines—such as GitHub Actions—to automatically deploy updates to staging or production Kibana instances.

Where are the Kibana configuration files located in the LogSentinelAI repository?

The pre-configured Kibana saved objects are located in the repository root as Kibana-9.0.3-Advanced-Settings.ndjson and Kibana-9.0.3-Dashboard-LogSentinelAI.ndjson. Configuration for the Elasticsearch connection and index naming is found in src/logsentinelai/core/config.py, while the indexing logic resides in src/logsentinelai/core/elasticsearch.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}