# What Data Sources Can Semantica Ingest? A Complete Guide to 15+ Supported Types

> Explore over 15 data sources Semantica ingests, from local files and web pages to databases, Git, and enterprise systems. Unlock your data's potential.

- Repository: [Semantica /semantica](https://github.com/semantica-agi/semantica)
- Tags: how-to-guide
- Published: 2026-09-06

---

**Semantica can ingest data from more than 15 distinct source categories including local files (PDF, DOCX, CSV, JSON), web pages, databases (PostgreSQL, MySQL, Snowflake), message streams (Kafka, RabbitMQ), Git repositories, and enterprise systems like SAP and Databricks.**

The ingestion framework in the **semantica-agi/semantica** repository provides dedicated *Ingestor* classes for each source type, all implemented under the `semantica.ingest` package. Every ingestor converts raw inputs into semantically-rich entities ready for the knowledge graph pipeline, with built-in provenance tracking and flexible authentication options.

## Files and Documents

Semantica handles virtually any document format through `FileIngestor` in [[`semantica/ingest/file_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/file_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/file_ingestor.py). The `FileTypeDetector` subsystem identifies formats via extension, MIME type, or magic-number analysis.

Supported file types include:

- **PDF, DOCX, PPTX, HTML, TXT** — standard office and web documents
- **CSV, JSON, YAML, XML** — structured data formats
- **Excel** — spreadsheet files

```python
from semantica.ingest import FileIngestor

ingestor = FileIngestor()
documents = ingestor.ingest_directory("./contracts/", recursive=True)

print(f"Ingested {len(documents)} documents")

```

## Web and API Sources

For external data, Semantica offers multiple specialized ingestors:

### Web Pages

`WebIngestor` ([[`web_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/web_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/web_ingestor.py)) fetches publicly reachable URLs with configurable crawl behavior:

```python
from semantica.ingest import WebIngestor

ingestor = WebIngestor(delay=1.0, respect_robots=True)
page = ingestor.ingest_url("https://example.com/annual-report.html")

print(page.text[:200])

```

### RSS and Atom Feeds

`FeedIngestor` handles syndicated content streams via ([[`feed_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/feed_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/feed_ingestor.py)).

### REST APIs

`RESTIngestor` and `PublicAPIIngestor` in ([[`api_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/api_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/api_ingestor.py)) support generic HTTP/HTTPS endpoints with optional authentication headers.

## Databases and Data Warehouses

### Relational Databases

`DBIngestor` ([[`db_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/db_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/db_ingestor.py)) supports PostgreSQL, MySQL, SQLite, Oracle, and SQL Server:

```python
from semantica.ingest import DBIngestor

ingestor = DBIngestor()
rows = ingestor.ingest_database(
    dialect="postgresql",
    host="db.example.com",
    port=5432,
    database="sales",
    username="alice",
    password="••••••"
)

```

### Cloud Data Warehouses

- **Snowflake** — via `SnowflakeIngestor` in ([[`snowflake_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/snowflake_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/snowflake_ingestor.py))
- **Databricks** — Unity Catalog and Delta Lake via `DatabricksIngestor` in ([[`databricks_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/databricks_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/databricks_ingestor.py))

## Columnar and Arrow Formats

### Parquet Datasets

`ParquetIngestor` ([[`parquet_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/parquet_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/parquet_ingestor.py)) handles high-performance columnar storage:

```python
from semantica.ingest import ParquetIngestor

ingestor = ParquetIngestor()
records = ingestor.ingest("./data/transactions.parquet")

```

### Apache Arrow

`ArrowIngestor` ([[`arrow_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/arrow_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/arrow_ingestor.py)) supports Arrow, Feather, and IPC files:

```python
from semantica.ingest import ArrowIngestor

arrow_ingestor = ArrowIngestor()
table = arrow_ingestor.ingest("./datasets/sample.arrow")

```

## Enterprise and Specialized Sources

### SAP ERP

`SAPIngestor` ([[`sap_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/sap_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/sap_ingestor.py)) connects to SAP systems via OData v2/v4 protocols.

### Version Control

`RepoIngestor` ([[`repo_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/repo_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/repo_ingestor.py)) clones and ingests Git repositories:

```python
from semantica.ingest import RepoIngestor

repo_ingestor = RepoIngestor()
repo_data = repo_ingestor.ingest_repository(
    url="https://github.com/example/project.git",
    branch="main"
)

```

### Email Systems

`EmailIngestor` ([[`email_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/email_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/email_ingestor.py)) connects to IMAP and POP3 mailboxes.

## Message Streams and Real-Time Data

For continuous data flows, Semantica implements `StreamIngestor` and `MessageIngestor` supporting:

- **Kafka**
- **RabbitMQ**
- **Amazon Kinesis**
- **Apache Pulsar**

These classes enable real-time ingestion into the knowledge graph pipeline.

## MCP Resources

`MCPIngestor` ([[`mcp_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/mcp_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/mcp_ingestor.py)) handles custom "MCP" (Model Context Protocol) assets for extended integration scenarios.

## How Ingestion Architecture Works

The `semantica.ingest` package follows a consistent five-stage pattern across all ingestors:

1. **Discovery & Loading** — Top-level functions (`ingest_file`, `ingest_web`, `ingest_database`) instantiate the appropriate ingestor
2. **Type Detection** — `FileTypeDetector` identifies formats automatically
3. **Parsing & Normalisation** — Sources convert to canonical triplets (`source`, `target`, `type`)
4. **Provenance Tracking** — `IngestProvenanceMixin` adds source lineage metadata
5. **Streaming & Batching** — `StreamIngestor` handles continuous flows; `FileIngestor` supports recursive directory processing

## Summary

- **15+ source categories** are supported in Semantica's ingestion framework
- **Dedicated ingestor classes** in `semantica.ingest` handle each source type with consistent interfaces
- **File formats** span documents, structured data, columnar storage, and Apache Arrow
- **Databases** include PostgreSQL, MySQL, SQLite, Oracle, SQL Server, Snowflake, and Databricks Delta Lake
- **Real-time streams** via Kafka, RabbitMQ, Kinesis, and Pulsar use `StreamIngestor` and `MessageIngestor`
- **Enterprise integration** covers SAP OData, Git repositories, IMAP/POP3 email, and custom MCP resources
- **Provenance tracking** and **batch/recursive processing** are built into the core architecture

## Frequently Asked Questions

### Does Semantica support Microsoft Office documents?

Yes. `FileIngestor` in [[`file_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/file_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/file_ingestor.py) natively handles DOCX, PPTX, and Excel formats through automatic type detection.

### Can Semantica ingest from private APIs requiring authentication?

Yes. `RESTIngestor` and `PublicAPIIngestor` in [[`api_ingestor.py`](https://github.com/semantica-agi/semantica/blob/main/api_ingestor.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/ingest/api_ingestor.py) support custom headers and authentication tokens passed via environment variables or explicit arguments.

### Is streaming data ingestion supported for real-time pipelines?

Yes. `StreamIngestor` and `MessageIngestor` enable continuous ingestion from Kafka, RabbitMQ, Kinesis, and Pulsar, integrating streaming sources directly with the knowledge graph pipeline.