# How Outfancy Auto-Detects Column Data Types: id, name, date, time, value, and desc

> Discover how Outfancy auto-detects column data types like id name date time value and desc using row sampling and heuristic analysis within Table.check_data_type_list_integrity for accurate classification.

- Repository: [Carlos A. Planchón/outfancy](https://github.com/carlosplanchon/outfancy)
- Tags: internals
- Published: 2026-02-26

---

**Outfancy automatically detects column data types by sampling up to 10 rows from your dataset and applying heuristics inside `Table.check_data_type_list_integrity` to classify fields as identifiers, timestamps, or text descriptions based on content patterns.**

Outfancy is a Python library for rendering tabular data in terminal environments. When you render a dataset without explicitly providing a `data_type_list`, the library analyzes your data to determine whether each column represents an **id**, **name**, **date**, **time**, **value**, or **desc** (description). This automatic detection happens in [`outfancy/table.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/table.py) and relies on configurable thresholds and pattern matching heuristics.

## The Detection Pipeline

The core logic resides in `Table.check_data_type_list_integrity` (lines 699‑754 of [`outfancy/table.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/table.py)). This method examines a representative sample of rows when `data_type_list` is missing or incomplete.

### Sampling Strategy

To maintain performance on large datasets, Outfancy limits analysis to a configurable **analysis threshold** defined by `self.analyze_threshold` (default: **10** rows). The method iterates over this subset and converts each field to a string for inspection. If you supply a partial `data_type_list`, only columns marked with `None` undergo automatic detection, while specified types remain untouched.

### Heuristic Classification

For each sampled field, Outfancy applies the following checks in order:

- **Hour detection**: Uses `widgets.is_complete_hour(field)` to validate against `%H:%M:%S` or `%H:%M` formats via `time.strptime`.
- **Date detection**: Uses `widgets.is_date(field)` to normalize text and test multiple date formats.
- **Numeric detection**: Checks `field.isdigit()` to identify potential **id** or **value** columns.
- **Long text detection**: If the field exceeds `self.chk_dlti_num_letters_in_field` (default **15** characters) and contains more than `self.chk_dlti_pecentage_letters_in_field` (default **90%**) alphabetic characters, it is classified as **name**.
- **Fallback**: Fields that are short or contain mixed content default to **desc**.

### Distinguishing ID from Value

When a field contains only digits, Outfancy performs a **sequence analysis** to differentiate sequential identifiers from arbitrary numeric values. The algorithm examines neighboring rows in the sample:

- If the next row exists and `int(next_field) - int(field) == 1`, the column is marked as **id**.
- If the previous row exists and `int(field) - int(prev_field) == 1`, this reinforces the **id** classification.
- Non-consecutive numeric fields or single-row datasets default to **value**.

### Threshold-Based Final Selection

After processing the sample, Outfancy counts occurrences of each temporary type and converts them to percentages. The final column type is the first category to meet its configured threshold (checked in priority order):

| Type | Threshold Attribute | Default |
|------|---------------------|---------|
| **date** | `self.chk_dtli_date` | 70% |
| **time** | `self.chk_dtli_time` | 70% |
| **id** | `self.chk_dtli_id` | 50% |
| **value** | `self.chk_dlti_value` | 100% |
| **name** | `self.chk_dlti_name` | 60% |
| **desc** | `self.chk_dlti_desc` | 60% |

If no threshold is satisfied, the column falls back to **desc**. Once complete, the method logs `column types auto-detected successfully` and returns the finalized list.

## Configuration and Threshold Defaults

The detection behavior is governed by attributes initialized in `Table.__init__` (lines 129‑146 of [`outfancy/table.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/table.py)). You can adjust these thresholds after instantiation to tune sensitivity:

- **ID sensitivity**: Lower `chk_dtli_id` (default 50) to catch sparser sequences, or raise it to require nearly perfect sequentiality.
- **Name length**: Modify `chk_dlti_num_letters_in_field` (default 15) to change the minimum character count for **name** classification.
- **Sample size**: Increase `analyze_threshold` (default 10) to scan more rows for detection, improving accuracy on heterogeneous columns.

## Practical Code Examples

### Automatic Detection

Render a dataset without specifying types to trigger auto-detection:

```python
from outfancy import Table
from outfancy.example_dataset import dataset

tbl = Table()
print(tbl.render(dataset))

```

### Overriding Specific Columns

Pass a `data_type_list` with `None` for auto-detected columns and explicit types for others:

```python
tbl = Table()

# Force column 3 to desc, keep auto-detection for the rest

custom_types = [None, None, 'desc', None, None, None, 'value', None]
print(tbl.render(dataset, data_type_list=custom_types))

```

### Tuning Detection Thresholds

Make ID detection stricter by requiring 80% consecutive sequences:

```python
tbl = Table()
tbl.chk_dtli_id = 80  # Default is 50

tbl.analyze_threshold = 20  # Scan more rows for better accuracy

print(tbl.render(dataset))

```

## Summary

- Outfancy detects column types automatically when `data_type_list` is omitted, using `Table.check_data_type_list_integrity` in [`outfancy/table.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/table.py).
- The algorithm samples **10 rows by default** and applies heuristics to identify **date**, **time**, **id**, **value**, **name**, and **desc** fields.
- **ID** columns are distinguished from **value** columns by checking for consecutive integer sequences across neighboring rows.
- Final classification uses **percentage thresholds** (e.g., 70% for dates, 100% for values) with a fallback to **desc**.
- All thresholds and sample sizes are configurable via attributes on the `Table` instance.

## Frequently Asked Questions

### How does Outfancy distinguish between id and value columns?

Outfancy checks if numeric fields form a consecutive sequence. If `int(next_field) - int(current_field) == 1` (or the reverse for previous rows), the column is classified as **id**; otherwise, it becomes **value**. Single-row datasets default to **value** since no sequence can be established.

### Can I disable automatic type detection in Outfancy?

Yes. Provide a complete `data_type_list` parameter to `Table.render()` with no `None` values. When every column has an explicit type assigned, `check_data_type_list_integrity` skips analysis entirely and uses your specifications.

### What happens if no data type thresholds are met?

If none of the percentage thresholds (70% for date/time, 50% for id, 100% for value, 60% for name/desc) are satisfied during sampling, the column automatically falls back to **desc** (description), ensuring the renderer always has a valid type assignment.

### Which source files handle the detection logic?

The primary logic lives in [`outfancy/table.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/table.py) (lines 699‑754) within the `check_data_type_list_integrity` method. Helper functions for time and date validation reside in [`outfancy/widgets.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/widgets.py) (`is_complete_hour` and `is_date`), while default threshold values are defined in the `Table` class constructor (lines 129‑146).