# Outfancy Data Type Detection Edge Cases: Common Pitfalls and Solutions

> Discover common pitfalls in Outfancy data type detection including negative numbers and ISO dates. Learn solutions to prevent misclassification and ensure accurate data analysis.

- Repository: [Carlos A. Planchón/outfancy](https://github.com/carlosplanchon/outfancy)
- Tags: deep-dive
- Published: 2026-02-26

---

**Outfancy’s automatic column type detection can misclassify negative numbers, ISO dates, unsorted IDs, and non-ASCII text as generic descriptions due to strict regex patterns and ASCII-only heuristics in the `check_data_type_list_integrity` method.**

Outfancy is a Python library for rendering terminal tables with intelligent column formatting. When processing datasets, the library attempts to automatically determine whether a column contains identifiers, names, dates, times, values, or descriptions. However, the **outfancy data type detection** algorithm employs specific heuristics that fail under several common edge cases, potentially causing misaligned columns and incorrect rendering priorities.

## How Outfancy Detects Data Types

The detection logic resides in **`Table.check_data_type_list_integrity`** within [`outfancy/table.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/table.py) (lines 699–917). This method samples the first *N* rows—controlled by `self.analyze_threshold` (default 10)—and applies a cascading series of tests to classify each column as `id`, `name`, `date`, `time`, `value`, or `desc`.

The algorithm prioritizes numeric detection first, then time, then date, and finally falls back to text analysis. Each classification relies on specific string methods and helper functions from [`outfancy/widgets.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/widgets.py), creating multiple failure points when data deviates from expected formats.

## Critical Edge Cases in Numeric Detection

### Negative Numbers and Floats

The library uses **`field.isdigit()`** as the sole test for numeric content. This method returns `False` for negative integers (e.g., `-42`), floating-point values (e.g., `3.14`), and formatted numbers containing spaces or commas (e.g., `' 42'` or `'1,000'`).

```python
from outfancy.table import Table

tbl = Table()

# Negative numbers and floats fall through to text branch

data = [(-1, "foo"), (-2.5, "bar")]
print(tbl.check_data_type_list_integrity(data=data))

# Output: ['desc', 'desc']

```

### Unsorted IDs and Single-Row Tables

To distinguish `id` from `value`, Outfancy checks for **continuous numeric sequences** both forward and backward using `int(next_field) - int(field) == 1`. Unsorted identifiers (e.g., `[5, 1, 2, 3]`) fail this consecutive pattern test and are classified as `value`. Additionally, single-row tables trigger an explicit `else: the_type = 'value'` fallback, mislabeling legitimate primary keys.

```python

# Unsorted IDs become 'value' instead of 'id'

data = [(5, "A"), (1, "B"), (2, "C")]
print(tbl.check_data_type_list_integrity(data=data))

# Output: ['value', 'desc']

```

## Date and Time Format Limitations

### Strict Date Patterns

The **`widgets.is_date`** function only attempts four specific `strptime` patterns: `%d-%m-%Y`, `%d-%m-%y`, `%d-%m-%Y %H-%M-%S`, and `%d-%m-%y %H-%M-%S`. ISO dates (`2023-01-15`), US-style dates (`01/15/2023`), or any slash-separated formats are rejected, causing the column to default to `desc`.

### Time Format Restrictions

Time detection delegates to **`widgets.is_complete_hour`**, which strictly accepts `%H:%M:%S` or `%H:%M`. Formats containing AM/PM markers (`12:30 PM`), hour suffixes (`12h30`), or millisecond precision (`12:30:00.000`) are not recognized.

```python

# ISO date and 12-hour time formats fail detection

data = [("2023-01-15", "12:30 PM")]
print(tbl.check_data_type_list_integrity(data=data))

# Output: ['desc', 'desc']

```

## Text Classification Biases

### ASCII-Only Letter Counting

When distinguishing `name` from `desc`, the algorithm counts characters in **`string.ascii_letters`** only. Unicode letters (e.g., accented characters in `'José'` or Cyrillic text) are excluded from the letter count, causing the proportion check to fail. If fewer than 90% of characters are ASCII letters (default threshold `chk_dlti_pecentage_letters_in_field`), the column becomes `desc` regardless of semantic content.

### Short String Override

Strings with length ≤ 15 characters automatically classify as `name`, even when they represent codes or abbreviations (e.g., `'NY'`, `'UK'`) that might semantically function as IDs or values.

```python

# Accented names become 'desc'; short codes forced to 'name'

data = [("José", "NY"), ("María", "CA")]
print(tbl.check_data_type_list_integrity(data=data))

# Output: ['desc', 'name']

```

## Sampling and Input Validation Issues

### Small Dataset Sampling

With fewer than 10 rows, the statistical sample size may be insufficient to establish numeric continuity for ID detection or meaningful proportions for text classification. The algorithm analyzes `min(len(data), self.analyze_threshold)` rows, potentially missing patterns visible only in larger datasets.

### Silent List Truncation

When `data_type_list` is provided manually but mismatches the column count, **`check_data_type_list_integrity`** silently truncates excess entries or pads missing entries with `None`. This hidden mutation can cause downstream rendering errors when users assume their manual type specifications were preserved.

## Summary

- **Numeric limitations**: `isdigit()` rejects negatives, floats, and formatted numbers.
- **ID detection failures**: Unsorted sequences and single-row tables default to `value`.
- **Temporal rigidity**: Only `dd-mm-yyyy` dates and 24-hour times are recognized.
- **Unicode blindness**: Non-ASCII letters trigger false `desc` classifications.
- **Length bias**: Strings ≤15 chars are forced to `name`.
- **Input mutation**: Manual type lists are silently truncated or padded to match column counts.

## Frequently Asked Questions

### Why does Outfancy classify my negative numbers as descriptions?

Outfancy uses Python’s native `isdigit()` string method to detect numeric columns. Because `isdigit()` returns `False` for strings containing minus signs or decimal points (e.g., `'-42'` or `'3.14'`), these values bypass numeric detection and fall through to the text classification branch, where they typically become `desc` type.

### How can I force Outfancy to recognize ISO dates or US date formats?

The `widgets.is_date` helper in [`outfancy/widgets.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/widgets.py) hardcodes only European-style day-month-year patterns. To handle ISO (`yyyy-mm-dd`) or US (`mm/dd/yyyy`) formats, you must either preprocess your data to match `dd-mm-yyyy` format or explicitly pass a `data_type_list` parameter with `'date'` specified for the relevant column index.

### What happens if I provide a data_type_list with the wrong number of entries?

According to the source code in [`outfancy/table.py`](https://github.com/carlosplanchon/outfancy/blob/main/outfancy/table.py), **`check_data_type_list_integrity`** reconciles list lengths silently. If your list is longer than the column count, it truncates the excess entries. If shorter, it pads with `None` values, triggering automatic detection for those columns. The function returns the modified list without raising warnings, which can mask configuration errors.