Outfancy Data Type Detection Edge Cases: Common Pitfalls and Solutions

Outfancy’s automatic column type detection can misclassify negative numbers, ISO dates, unsorted IDs, and non-ASCII text as generic descriptions due to strict regex patterns and ASCII-only heuristics in the check_data_type_list_integrity method.

Outfancy is a Python library for rendering terminal tables with intelligent column formatting. When processing datasets, the library attempts to automatically determine whether a column contains identifiers, names, dates, times, values, or descriptions. However, the outfancy data type detection algorithm employs specific heuristics that fail under several common edge cases, potentially causing misaligned columns and incorrect rendering priorities.

How Outfancy Detects Data Types

The detection logic resides in Table.check_data_type_list_integrity within outfancy/table.py (lines 699–917). This method samples the first N rows—controlled by self.analyze_threshold (default 10)—and applies a cascading series of tests to classify each column as id, name, date, time, value, or desc.

The algorithm prioritizes numeric detection first, then time, then date, and finally falls back to text analysis. Each classification relies on specific string methods and helper functions from outfancy/widgets.py, creating multiple failure points when data deviates from expected formats.

Critical Edge Cases in Numeric Detection

Negative Numbers and Floats

The library uses field.isdigit() as the sole test for numeric content. This method returns False for negative integers (e.g., -42), floating-point values (e.g., 3.14), and formatted numbers containing spaces or commas (e.g., ' 42' or '1,000').

from outfancy.table import Table

tbl = Table()

# Negative numbers and floats fall through to text branch

data = [(-1, "foo"), (-2.5, "bar")]
print(tbl.check_data_type_list_integrity(data=data))

# Output: ['desc', 'desc']

Unsorted IDs and Single-Row Tables

To distinguish id from value, Outfancy checks for continuous numeric sequences both forward and backward using int(next_field) - int(field) == 1. Unsorted identifiers (e.g., [5, 1, 2, 3]) fail this consecutive pattern test and are classified as value. Additionally, single-row tables trigger an explicit else: the_type = 'value' fallback, mislabeling legitimate primary keys.

# Unsorted IDs become 'value' instead of 'id'

data = [(5, "A"), (1, "B"), (2, "C")]
print(tbl.check_data_type_list_integrity(data=data))

# Output: ['value', 'desc']

Date and Time Format Limitations

Strict Date Patterns

The widgets.is_date function only attempts four specific strptime patterns: %d-%m-%Y, %d-%m-%y, %d-%m-%Y %H-%M-%S, and %d-%m-%y %H-%M-%S. ISO dates (2023-01-15), US-style dates (01/15/2023), or any slash-separated formats are rejected, causing the column to default to desc.

Time Format Restrictions

Time detection delegates to widgets.is_complete_hour, which strictly accepts %H:%M:%S or %H:%M. Formats containing AM/PM markers (12:30 PM), hour suffixes (12h30), or millisecond precision (12:30:00.000) are not recognized.

# ISO date and 12-hour time formats fail detection

data = [("2023-01-15", "12:30 PM")]
print(tbl.check_data_type_list_integrity(data=data))

# Output: ['desc', 'desc']

Text Classification Biases

ASCII-Only Letter Counting

When distinguishing name from desc, the algorithm counts characters in string.ascii_letters only. Unicode letters (e.g., accented characters in 'José' or Cyrillic text) are excluded from the letter count, causing the proportion check to fail. If fewer than 90% of characters are ASCII letters (default threshold chk_dlti_pecentage_letters_in_field), the column becomes desc regardless of semantic content.

Short String Override

Strings with length ≤ 15 characters automatically classify as name, even when they represent codes or abbreviations (e.g., 'NY', 'UK') that might semantically function as IDs or values.

# Accented names become 'desc'; short codes forced to 'name'

data = [("José", "NY"), ("María", "CA")]
print(tbl.check_data_type_list_integrity(data=data))

# Output: ['desc', 'name']

Sampling and Input Validation Issues

Small Dataset Sampling

With fewer than 10 rows, the statistical sample size may be insufficient to establish numeric continuity for ID detection or meaningful proportions for text classification. The algorithm analyzes min(len(data), self.analyze_threshold) rows, potentially missing patterns visible only in larger datasets.

Silent List Truncation

When data_type_list is provided manually but mismatches the column count, check_data_type_list_integrity silently truncates excess entries or pads missing entries with None. This hidden mutation can cause downstream rendering errors when users assume their manual type specifications were preserved.

Summary

• Numeric limitations: isdigit() rejects negatives, floats, and formatted numbers.
• ID detection failures: Unsorted sequences and single-row tables default to value.
• Temporal rigidity: Only dd-mm-yyyy dates and 24-hour times are recognized.
• Unicode blindness: Non-ASCII letters trigger false desc classifications.
• Length bias: Strings ≤15 chars are forced to name.
• Input mutation: Manual type lists are silently truncated or padded to match column counts.

Frequently Asked Questions

Why does Outfancy classify my negative numbers as descriptions?

Outfancy uses Python’s native isdigit() string method to detect numeric columns. Because isdigit() returns False for strings containing minus signs or decimal points (e.g., '-42' or '3.14'), these values bypass numeric detection and fall through to the text classification branch, where they typically become desc type.

How can I force Outfancy to recognize ISO dates or US date formats?

The widgets.is_date helper in outfancy/widgets.py hardcodes only European-style day-month-year patterns. To handle ISO (yyyy-mm-dd) or US (mm/dd/yyyy) formats, you must either preprocess your data to match dd-mm-yyyy format or explicitly pass a data_type_list parameter with 'date' specified for the relevant column index.

What happens if I provide a data_type_list with the wrong number of entries?

According to the source code in outfancy/table.py, check_data_type_list_integrity reconciles list lengths silently. If your list is longer than the column count, it truncates the excess entries. If shorter, it pads with None values, triggering automatic detection for those columns. The function returns the modified list without raising warnings, which can mask configuration errors.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}