# Import System Architecture for Manual Data Imports in Maybe Finance

> Discover the import system architecture for manual data imports in Maybe Finance. Explore its layered pipeline processing CSV files for transactions, trades, and accounts.

- Repository: [Maybe/maybe](https://github.com/maybe-finance/maybe)
- Tags: architecture
- Published: 2026-03-07

---

**The Maybe finance application implements a layered, extensible import pipeline that processes CSV files through controllers, background jobs, domain models, and mapping systems to create transactions, trades, accounts, or Mint exports.**

The open-source Maybe application (maybe-finance/maybe) provides a robust import system architecture for manual data imports that allows users to upload CSV files containing financial data. This architecture handles everything from file parsing and column mapping to background processing and domain object creation, supporting transaction imports, trade imports, account imports, and Mint CSV migrations.

## Core Architecture Layers

The import system architecture for manual data imports follows a deliberate layered design that separates concerns between user interface, background processing, and data transformation.

### Controller and UI Layer

The `ImportsController` in [`app/controllers/imports_controller.rb`](https://github.com/maybe-finance/maybe/blob/main/app/controllers/imports_controller.rb) manages the wizard-style interface for manual data imports. It handles the upload step, column mapping configuration, confirmation screens, and background execution triggers. The controller stitches together the multi-step flow: `new` displays the import type selection, `create` persists a pending import record optionally scoped to an existing account, `show` redirects to the upload page if no CSV exists or to the mapping/confirmation page otherwise, `publish` triggers the background job, `revert` initiates rollback operations, and `apply_template` copies column-mapping configurations from previous successful imports of the same type.

### Background Job Processing

Heavy-lifting import work runs asynchronously via `ImportJob` in [`app/jobs/import_job.rb`](https://github.com/maybe-finance/maybe/blob/main/app/jobs/import_job.rb), which is queued as `high_priority`. The job simply delegates to `import.publish`, which executes the entire import flow within a database transaction. This separation ensures the UI remains responsive while processing large CSV files and complex mapping operations.

### Domain Model Layer

The `Import` class in [`app/models/import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import.rb) serves as the abstract base orchestrator for all manual data imports. Concrete implementations include `TransactionImport` in [`app/models/transaction_import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/transaction_import.rb), `TradeImport` in [`app/models/trade_import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/trade_import.rb), `AccountImport` in [`app/models/account_import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/account_import.rb), and `MintImport` in [`app/models/mint_import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/mint_import.rb), each defining specific behavior for their domain. This layer coordinates CSV parsing, row generation, mapping synchronization, and the final publish operation that creates domain objects.

### Row and Mapping Systems

The architecture includes specialized subsystems for data validation and transformation. `Import::Row` in [`app/models/import/row.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/row.rb) represents individual CSV lines, handling validation, number sanitization, and signed amount calculation. The mapping system, centered on `Import::Mapping` in [`app/models/import/mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/mapping.rb), bridges raw CSV values to existing domain objects through specialized subclasses: `Import::CategoryMapping` in [`app/models/import/category_mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/category_mapping.rb), `Import::TagMapping` in [`app/models/import/tag_mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/tag_mapping.rb), `Import::AccountMapping` in [`app/models/import/account_mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/account_mapping.rb), and `Import::AccountTypeMapping` in [`app/models/import/account_type_mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/account_type_mapping.rb).

## The Import Orchestration Model

The `Import` base class in [`app/models/import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import.rb) provides the core machinery that all manual data import types leverage.

### CSV Parsing and Row Generation

The `parse_csv_str` method uses Ruby's standard `CSV` library with configurable column separators (`col_sep`) to handle different regional formats. The `generate_rows_from_csv` method, overridden by each subclass, constructs a hash for every CSV line and bulk-inserts records via `Import::Row.insert_all!`. This bulk insertion approach minimizes database round-trips when processing thousands of rows.

### Mapping Synchronization

The `sync_mappings` method iterates over `mapping_steps` defined by each subclass to build or update `Import::Mapping` records. This process ensures that each raw value from the CSV (such as "Groceries" or "Checking Account") maps to a concrete domain object like `Category` or `Account`. The mapping system supports both matching existing records and creating new ones on-the-fly.

### Publish and Revert Flows

The `publish` method validates row counts, executes the subclass-specific `import!` method inside a database transaction, and updates the import status to `complete` or `failed`. Upon successful completion, it triggers a family-wide sync to update balances and calculations. The `revert` method provides transactional safety by deleting any accounts or entries created during the import and resetting the import status to `pending`, allowing users to correct errors and retry.

## Concrete Import Types

The import system architecture for manual data imports supports four distinct import types, each tailored to specific financial data domains.

### Transaction Imports

`TransactionImport` in [`app/models/transaction_import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/transaction_import.rb) creates `Transaction` and `Entry` records from CSV files containing personal finance transactions. It requires `date` and `amount` columns, with optional support for `name`, `currency`, `category`, `tags`, `account`, and `notes`. The mapping steps include `CategoryMapping`, `TagMapping`, and optional `AccountMapping`. This import type supports both signed amounts and custom column-based signage detection.

### Trade Imports

`TradeImport` in [`app/models/trade_import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/trade_import.rb) handles investment activity by creating `Trade` and `Entry` records. It requires `date`, `ticker`, `qty`, and `price` columns, with optional `account` mapping. This import type is essential for users tracking brokerage transactions and investment portfolios, allowing bulk entry of buy and sell orders with proper cost basis calculations.

### Account Imports

`AccountImport` in [`app/models/account_import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/account_import.rb) creates new `Account` records directly from CSV data, utilizing `OpeningBalanceManager` to establish initial balances. It requires `name` and `amount` columns, with `AccountTypeMapping` to determine the accountable class (such as `Depository` or `CreditCard`). This import type is useful for migrating account lists from other financial systems or setting up initial account structures.

### Mint Imports

`MintImport` in [`app/models/mint_import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/mint_import.rb) provides specialized handling for CSV exports from Mint.com, creating `Transaction` records while preserving Mint-specific categorization and tagging. It follows similar patterns to `TransactionImport` but includes defaults optimized for Mint's export format, such as specific signage conventions and column expectations.

## Row Processing and Validation

The `Import::Row` class in [`app/models/import/row.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/row.rb) serves as the data validation and transformation layer for individual CSV lines.

### Date Parsing and Amount Calculation

Each row stores raw CSV values and provides computed properties for downstream processing. The `date_iso` method parses the family-specific `date_format` into ISO-8601 format, handling regional date variations. The `signed_amount` method applies the import's **signage convention**—either `inflows_positive` or `inflows_negative`—and respects the **amount-type strategy**, which can be `signed_amount` or `custom_column` for complex CSV structures.

The row model also handles tag parsing through `tags_list`, splitting pipe-delimited strings (such as "groceries|essentials") into arrays for association with transaction records.

## The Mapping System

The mapping system bridges the gap between raw CSV strings and typed domain objects, centered on the `Import::Mapping` base class in [`app/models/import/mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/mapping.rb).

### Category, Tag, and Account Mappings

Specialized mapping subclasses handle different domain associations:

- **CategoryMapping** ([`app/models/import/category_mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/category_mapping.rb)): Matches CSV category strings to existing `Category` records, creates a new category when the user selects "Add as new category".

- **TagMapping** ([`app/models/import/tag_mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/tag_mapping.rb)): Similar to category mapping but does **not** require explicit selection because tags can be freely created.

- **AccountMapping** ([`app/models/import/account_mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/account_mapping.rb)): Links CSV account strings to existing `Account` objects; if the import isn't scoped to a specific family account, users can create a new account on-the-fly.

- **AccountTypeMapping** ([`app/models/import/account_type_mapping.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import/account_type_mapping.rb)): Used only by `AccountImport` to map the "entity_type" column to an accountable class (e.g., `Depository`, `CreditCard`).

Each mapping class implements `mappables_by_key(import)` for fast lookup, `selectable_values` for UI dropdowns, `requires_selection?` to determine mandatory selection, and `create_mappable!` to instantiate new domain objects.

## Practical Implementation Examples

### Creating a Manual Transaction Import

To create a manual transaction import programmatically:

```ruby
family = Family.find_by(name: "Acme Corp")
import = family.imports.create!(
  type: "TransactionImport",
  account: nil,                     # nil => user will map accounts from CSV

  date_format: "%m/%d/%Y"
)

# Upload CSV content (as a string)

csv = <<~CSV
  date,amount,name,currency,category,tags,account,notes
  05/15/2024,-45.99,Grocery Store,USD,Food,groceries|essentials,Checking Account,Weekly groceries
CSV

import.update!(raw_file_str: csv)               # raw_file_str stores the raw CSV

import.generate_rows_from_csv                  # parses CSV → Import::Row records

import.sync_mappings                           # creates/updates Import::Mapping rows

# Preview mapping UI (in Rails console we can inspect)

import.mappings.accounts.pluck(:key, :mappable_id) # => [["Checking Account", 12]]

import.mappings.categories.pluck(:key, :mappable_id) # => [["Food", 3]]

# Once mappings are set (or left as "add new"), publish the import

import.publish_later   # enqueues ImportJob

```

### Customizing Signage and Amount-Type Strategy

Configure how the system interprets amount signs:

```ruby
import = family.imports.find(import_id)

# For inflows to be positive (default for Mint imports)

import.update!(signage_convention: "inflows_positive", amount_type_strategy: "signed_amount")

# Or use a custom column to decide inflow/outflow (e.g., a column called "Transaction Type")

import.update!(
  amount_type_strategy: "custom_column",
  amount_type_inflow_value: "credit"   # rows where entity_type == "credit" become positive

)

```

### Adding New Accounts via Mapping

When the import UI shows the *Account* dropdown, selecting **"Add as new account"** creates a fresh `Account` record during `import!`:

```ruby

# The mapping row for a new account will have key = "New Savings"

# and CREATE_NEW_KEY = "internal_new_resource"

# During import!:

Import::AccountMapping#create_mappable!  # creates the Account with default balance 0

```

## Summary

The import system architecture for manual data imports in Maybe provides a comprehensive, modular pipeline for CSV data ingestion:

- **Layered architecture** separates UI concerns (`ImportsController`), background processing (`ImportJob`), and domain logic (`Import` base class and subclasses).
- **Four concrete import types** handle specific domains: `TransactionImport`, `TradeImport`, `AccountImport`, and `MintImport`.
- **Row-level processing** via `Import::Row` validates data, parses dates according to family-specific formats, and calculates signed amounts based on configurable signage conventions.
- **Dynamic mapping system** bridges CSV strings to domain objects through `CategoryMapping`, `TagMapping`, `AccountMapping`, and `AccountTypeMapping`, supporting both existing record matching and on-the-fly creation.
- **Transactional safety** ensures imports either complete successfully or can be fully reverted via the `revert` method, which deletes created records and resets import status.

## Frequently Asked Questions

### How does the Maybe import system handle different CSV formats and date conventions?

The import system architecture for manual data imports accommodates regional variations through configurable parsing options. The `Import` model stores `date_format` strings (such as `%m/%d/%Y`) and `col_sep` values for different column separators. During processing, `Import::Row#date_iso` parses dates according to the family-specific format, while `Import.parse_csv_str` uses Ruby's standard CSV library with the configured separator to handle semicolon or tab-delimited files.

### What happens if a CSV contains categories or accounts that don't exist in the Maybe database?

The mapping system automatically handles unknown values through the `Import::Mapping` subclasses. When `sync_mappings` runs, it identifies unique keys from the CSV and creates mapping records. For categories and accounts, the UI presents dropdowns with existing values plus an "Add as new" option. If selected, `CategoryMapping#create_mappable!` or `AccountMapping#create_mappable!` instantiates new domain objects during the `import!` phase. Tags work similarly but don't require explicit selection, allowing free-form creation via `TagMapping`.

### Can the import system handle both positive and negative amount conventions?

Yes, the architecture supports configurable signage conventions through the `signage_convention` attribute, which accepts `inflows_positive` or `inflows_negative` values. Additionally, the `amount_type_strategy` setting allows either `signed_amount` (using the numeric sign directly) or `custom_column` (where a separate column like "Transaction Type" determines signage). The `Import::Row#signed_amount` method applies these rules during processing, ensuring that inflows and outflows are recorded correctly regardless of the source CSV's conventions.

### How does Maybe ensure data integrity during large CSV imports?

Data integrity is maintained through a combination of transactional boundaries and reversible operations. The `publish` method in [`app/models/import.rb`](https://github.com/maybe-finance/maybe/blob/main/app/models/import.rb) wraps the subclass-specific `import!` logic in a database transaction, ensuring that either all rows import successfully or none do. For error recovery, the `revert` method deletes any accounts or entries created during the import and resets the import status to `pending`. Additionally, `Import::Row` validates required columns (such as `date` and `amount` for transactions) before the publish phase, preventing malformed data from reaching the domain models.