# How TeslaMate's Import Module Migrates Data from TeslaFi

> Learn how TeslaMate's import module migrates data from TeslaFi by converting CSV files into vehicle state events, recreating historical records without live API access.

- Repository: [TeslaMate/teslamate](https://github.com/teslamate-org/teslamate)
- Tags: migration-guide
- Published: 2026-06-16

---

**TeslaMate's TeslaFi import module converts exported CSV files into vehicle-state events through a fake API pipeline, parsing and normalizing historical data in [`lib/teslamate/import.ex`](https://github.com/teslamate-org/teslamate/blob/main/lib/teslamate/import.ex) and [`lib/teslamate/import/line_parser.ex`](https://github.com/teslamate-org/teslamate/blob/main/lib/teslamate/import/line_parser.ex) to recreate vehicle records without requiring live Tesla API access.**

The `teslamate-org/teslamate` repository provides a dedicated import pipeline that allows users to migrate historical driving and charging data from TeslaFi into TeslaMate's database. This module treats CSV exports as event streams, transforming them into native `TeslaApi.Vehicle` structs that the application processes as if they were real-time API responses.

## Overview of the TeslaFi Import Architecture

The migration workflow relies on two core components working in concert. `TeslaMate.Import` orchestrates the state machine, directory scanning, and process supervision, while `TeslaMate.Import.LineParser` handles the granular transformation of CSV rows into structured vehicle data. Together, they create a **fake API process** that feeds historical data into TeslaMate's standard vehicle processing pipeline.

## Step 1: CSV Discovery and File Validation

When `TeslaMate.Import.run/1` initiates the import, the state machine transitions from `:idle` to process the `:read_directory` event (lines 100‑112 of [`lib/teslamate/import.ex`](https://github.com/teslamate-org/teslamate/blob/main/lib/teslamate/import.ex)). The module scans the directory specified by the `IMPORT_DIR` environment variable (defaulting to `./import`) and identifies valid TeslaFi export files:

```elixir
{:ok, files} = File.ls(path)
files
|> Enum.map(fn n -> %{date: parse_fname(n), path: Path.join([path, n])} end)
|> Enum.reject(fn %{date: date} -> is_nil(date) end)
|> Enum.sort_by(fn %{date: date} -> date end)

```

The `parse_fname/1` function (lines 93‑104) recognizes two specific naming patterns:
- `MMYYYY.csv` (legacy format)
- `TeslaFiMMYYYY.csv` (current TeslaFi export format)

Files that do not match these patterns are silently ignored, ensuring only relevant CSV data enters the pipeline.

## Step 2: Concurrent CSV Streaming and Parsing

Upon receiving the `:import` event (lines 120‑131), the module calls `create_event_streams/2` to process each file. The implementation uses Elixir's `Task.async_stream/4` to parse rows concurrently while maintaining order:

```elixir
rows
|> Task.async_stream(&LineParser.parse(&1, tz), timeout: :infinity, ordered: true)
|> Stream.map(fn {:ok, vehicle} -> vehicle end)

```

Each CSV is streamed via `File.stream!/2` and parsed with `CSV.parse/1`, then handed to `LineParser.parse/2` (lines 238‑245) for transformation into vehicle structs.

## Step 3: Data Normalization and Vehicle Struct Creation

The `LineParser.parse/2` function (lines 18‑22) reduces each CSV row into a vehicle map by folding column values into a `@default_vehicle` base struct. The `into_vehicle/3` function (lines 74‑133) handles specific data types:

- **Timestamp conversion**: Parses the `Date` column using Timex (lines 84‑88) and converts it to Unix milliseconds in the user-selected timezone, injecting this timestamp into all nested state maps (`charge_state`, `climate_state`, `drive_state`, `vehicle_config`, `vehicle_state`) at lines 108‑110.
- **Boolean and numeric coercion**: Helpers like `map_value/2` and `to_float/1` convert string representations of `"TRUE"`, `"FALSE"`, and numeric values into proper Elixir types.
- **Vehicle ID fallback**: When the CSV contains an empty `vehicle_id` field, the parser injects the value from the `TESLAFI_IMPORT_VEHICLE_ID` environment variable (lines 79‑81).
- **Validation**: If a row contains a VIN different from the initially detected vehicle, the parser raises a `:vehicle_changed` error (lines 51‑57) to prevent cross-contamination of data.

The assembled map is finally converted into a `%TeslaApi.Vehicle{}` struct via `Vehicle.result/1`.

## Step 4: Fake API Simulation and Database Persistence

Before streaming begins, `create_car/1` (lines 77‑99) iterates through the event streams to locate the first valid vehicle entry containing a `vin`, `vehicle_id`, and internal `id`. It then persists the car using `Vehicles.create_or_update!/1` and attaches a minimal `CarSettings` struct (lines 91‑96) that disables streaming while keeping the car enabled.

The system then starts a **fake API process** (`TeslaMate.Import.FakeApi`, lines 40‑46) seeded with the prepared event streams. This process mimics the real Tesla API but yields imported historical data instead of live responses. A vehicle process (`TeslaApi.Vehicle`) is started with `import?: true` (lines 48‑54) and linked to the fake API, allowing the rest of TeslaMate to consume the historical data transparently.

## Step 5: Import Completion and Cleanup

As each stream finishes processing, it sends a `{:done, date}` message to the state machine (lines 63‑68). When all streams report completion, the `:done` handler (lines 70‑80) executes:
- Terminates the vehicle process
- Normalizes the car's final state via `Log.complete_current_state/1` and `Log.create_current_state/1`
- Broadcasts a `:complete` status via Phoenix PubSub to update the UI (`lib/teslamate_web/live/import_live/index.html.heex`)

If any stream encounters an error—such as detecting data for a different vehicle—the import aborts immediately with `{:error, reason}`, preventing partial or corrupted migrations.

## Practical Usage Examples

### Starting an Import via Console

Ensure CSV files are placed in the import directory, then execute:

```elixir

# Set the import directory (optional)

# Files should be named like: ./import/TeslaFi062019.csv

{:ok, _pid} = TeslaMate.Import.start_link(name: :teslamate_import, directory: "./import")
:ok = TeslaMate.Import.run("Europe/Berlin")

```

### Minimal TeslaFi CSV Format

```csv
Date,State,charge_state,drive_state,climate_state,vehicle_id,vin,latitude,longitude
2020-06-01 12:34:56,online,85,37.7749,-122.4194,38.5,1,5YJ3E1EA5KF....

```

### Handling Environment Variables

```bash
export TESLAFI_IMPORT_VEHICLE_ID=42   # Default vehicle ID for empty CSV fields

export IMPORT_DIR=/path/to/csv/files
mix run -e "TeslaMate.Import.run(\"UTC\")"

```

## Summary

- **TeslaMate's import module** migrates TeslaFi data by treating CSV exports as event streams consumed through a fake API interface.
- **File discovery** relies on specific naming patterns (`TeslaFiMMYYYY.csv`) scanned from `IMPORT_DIR`.
- **Concurrent parsing** uses `Task.async_stream/4` and `LineParser.parse/2` to transform rows into `%TeslaApi.Vehicle{}` structs with timezone-aware timestamps.
- **Data integrity** is enforced through VIN validation and the `TESLAFI_IMPORT_VEHICLE_ID` fallback for missing identifiers.
- **Database integration** occurs via `Vehicles.create_or_update!/1`, creating a car record that the standard TeslaMate pipeline populates with historical states.

## Frequently Asked Questions

### What file naming conventions does the TeslaFi import module require?

The import module recognizes two patterns in `parse_fname/1`: `MMYYYY.csv` for legacy exports and `TeslaFiMMYYYY.csv` for current TeslaFi formats. Files must use these exact patterns to be processed; all other filenames are ignored during the directory scan.

### How does the module handle different timezones in historical data?

The `LineParser.parse/2` function accepts a timezone parameter (e.g., `"Europe/Berlin"`) and uses Timex to convert the CSV's `Date` column into Unix millisecond timestamps normalized to that timezone. This timestamp is then injected into all nested vehicle state maps to ensure consistent temporal alignment.

### What happens if the CSV files contain data from multiple vehicles?

If `LineParser` detects a VIN mismatch between rows (lines 51‑57), it raises a `:vehicle_changed` error that aborts the entire import process. This prevents accidental merging of data from different vehicles into a single TeslaMate car record.

### Can I import TeslaFi data without using the web interface?

Yes. The import can be initiated programmatically by calling `TeslaMate.Import.run/1` with a timezone string from any Elixir context, such as a Mix task or IEx session, provided the `IMPORT_DIR` contains the properly named CSV files and the environment variables are configured.