# Ossie Dataset Definition Structure: Complete Guide to Schema Modeling > Understand the Ossie dataset definition structure. Learn about schema modeling with name, source, primaryKey, uniqueKeys, description, fields, and customExtensions defined in OsiModel. - Repository: [The Apache Software Foundation/ossie](https://github.com/apache/ossie) - Tags: deep-dive - Published: 2026-07-19 --- **An Ossie dataset definition consists of seven core properties—name, source, primaryKey, uniqueKeys, description, fields, and customExtensions—defined within the `Dataset` inner class of `OsiModel` and serialized to JSON for cross-platform semantic modeling.** An **Ossie dataset** represents the fundamental logical table abstraction inside the Apache Ossie semantic model. As implemented in `apache/ossie`, this structure bridges business concepts with physical implementations across diverse backends like Iceberg, Polaris, and Salesforce. Understanding the exact schema definition is essential for building conversion pipelines and maintaining data integrity across federated systems. ## Core Dataset Properties The dataset definition lives in [`OsiModel.java`](https://github.com/apache/ossie/blob/main/OsiModel.java) (lines 76‑105) and encapsulates metadata required to generate backend-specific schemas. Each property maps directly to a JSON field during serialization. ### Required Identifiers The **`name`** property serves as the unique table identifier. This string value becomes the canonical reference used throughout conversion handlers and must be unique within the model scope. ### Data Integrity Constraints Two properties enforce uniqueness constraints: - **`primaryKey`** – A `List` containing ordered column names that constitute the table's primary key - **`uniqueKeys`** – A `List>` containing zero or more column sets that must remain unique together These constraints propagate to backend-specific DDL generation through the conversion pipeline. ### Metadata and Extensibility Additional descriptive and flexible properties include: - **`source`** – Optional provenance string tracking the originating system or file (e.g., `"customers__dll"`) - **`description`** – Human-readable documentation for the dataset - **`customExtensions`** – A `List` providing key-value pairs for vendor-specific attributes and future-proofing ## Field Definitions Each dataset contains a **`fields`** property—a `List` where individual columns are defined. The `Field` class (lines 107‑124 in [`OsiModel.java`](https://github.com/apache/ossie/blob/main/OsiModel.java)) specifies four critical attributes. ### Column Structure Every field requires: - **`name`** – The column identifier used in queries and mappings - **`description`** – Optional documentation string for business context - **`isTime`** – Boolean flag indicating whether the column holds temporal data, used for time-series optimizations and partitioning hints ### Dialect Expressions The **`expressions`** property contains a `List` objects. Each expression pairs a dialect name (e.g., `"iceberg"`, `"sql"`) with a dialect-specific type definition or expression string. This polymorphic approach allows a single logical field to render as `int` for Iceberg while simultaneously mapping to `VARCHAR(255)` for SQL backends. ## Implementation in Source Code According to the `apache/ossie` repository, the data structures are implemented as static inner classes within `OsiModel`: | Source File | Lines | Content | |-------------|-------|---------| | [`OsiModel.java`](https://github.com/apache/ossie/blob/main/OsiModel.java) | 76‑105 | `Dataset` class definition with all seven properties | | [`OsiModel.java`](https://github.com/apache/ossie/blob/main/OsiModel.java) | 107‑124 | `Field` class with name, description, expressions, and time flag | These classes utilize standard Java bean conventions with getters and setters, enabling Jackson serialization to the JSON format consumed by downstream handlers. ## Practical Example: Creating a Dataset Programmatically The following Java code demonstrates constructing a complete dataset definition with multiple fields and dialect expressions: ```java import org.apache.ossie.converter.polaris.model.OsiModel; import org.apache.ossie.converter.polaris.model.OsiModel.Dataset; import org.apache.ossie.converter.polaris.model.OsiModel.Field; import org.apache.ossie.converter.polaris.model.OsiModel.DialectExpression; import java.util.List; OsiModel model = new OsiModel(); Dataset customers = new Dataset(); customers.setName("Customers"); customers.setSource("customers__dll"); customers.setDescription("Customer master data"); customers.setPrimaryKey(List.of("customer_id")); Field id = new Field(); id.setName("customer_id"); id.setDescription("Unique identifier"); id.setTime(false); id.setExpressions(List.of(new DialectExpression("iceberg", "int"))); Field email = new Field(); email.setName("email"); email.setDescription("Customer email address"); email.setExpressions(List.of( new DialectExpression("iceberg", "string"), new DialectExpression("sql", "VARCHAR(255)") )); email.setTime(false); customers.setFields(List.of(id, email)); model.setDatasets(List.of(customers)); ``` When serialized, this produces the following JSON structure: ```json { "datasets": [ { "name": "Customers", "source": "customers__dll", "primaryKey": ["customer_id"], "description": "Customer master data", "fields": [ { "name": "customer_id", "description": "Unique identifier", "isTime": false, "expressions": [ { "dialect": "iceberg", "expression": "int" } ] }, { "name": "email", "description": "Customer email address", "isTime": false, "expressions": [ { "dialect": "iceberg", "expression": "string" }, { "dialect": "sql", "expression": "VARCHAR(255)" } ] } ] } ] } ``` ## Conversion Pipeline Integration The `Dataset` and `Field` definitions serve as the immutable contract passed between conversion handlers. The Ossie conversion pipeline employs specialized handlers to transform these structures: - **`DatasetMappingHandler`** – Processes dataset-level metadata and constraints - **`FieldMappingHandler`** – Translates field definitions into backend-specific column specs - **`CustomExtensionHandler`** – Routes vendor-specific extensions to appropriate transformation logic These components consume the JSON-serialized dataset definitions to generate Iceberg schemas, SQL DDL, or Salesforce object mappings while preserving the semantic intent captured in the Ossie model. ## Summary - An **Ossie dataset definition** comprises seven properties: name, source, primaryKey, uniqueKeys, description, fields, and customExtensions. - **Fields** contain dialect-agnostic metadata plus a list of `DialectExpression` objects enabling multi-backend deployment. - The implementation resides in [`OsiModel.java`](https://github.com/apache/ossie/blob/main/OsiModel.java) lines 76‑124, using inner classes with Jackson-compatible serialization. - **Primary keys** and **unique keys** are specified as string lists and enforced during conversion. - The structure supports **custom extensions** for vendor-specific attributes without breaking core schema contracts. ## Frequently Asked Questions ### How are primary keys defined in an Ossie dataset? Primary keys are defined using the `primaryKey` property, which accepts a `List` of ordered column names. This list determines the column composition and sequence of the primary key constraint, which conversion handlers translate into backend-specific DDL statements such as Iceberg's `PRIMARY KEY` clause or SQL constraints. ### Can a single Ossie field have multiple type definitions? Yes. The `expressions` field within a `Field` object contains a list of `DialectExpression` objects. Each expression pairs a dialect identifier (like `"iceberg"` or `"sql"`) with a type-specific string, allowing the same logical column to map to `int` for Iceberg tables and `INTEGER` for traditional SQL databases simultaneously. ### What is the purpose of customExtensions in a dataset definition? The `customExtensions` property provides a `List` for storing vendor-specific or application-specific key-value pairs. This extensibility mechanism allows teams to attach proprietary metadata (such as Salesforce API names or internal data lineage IDs) without modifying the core Ossie schema, ensuring backward compatibility. ### Where is the Dataset class located in the Apache Ossie repository? The `Dataset` class is defined as a static inner class within [`OsiModel.java`](https://github.com/apache/ossie/blob/main/OsiModel.java) at lines 76‑105, while the nested `Field` class occupies lines 107‑124. Both classes reside in the `org.apache.ossie.converter.polaris.model` package and serve as the backbone for the Polaris conversion module.