How pgrust Handles JSON and JSONB Data Types: A Complete Rust Implementation

pgrust implements PostgreSQL's JSON and JSONB handling entirely in safe Rust through a three-layer architecture that parses text input, serializes binary JSONB representations, and exposes zero-copy access via PostgreSQL-compatible APIs.

The malisper/pgrust project reimplements PostgreSQL's JSON and JSONB machinery in pure Rust, eliminating C dependencies while maintaining byte-for-byte compatibility with the upstream binary format. This implementation provides a complete pipeline from text parsing to on-disk storage and back again, using zero-copy techniques wherever possible.

Parsing JSON Text Input

Textual JSON enters the system through the json_in function in crates/backend/utils/adt/adt_json/src/lib.rs. This function receives a C-string, validates UTF-8 encoding, and forwards raw bytes to the core parser.

The JSON parser lives in crates/common/jsonapi/src/lib.rs and implements a SAX-style streaming interface. The entry point pg_parse_json drives a lexer (make_json_lex_context_cstring_len) and invokes callbacks on a SaxSink for every token encountered:

pub fn json_in<'mcx>(mcx: Mcx<'mcx>, txt: &[u8], _escontext: Option<&'static str>) -> PgResult<Option<PgVec<'mcx, u8>>> {
    // Creates a Sax sink that validates the input
    jsonapi::pg_parse_json(txt, encoding, false)   // false → no escaping needed
}

For plain json text validation, pgrust uses a null sink that simply verifies the input structure without allocating intermediate representations. This ensures minimal overhead when only validation is required.

Building Binary JSONB Representations

When binary format is required—such as casting json to jsonb or storing values—the system invokes semantic actions that construct the on-disk PostgreSQL format.

The entry point jsonb_in in crates/backend/utils/adt/adt_jsonb/src/lib.rs creates a JsonbInState and hands it to the parser:

let mut state = JsonbInState::new();
jsonapi::pg_parse_json(txt, encoding, true)?;   // true → handle escapes for binary
jsonb_in(mcx, &mut state)                     // drives the seam layer

The seam layer in crates/backend/utils/adt/jsonb_seams/src/lib.rs implements the callbacks expected by the parser. These functions convert parser events into binary tokens:

pub fn jsonb_in_object_start<'mcx>(mcx: Mcx<'mcx>, state: &mut JsonbInState<'mcx>) -> PgResult<()> {
    // Allocate a new JsonbContainer in the arena and push a WJB_BEGIN_OBJECT token
    jsonb_util::pushJsonbValue(mcx, &mut state.stack, JsonbIteratorToken::WJB_BEGIN_OBJECT, None)
}

All jsonb_in_* functions eventually call jsonb_util::pushJsonbValue, which appends JsonbIteratorToken values (WJB_KEY, WJB_VALUE, etc.) to a temporary JsonbParseState. When parsing completes, jsonb_util::JsonbValueToJsonb (located in crates/_support/types/types_jsonb/src/jsonb_util.rs) serializes the accumulated state into the exact on-disk layout required by PostgreSQL, including the header word, JEntry array, and variable-length payload.

Consuming Binary JSONB Values

Reading stored jsonb data uses zero-copy deserialization via JsonbToJsonbValue in crates/_support/types/types_jsonb/src/jsonb_util.rs. This function interprets a varlena byte slice as a JsonbValue tree without copying data:

let off = jsonb_vardata_off(jsonb);
val.typ = jbvBinary;
val.val = JsonbValueData::Binary {
    len: (jsonb.len() - off) as i32,
    data: &jsonb[off..],
    offset: 0,
};

The resulting JsonbValue can be traversed using jsonb_util::JsonbIterator or converted back to Rust types via convenience functions in crates/backend/utils/adt/jsonb_util/src/lib.rs.

High-level operations such as jsonb_typeof, jsonb_array_length, and jsonb_extract_path are thin wrappers around these low-level helpers. For example, jsonb_typeof in crates/backend/utils/adt/adt_jsonb/src/lib.rs reconstructs the value tree and inspects the top-level type:

pub fn jsonb_typeof<'mcx>(mcx: Mcx<'mcx>, jb: &'mcx [u8]) -> PgResult<Option<Datum>> {
    let mut state = JsonbInState::new();
    jsonb_util::JsonbToJsonbValue(jb, &mut state.val)?;
    state.val.typ_of().map(|s| s.into())
}

Practical Code Examples

Parse a JSON String and Convert to JSONB

use pgrust::crates::backend::utils::adt::adt_jsonb::jsonb_in;
use pgrust::crates::common::jsonapi;
use pgrust::crates::utils::mcx::Mcx;

let mcx = Mcx::new();                     // Arena for allocations
let txt = b"{\"name\":\"Alice\",\"age\":30}";

// Step 1: Parse text (validation only)
jsonapi::pg_parse_json(txt, pg_sys::PG_UTF8, false)?; 

// Step 2: Build binary jsonb
let jsonb_bytes = jsonb_in(&mcx, txt, None)?; // Returns Vec<u8> with varlena header
println!("jsonb size: {}", jsonb_bytes.len());

Source: jsonapi::pg_parse_json → crates/common/jsonapi/src/lib.rs:173; jsonb_in → crates/backend/utils/adt/adt_jsonb/src/lib.rs:182

Retrieve the Type of a JSONB Value

use pgrust::crates::backend::utils::adt::adt_jsonb::{jsonb_in, jsonb_typeof};
use pgrust::crates::utils::mcx::Mcx;

let mcx = Mcx::new();
let jb = jsonb_in(&mcx, b"[1,2,3]", None)?;   // Binary jsonb array

let typ = jsonb_typeof(&mcx, &jb)?;          // Returns Datum containing "array"
println!("top-level type = {}", typ.unwrap().to_string());

Source: jsonb_typeof → crates/backend/utils/adt/adt_jsonb/src/lib.rs:137

Iterate Over JSONB Object Fields

use pgrust::crates::backend::utils::adt::adt_jsonb::{jsonb_in, jsonb_iter_fields};
use pgrust::crates::utils::mcx::Mcx;

let mcx = Mcx::new();
let jb = jsonb_in(&mcx, br#"{"a":1,"b":true,"c":"x"}"#, None)?;

let mut iter = jsonb_iter_fields(&mcx, &jb)?;
while let Some((key, value)) = iter.next()? {
    println!("field {} => {:?}", std::str::from_utf8(key)?, value);
}

Source: Iterator helper → crates/_support/types/types_jsonb/src/lib.rs:1351

Summary

• Three-layer architecture: Text parsing (jsonapi), binary serialization (jsonb_seams), and high-level API (adt_jsonb).
• Zero-copy deserialization: JsonbToJsonbValue reads varlena data directly without allocation.
• PostgreSQL compatibility: jsonb_util::JsonbValueToJsonb produces byte-for-byte compatible on-disk format matching PostgreSQL's jsonb.h specification.
• Safe Rust implementation: No extern "C" calls required; uses arena allocation via Mcx for memory management.
• Complete type coverage: Supports all PostgreSQL JSONB operations including jsonb_typeof, jsonb_array_length, and path extraction.

Frequently Asked Questions

What is the difference between JSON and JSONB handling in pgrust?

JSON text is handled by json_in in crates/backend/utils/adt/adt_json/src/lib.rs, which validates the input and returns the original UTF-8 bytes. JSONB binary requires the additional jsonb_in pipeline in crates/backend/utils/adt/adt_jsonb/src/lib.rs, which parses the text and constructs the binary format using the seam layer and jsonb_util serializers.

How does pgrust ensure binary compatibility with PostgreSQL's JSONB format?

pgrust implements the exact binary layout defined in PostgreSQL's jsonb.h header through JsonbValueToJsonb in crates/_support/types/types_jsonb/src/jsonb_util.rs. This function writes the header word, JEntry array, and payload in the same order and endianness as the C implementation, ensuring byte-for-byte compatibility.

Is pgrust's JSON parsing zero-copy?

For reading JSONB, yes. The JsonbToJsonbValue function creates a JsonbValue that references the original byte slice without copying data. For writing JSONB, the system uses arena allocation via Mcx to minimize heap fragmentation, but requires building the binary representation since PostgreSQL's format is a normalized tree structure.

Where are the core JSONB container types defined?

The low-level types (Jsonb, JsonbContainer, JEntry, JsonbIteratorToken) are defined in crates/_support/types/types_jsonb/src/lib.rs, while the high-level manipulation functions reside in crates/backend/utils/adt/jsonb_util/src/lib.rs and crates/backend/utils/adt/adt_jsonb/src/lib.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}