Export: Polars

Converts the data contract into a Python module that defines a Polars pl.Schema for each schema. Polars does not need to be installed to run the export.

datacontract export polars orders.odcs.yaml --output orders_schema.py

Running this against the example orders contract produces:

import polars as pl

orders = pl.Schema(
    {
        "order_id": pl.String,
        "order_timestamp": pl.Datetime(time_unit="us"),
        "customer_id": pl.String,
        "order_total": pl.Int64,
        "status": pl.String,
    }
)

line_items = pl.Schema(
    {
        "line_item_id": pl.String,
        "order_id": pl.String,
        "sku": pl.String,
        "quantity": pl.Int64,
    }
)

Types

The physicalType decides when it names a width or a precision, then the format of an integer or number, then the logicalType.

Contract	Polars type
`int16`	`Int16`
`int32`	`Int32`
`bigint`, `int64`	`Int64`
`hugeint`, `int128`	`Int128`
`uint8` to `uint128`, and the unsigned MySQL and DuckDB integer types	`UInt8` to `UInt128`
`float16`	`Float16`
`float32`	`Float32`
`double`, `float64`	`Float64`
`decimal(p,s)`, `numeric(p,s)`, `number(p,s)`	`Decimal(precision=p, scale=s)`
`decimal`, `numeric`	`Decimal()`, whose precision Polars chooses and whose scale is 0
`timestamp(n)`, `timestamp_ntz(n)`	`Datetime` in milliseconds for `n` up to 3, microseconds up to 6, nanoseconds above it, and microseconds without `n`
`timestamp_tz`, `timestamp with time zone`, or `logicalType: timestamp` with `timezone: true`	`Datetime` in the `defaultTimezone`, or in UTC
`logicalType: integer` with `format` `i8` to `i128` or `u8` to `u128`	`Int8` to `Int128`, `UInt8` to `UInt128`
`logicalType: number` with `format` `f32` or `f64`	`Float32`, `Float64`
`logicalType: integer`, `number`, `string`, `boolean`, `date`, `time`	`Int64`, `Float64`, `String`, `Boolean`, `Date`, `Time`
`logicalType: array`, `object`	`List`, `Struct`
`logicalType: map`	`List` of `Struct` with `key` and `value`, as Polars reads Arrow maps
`logicalType: vector`	`Array` of the `elementType` with `dimensions` elements; `binary` elements are `Boolean`, `bfloat16` ones `Float32`

Some names have no width of their own, so the logicalType decides: int8 is a 64-bit integer in Postgres and DuckDB and an 8-bit one in ClickHouse, tinyint, smallint, int and integer are 64-bit in BigQuery and NUMBER(38,0) in Snowflake, and float, float4 and real are 64-bit in Snowflake. A decimal with more than 38 digits fails the export, since a Polars decimal cannot hold it. A property with neither a logicalType nor a physicalType is Null.

Polars 1.0 has no Int128, UInt128 or Float16. The module uses them only for a contract that declares those widths.

All options: datacontract export polars.

Types​

Types