Skip to main content

Local files

Test local files in Parquet, JSON, CSV, XML, Excel, or Delta format. This is the fastest way to see the CLI in action — no warehouse, no credentials.

1. Install​

uv tool install --python python3.11 --upgrade 'datacontract-cli[duckdb]'

See Installation for pip, pipx, and Docker.

2. Create a contract from a file​

Take any CSV file — or use this one, saved as orders.csv:

order_id,order_timestamp,customer_id,order_total,status
ORD-1001,2024-01-01T10:00:00Z,CUST-1,4999,delivered
ORD-1002,2024-01-02T11:30:00Z,CUST-2,2500,shipped
ORD-1003,2024-01-03T09:15:00Z,CUST-3,1299,pending

Import it. The CLI profiles the data and generates a contract with a schema, value ranges, and a ready-to-test servers block:

datacontract import csv --source orders.csv --output datacontract.yaml

3. Test the data against the contract​

datacontract test datacontract.yaml
Testing datacontract.yaml
Server: production (type=local, format=csv, path=orders.csv)
╭────────┬──────────────────────────────────────────────────────┬─────────────┬─────────╮
│ Result │ Check                                                │ Field       │ Details │
├────────┼──────────────────────────────────────────────────────┼─────────────┼─────────┤
│  ...   │                                                      │             │         │
│ passed │ Check that field order_total has a minimum of 1299.0 │ order_total │         │
│ passed │ Check that field order_total has a maximum of 4999.0 │ order_total │         │
│  ...   │                                                      │             │         │
╰────────┴──────────────────────────────────────────────────────┴─────────────┴─────────╯
🟢 Data contract is valid. Ran 17 checks. Took 1.2 seconds.

4. Let it catch a violation​

Now break the data — append a row with a negative total, a duplicate customer and a repeated status:

echo 'ORD-1004,2024-01-04T12:00:00Z,CUST-1,-100,delivered' >> orders.csv
datacontract test datacontract.yaml
🔴 Data contract is invalid, found the following errors:
1) customer_id Check that unique field customer_id has no duplicate values:
Actual duplicate_count(customer_id) was 1, expected = 0
2) order_total Check that field order_total has a minimum of 1299.0: Actual
invalid_count(order_total) was 1, expected = 0
3) status Check that unique field status has no duplicate values: Actual
duplicate_count(status) was 1, expected = 0

The command exits with code 1, so the same call works as a gate in CI/CD pipelines.

Excel workbooks​

Excel workbooks (format: xlsx) need pip install 'datacontract-cli[xlsx]'. Each schema is one sheet: its physicalName is the sheet's name, matched case-sensitively, and a contract with a single schema may leave it out to read the first sheet. The first row of a sheet holds the column names. Columns before the first and after the last header cell are not read, and a column with an empty header cell between them gets a generated name. Rows that are empty in every column are skipped.

servers:
  - server: local
    type: local
    path: ./delivery.xlsx
    format: xlsx
schema:
  - name: orders
    physicalName: Orders
  - name: country_codes
    physicalName: Country Codes

The path names a single workbook, without a glob. When several schemas share it, each needs its physicalName. Formulas are read as the values Excel last stored for them, and the older .xls format is not supported.

XML files​

XML documents (format: xml) have a guide of their own, from importing their XML Schema to testing nested elements and attributes: XML files.

Reference​

No environment variables are needed for local files. Data type inference and the per-format type handling: Local Files Reference.

CSV files are read as UTF-8 unless the server declares an encoding (ODCS v3.2.0), such as ISO-8859-1 or UTF-16. JSON files are always read as UTF-8.

JSON files (.json, .jsonl, .ndjson) are also validated against a JSON Schema derived from the contract, and every record that breaks it is reported, without stopping the other checks. Without a delimiter, a file holds one record, an array of records, or one record per line.

Troubleshooting​

  • No files found that match the pattern — the path is a glob over file paths, not a directory, and it is resolved relative to the working directory rather than to the contract.
  • No checks run at all — the format in the servers block must be one of csv, json, parquet, xml, xlsx, or delta. It is never guessed at test time (only datacontract import infers it from the file suffix), so a missing or misspelled format leaves the table unreadable.
  • Every schema reads the same files — with more than one schema in the contract, put the {model} placeholder in the path (e.g. ./data/{model}/*.parquet); it is substituted with each schema's name.
  • A CSV or XML value fails as a read error instead of a type check — CSV and XML files are read as the contract's types, so a value that cannot be coerced surfaces while reading. See Data types.

Ready for your real data? Do the same against Snowflake, BigQuery, or Databricks.