Connect your Data

To test a data contract, the CLI connects to the data source defined in the contract’s servers block. The connection details (host, catalog, location, …) live in the contract; credentials are provided as environment variables.

servers:
  production:
    type: postgres   # selects the connection engine
    host: localhost
    port: 5432
    database: postgres
    schema: public

Environment variables are also loaded from a .env file in the current working directory (or the nearest parent directory containing one). Already-set environment variables take precedence over values from .env.

# .env
DATACONTRACT_POSTGRES_USERNAME=postgres
DATACONTRACT_POSTGRES_PASSWORD=postgres

Supported connections

Amazon AthenaAthena over data in S3 Amazon RedshiftRedshift (Postgres wire protocol) Amazon S3CSV, JSON, Delta, Parquet on S3 / S3-compatible storage Apache ImpalaImpala Azure Blob / ADLSFiles on Azure Blob storage or ADLS Gen2 DatabricksUnity Catalog / Hive metastore (warehouse or notebook) Google BigQueryBigQuery tables and views Google Cloud StorageFiles on GCS via S3 interoperability HTTP APIJSON HTTP APIs (GET only) KafkaKafka topics (experimental) Local filesLocal Parquet, JSON, CSV, or Delta files Microsoft SQL ServerSQL Server, Azure SQL, Synapse, Fabric MySQLMySQL / MariaDB OracleOracle Database PostgresPostgres and Postgres-compatible (e.g. RisingWave) SnowflakeSnowflake Spark DataFrameIn-memory Spark DataFrames (programmatic) TrinoTrino (basic, JWT, OAuth2)

:::tip Missing a source? Open an issue on GitHub. :::

Each connection requires the matching optional dependency (extra), or install everything with datacontract-cli[all].

This site is open source. Improve this page.