Amazon S3
:::info[Required extra]
This connection requires the s3 and duckdb extras. See Installation.
:::
Test data stored in S3 buckets or any S3-compatible endpoint, in CSV, JSON, Delta, or Parquet format.
Server (JSON)
servers:
- server: production
type: s3
endpointUrl: https://minio.example.com # not needed with AWS S3
location: s3://bucket-name/path/*/*.json
format: json
delimiter: new_line # new_line, array, or none
Server (Delta)
servers:
- server: production
type: s3
endpointUrl: https://minio.example.com # not needed with AWS S3
location: s3://bucket-name/path/table.delta # folder with parquet files and _delta_log
format: delta
Environment variables
| Variable | Example | Description |
|---|---|---|
DATACONTRACT_S3_REGION |
eu-central-1 |
Region of the S3 bucket |
DATACONTRACT_S3_ACCESS_KEY_ID |
AKIAXV5Q5Q... |
AWS Access Key ID |
DATACONTRACT_S3_SECRET_ACCESS_KEY |
93S7LRrJ... |
AWS Secret Access Key |
DATACONTRACT_S3_SESSION_TOKEN |
AQoDYXdzEJr... |
AWS temporary session token (optional) |
