Skip to content

Get started with data quality validation

Run your first SHACL validation locally in about five minutes: install the package, load CDF credentials, call run_validation() on a sample, and inspect a few violations.

For how rules are persisted and loaded, see Rule sources.

1. Install

pip install cognite-data-quality

After install, restart your kernel or interpreter so imports pick up the package.

2. Credentials

Create a TOML file with your CDF credentials (for example config.toml):

[cognite]
project = "your-project"
client_id = "your-client-id"
client_secret = "your-client-secret"

Load the client:

import cognite_data_quality

client = cognite_data_quality.load_cognite_client_from_toml("config.toml")
client.config.timeout = 300  # Optional: for large deployments

3. Run run_validation() on a sample

Use run_validation() to validate instances from DMS without deploying Cognite Functions or workflows. This runs entirely in your local Python environment.

import cognite_data_quality

result = cognite_data_quality.run_validation(
    client=client,
    rules_path="views/my_view.yaml",
    rules_format="yaml",
    post_to_records=True,
    records_config=cognite_data_quality.RecordsConfig(
        stream_id="dq_validation_stream",
        rule_set_id="MyViewSHACLv1",
        rule_set_version="1.0",
    ),
    limit=10,
    print_output=True,
)

print("Instance count:", result.instance_count)

4. Inspect a few violations

violations = [
    v for v in result.violations
    if (v.resultSeverity or "").endswith("Violation")
]

print(f"Violations: {len(violations)}")

for v in violations[:5]:
    print(v.resultMessage, "—", v.focusNode)

5. Next steps