Skip to content

Config (TOML and view config)

TOML credentials

Create a TOML file in your working directory (e.g. config.toml or config.toml):

[cognite]
project = "<cdf-project>"
client_id = "<client-id>"
client_secret = "<client-secret>"

Optional fields (depending on your login flow):

[cognite]
tenant_id = "<tenant-id>"
cdf_cluster = "<cdf-cluster>"   # e.g. "api", "westeurope-1"
login_flow = "client_credentials"

Load the client:

from cognite_data_quality import load_cognite_client_from_toml

client = load_cognite_client_from_toml("config.toml")

One TOML file per environment or project.

View config (YAML)

View configs live next to your SHACL rules (e.g. under config/environments/<env>/views/). Each YAML file describes one view: data model, SHACL file, partition settings, and optional records config. Example structure:

  • settings.yaml – environment settings (records stream, workflow prefix, etc.)
  • views/my_view.yaml – view-specific config (datamodel, SHACL rules, partition field)

When you call deploy_validation_infrastructure(settings_path=..., views_dir=...), the package reads all view YAMLs from views_dir and deploys workflows and triggers accordingly. When you call run_validation(rules_path="path/to/view.yaml", rules_format="yaml"), datamodel and instance space are taken from that YAML.

Standard view config

view:
  space: "my_space"
  external_id: "MyView"
  version: "v1"

instance_space: "my_instances"

shacl_rules:
  file: "my_view_shacl.ttl"
  external_id: "my_view_shacl"

datamodel:
  space: "my_space"
  external_id: "MyDataModel"
  version: "v1"

validation:
  auto_load_depth: 2
  verbose: true

partition_count: 10
partition_field: lastUpdatedTime
chunk_size: 200

records:
  rule_set_id: "MyViewSHACLv1"
  rule_set_version: "1.0"
  # Optional: write records into the validated instance's space instead of the default records space
  use_instance_space: false
  # Optional: override the space used to write records (overrides settings.records.space)
  records_space: null

Sync cursor mode

For views with frequent updates, enable cursor-based incremental validation. The workflow trigger fires as a lightweight signal (no instance data in payload); the function fetches and validates all changes since the last saved cursor.

# The view, instance_space, shacl_rules, datamodel, validation, and records
# sections are the same as in the standard config above. partition_count and
# partition_field are not used in sync cursor mode. Add the following fields:

chunk_size: 1000
sync_batch_size: 1000   # Trigger batch size (signal only, no instance payload)
use_sync_cursor_mode: true
max_concurrent_executions: 1  # Required — prevents cursor race conditions

initial_sync_cutoff is not set in the YAML; the orchestrator injects it automatically when creating the sync trigger to align with the historic validation cutoff date.

See Rule sources and SHACL and CDF data models for how rules and views interact.