Skip to content

Troubleshooting

View function logs

from cognite.client import CogniteClient

client = CogniteClient()

calls = client.functions.calls.list(
    function_external_id="data-quality-validation",
    limit=10,
)

logs = client.functions.calls.get_logs(call_id=calls[0].id)
print(logs)

Check workflow execution status

executions = client.workflows.executions.list(
    workflow_external_id="dq-shacl-equipment",
    limit=10,
)

for execution in executions:
    print(f"Status: {execution.status}, Started: {execution.started_at}")

Workflow not triggering on data changes

Symptoms: Instance changes in DMS don't start the validation workflow.

Check:

  1. Retrieve the trigger and verify it is active: 
    trigger = client.workflows.triggers.retrieve(external_id="dq-shacl-trigger-<view>")
print(trigger.trigger_type, trigger.is_active)
  2. Verify the instance space in the trigger filter matches the space where data changes occur.
  3. Confirm the workflow version in the trigger matches the deployed workflow version.
  4. If you deployed with --skip-trigger, redeploy without that flag.

Uniqueness workflow not deployed

Symptoms: View has dqs:unique in SHACL but no dq-{view}-uniqueness workflow in CDF.

Check:

  1. Confirm the constraint's sh:targetClass matches the view's data model class (deploy scans SHACL for the view's external_id).
  2. For DataProduct/RuleSet mode, verify RuleSetClient.get_version() returns TTL with the constraint (deploy fetches RuleSet content at deploy time).
  3. Check deploy logs for Skipping uniqueness workflow ... no SHACL uniqueness constraints found or SHACL parse warnings.
  4. Ensure uniqueness_cron is not set to null in view YAML (that disables the scheduled workflow).

Uniqueness not running after sync

Expected behaviour. Sync-cursor (instance_sync_cursor) runs incremental pyshacl only. Global uniqueness runs on the scheduled dq-{view}-uniqueness workflow, not after each sync completion.

To run on demand: call_validate_shacl() with validation_type: shacl or trigger the uniqueness workflow manually.

Uniqueness duplicates still exist but few/no new failure records

Often expected behaviour. Uniqueness now suppresses repeated failure writes for unchanged instances.

Suppression rule:

  • If latest existing failed record for a focusNode has record.lastUpdatedTime >= instance.lastUpdatedTime, runtime suppresses writing a new failure record.

Check:

  1. Inspect uniqueness run output fields:
  2. violatingInstanceCount
  3. recordsPosted
  4. recordsSuppressed
  5. If recordsSuppressed is high and recordsPosted is low, dedupe is likely working as intended.
  6. Update an affected instance and rerun uniqueness — a new failure record should appear if the violation remains.

From validation records:

  1. Check grouped uniqueness records for typed overflow signals:
  2. groupViolationType = "global_overflow" (run-level overflow)
  3. groupViolationType = "value_overflow" (single-value overflow)
  4. If either overflow type is present, this run hit a guardrail and more duplicate groups may still remain.
  5. Fix the reported duplicates and rerun uniqueness until overflow records disappear.

Note:

  • If no new records are written (for example due to dedupe suppression), users may not see a new overflow record for that run.
  • In that case, ask an operator to confirm run-level warnings from workflow/function output.

Grouped uniqueness records are explicitly marked with:

  • validationType = "group_violation"
  • groupViolationType = "global_overflow" | "value_overflow" | "pass"

"Failed to load SHACL rules"

Symptoms: Function logs show an error loading rules.

Check:

  1. Verify the SHACL file exists in CDF Files with the expected external ID.
  2. Confirm the external ID in the view config (shacl_rules.external_id) matches the file.
  3. Verify the file is in the correct CDF dataset (set in settings.yaml under shacl_rules.dataset_external_id).
  4. Validate Turtle syntax locally: 
    python scripts/validate_config.py

"No instances to validate"

Symptoms: Validation completes with zero instances processed.

Check:

  1. Confirm the DMS filter in the view config or time series config is correct.
  2. Verify instances exist in the configured space.
  3. Ensure the view external ID and version match the actual view in CDF.

Zero violations when you expect some

Symptoms: Validation runs but produces no violations for a rule you expect to fire.

Check:

  1. The sh:targetClass in the SHACL rule targets a concrete view type, not a CDF interface.
    Interfaces (e.g. MMSINumber) are not used as rdf:type in the RDF graph — concrete view types (e.g. NavigationAid) are. A shape targeting an interface silently matches nothing.
  2. Run with print_output=True and verbose=True to see per-instance output and zero-instance class warnings.
  3. Check auto_load_depth — if the rule uses properties from referenced instances, depth 0 won't load them.

See SHACL rules and CDF data models for details.

Function dependency errors

"ModuleNotFoundError: No module named 'cognite_data_quality'"

The function was deployed before the package was published to PyPI, or is pinned to an old version.

  1. Publish the latest package: python -m build && twine upload dist/*
  2. Force redeploy: python scripts/deploy_infrastructure.py --env <env> --force

RAW table validation issues

"RAW table not found"

  • Verify the RAW database and table names in the config match what exists in CDF.
  • Confirm the service principal has read access to the RAW table.

"No rows updated since last run"

Normal for incremental validation when no changes occurred. Check RawValidationState in the dataQuality DMS space for last_processed_timestamp to confirm.

Cursor state not saving

  • Ensure the dataQuality space exists in CDF.
  • Verify the service principal has write access to the dataQuality space.
  • Run python scripts/ensure_container.py to create the FunctionValidationState container if missing.

Records API returns no results

Symptoms: Filter queries return empty items.

Check:

  1. Confirm client.config.headers["cdf-version"] = "alpha" is set before the request.
  2. Verify the stream ID in the query matches records.stream_id in settings.yaml.
  3. Check the time range — Records API filters use milliseconds since epoch. Widen lastUpdatedTime to confirm records exist.