Skip to content

Time series datapoint rule examples

What this is

This guide documents practical datapoint checks for YourOrgTimeSeries, including scheduling guidance and how to run validation before enabling production workflows.

When to use it

Use this guide when authoring and validating datapoint-oriented SHACL checks for time series quality.

Use it with Deploy when moving from ad-hoc runs to scheduled production workflows.

User mental model

  • Input: SHACL rules + selected time windows
  • Execution: time-window datapoint functions (cdf_sdk, cdf_indsl) evaluate each series
  • Output: DataQualityValidationRecord pass/fail entries

Runtime behavior

Scheduling requirement

Time series datapoint validation should run on a schedule. The deployed workflow executes the time series handler on a cron cadence and evaluates SHACL window expressions (for example "1h-ago" to "now").

Recommended pattern:

  • Run on an hourly cron (for example 2 * * * *).
  • Use time-window expressions in SHACL that match your cadence.
  • Keep cadence and window aligned so records are predictable over time.

For deployment mechanics, see Deploy (time series workflows are deployed when timeseries_dir is provided).

Included example rule types

The examples in this guide cover:

  • minimum datapoint count (explicit pass + fail)
  • stale latest datapoint (modeled as no datapoints in the active window)
  • value range bounds (datapoints_min / datapoints_max)
  • flatline detection (max - min span threshold)
  • cross-signal consistency (average delta between two series)

Minimal happy path

1) Run validation and post to Records

Deploy validation infrastructure first (including the time series handler). call_validate_timeseries_shacl() invokes the deployed Cognite Function via client.functions.call — installing the PyPI package alone is not enough.

Provide ruleset_references, data model coordinates, explicit instance_ids (or a filter with space / external_id_prefix), and records_config with a unique job_run_id:

from cognite_data_quality import load_cognite_client_from_toml, call_validate_timeseries_shacl

client = load_cognite_client_from_toml("config.toml")

result = call_validate_timeseries_shacl(
    client,
    data={
        "ruleset_references": [
            {"externalId": "YourOrgTimeSeriesExampleRules", "version": "1.0.0"}
        ],
        "datamodel_space": "my_space",
        "datamodel_external_id": "YourOrgTimeSeries",
        "datamodel_version": "v1",
        "instance_ids": [
            {"space": "springfield_instances", "external_id": "pi:160554"},
            {"space": "springfield_instances", "external_id": "pi:160555"},
        ],
        "job_run_id": "job_manual_ts_examples_001",
        "records_config": {
            "stream_id": "dq_validation_stream",
            "rule_set_id": "YourOrgTimeSeriesExampleRules",
            "rule_set_version": "1.0.0",
        },
    },
)
print(result.get("timeseries_count"), result.get("records_posted"))

Alternatively, select series with filter using space and/or external_id_prefix instead of instance_ids.

Use inline shacl_rules instead of ruleset_references while iterating on TTL. Production deploy binds rules through the DataProduct RuleSet API.

2) Verify what was written to Records

Query DataQualityValidationRecord in your records space and filter by ruleSetId and jobRunId from the run above. Confirm pass/fail counts match the series and windows you expect.

See Records output for field names and Invoke for additional payload options.

Real examples

Example snippets

A) Minimum datapoint count (pass/fail pattern)

PREFIX cdf_sdk: <https://cognite.com/cdf/sdk/>

SELECT $this ?count
WHERE {
  BIND(cdf_sdk:datapoints_count($this, "24h-ago", "now") AS ?count)
  FILTER(?count < 1)
}

B) Stale latest datapoint

PREFIX cdf_sdk: <https://cognite.com/cdf/sdk/>

SELECT $this ?count
WHERE {
  BIND(cdf_sdk:datapoints_count($this, "24h-ago", "now") AS ?count)
  FILTER(?count = 0)
}

C) Value range bounds

PREFIX cdf_sdk: <https://cognite.com/cdf/sdk/>

SELECT $this ?minVal ?maxVal
WHERE {
  BIND(cdf_sdk:datapoints_min($this, "24h-ago", "now") AS ?minVal)
  BIND(cdf_sdk:datapoints_max($this, "24h-ago", "now") AS ?maxVal)
  FILTER(?minVal < -100 OR ?maxVal > 1000)
}

D) Flatline detection

PREFIX cdf_sdk: <https://cognite.com/cdf/sdk/>

SELECT $this ?span ?count
WHERE {
  BIND(cdf_sdk:datapoints_min($this, "24h-ago", "now") AS ?minVal)
  BIND(cdf_sdk:datapoints_max($this, "24h-ago", "now") AS ?maxVal)
  BIND(cdf_sdk:datapoints_count($this, "24h-ago", "now") AS ?count)
  BIND((?maxVal - ?minVal) AS ?span)
  FILTER(?count >= 2 && ?span <= 0.001)
}

E) Cross-signal consistency

PREFIX cdf_sdk: <https://cognite.com/cdf/sdk/>

SELECT $this ?avgA ?avgB ?delta
WHERE {
  BIND(cdf_sdk:datapoints_average($this, "24h-ago", "now") AS ?avgA)
  BIND(cdf_sdk:datapoints_average(<http://purl.org/cognite/springfield_instances/pi:160554>, "24h-ago", "now") AS ?avgB)
  BIND(ABS(?avgA - ?avgB) AS ?delta)
  FILTER(?delta > 1.0)
}

Best practices

  • Keep strict operational rules as sh:Violation, and advisory checks as sh:Warning.
  • Use a longer window (for example 24h) when data density is low.
  • For flatline/range/cross checks, only apply to numeric series.
  • Always validate new rules with one known pass and one known fail series before scheduling broadly.

Troubleshooting

  • No records in output: confirm schedule is enabled and rule set is bound to the correct data model/series.
  • Unexpected stale findings: verify cadence and window alignment (cron vs "xh-ago" -> "now").
  • Noisy flatline/range alerts: tune thresholds and window size before broad rollout.

Previous section

Next section