Pre-persist Save gate

This Save gate is implemented. Use validation_type: "pre_persist" on the Cognite Function.

What this is

validation_type: "pre_persist" is a Cognite Function contract that validates a proposed DMS payload against a published Data Product RuleSet before instances.apply.

It overlays the patch on an in-memory RDF graph, hydrates neighbourhood from DMS, and returns field-mappable violations[] and inferences[]. It never writes DMS.

When to use it

Use this for interactive Save (Flows / Fusion apps) that must fail closed:

TypeScript SHACL Core  →  pre_persist  →  apply(existingVersion) only if conforms

Do not use run_validation(), validation_type=instance, or validation_type=test as that gate. Those paths read persisted DMS or return counters only.

Compared with other entry points

Concern	pre_persist	instance	test	run_validation()	instance_sync_cursor
Lifecycle	Before persist	After-change / workflow	DMS dry-run	Local DMS read	After persist, incremental
Graph	Proposed overlay + hydrate	Payload + auto-load	Fetched from DMS	Fetched from DMS	DMS sync page
Response	violations[], inferences[]	Counters	violations[]	Python object	Counters + Records
Records	Default off	On when configured	Off	Optional	On
Empty input	Fail closed	conforms: true	No-instances message	N/A	Idle / skip

The Data Product picklist (instances.query) is a sibling of this Function. This Function runs the rule. If the dropdown and the RuleSet disagree, the rule wins — do not skip pre_persist because the list was Green.

Minimal payload

Catalog identity is Data Product + RuleSet + view, not a DMS data model. Name the published Data Product and RuleSet, and the view on hydrate_from_dms.views (the same view keys appear on proposed property bags). The function derives the engine namespace from that view.

from cognite_data_quality import call_validation

result = call_validation(
    client,
    validation_type="pre_persist",
    data={
        "job_run_id": "save-trace-001",
        "data_product": {"externalId": "people-product", "version": "1.0.0"},
        "ruleset_references": [{"externalId": "my-ruleset", "version": "1.0.0"}],
        "proposed": [
            {
                "space": "sp_people",
                "externalId": "person-1",
                "properties": {
                    "sp_people": {
                        "Person": {"name": "Ada"}
                    }
                },
            }
        ],
        "hydrate_from_dms": {
            "views": [{"space": "sp_people", "externalId": "Person", "version": "v1"}],
            "filter": {"equals": {"property": ["sp_people", "Person", "orgVersion"], "value": "org-v3"}},
            "limit": 250,
        },
        "auto_load_depth": 2,
        "max_auto_load_instances": 250,
        # post_to_records defaults false — omit records_config
    },
)

if result.get("conforms") is not True:
    # Map result["violations"] onto the form. Do not apply.
    ...

proposed must include at least one property bag per ID. ID-only dumps fail clearly (empty RDF + auto-load skip is a known trap).

auto_load_depth and max_auto_load_instances are required. The engine default of 10 000 auto-load instances is not a Save bound.

Hydrate vs auto-load

These are different mechanisms. Both are in scope.

Mechanism	What it loads	Typical Save use
hydrate_from_dms	Views + filter (preferred) or instance IDs, excluding proposed IDs	Org siblings the edited node does not reference
auto_load_depth	SHACL sh:node / reverse relations after overlay	POB / leave / cert neighbourhood

Prefer filter hydrate. Do not put a ~140-person neighbourhood in Function data.

If membership is MultiReverseDirectRelation, reverse auto-load may miss siblings. Use hydrate_from_dms.filter.

Overlay (partial upsert)

DMS apply is a partial upsert: omitted views/properties stay as they are. Save often sends a patch (Person.name only).

Required merge: proposed values win per (space, externalId, view, property); untouched properties remain from hydrate. Whole-node replace drops sibling facts and breaks SPARQL.

The handler must not call instances.apply.

Blocking policy

• Default blocking = real SHACL Violations (conforms from pyshacl).
• Optional blocking_inferences maps selected SHACL-AF inferences (for example { "resultValue": "Red" } or { "ruleId": "..." }) to conforms: false.
• Green/Amber/Red inferences are not failures unless matched.
• Uniqueness and dqs:edgeExistenceConstraint cannot gate Save (they read live persisted DMS, not the overlay). Dual-role on commit must be sh:sparql on proposed + hydrated siblings.

Fail-closed (client)

Outcome	Function call	Body	Client
Rule/constraint fail	Succeeded	conforms: false, violations[]	Do not apply; map fields
Contract/infra fail	Succeeded	conforms: false, error / error_code	Do not apply
Timeout, 429, missing body	Unknown	None	Do not apply

Do not treat CDF Failed / status: "error" as the Save validation channel — Flows may have no JSON to map.

A 10–15 s abort is a client timeout. The Function is sized for historic/sync runs, not as a Save SLO.

Payload size (Azure 36 kB)

Call data limits (Functions technical): Google 9 MB, AWS 240 kB, Azure 36 kB.

Keep proposed as a patch. Hydrate by filter. If a Save still cannot fit, use an in-process with_shacl library call (bypasses Function data). That fallback is a product choice, not a reason to skip this handler.

Response: no report_text. violations[] / inferences[] may be truncated so Function responses stay within size limits. This version does not return auto_loaded_count.

Out of scope here

OCC / existingVersion, edit locks, Form Constraint Pack, picklist query siblings, Function warm-up on login, Records schema, uniqueness/edge-existence executors, staging-space writes.

Login cold-start reduction is a generic Function ping from the app (no validation_type). It is not a pre_persist flag.

Next

• Invoke Cognite Functions — other validation_type payloads