Skip to content

Editing Data Products and RuleSets in Flows apps

Guide for app developers who build editing UX that reads and changes CDF Data Products and RuleSets — for example Flows dashboards and industrial workspaces that manage quality rules for one or more views.

This page is the product contract for in-app edit and Publish. For how the validation package resolves RuleSets at deploy/runtime, see Rule sources. For chat-time drafting without Publish, see Atlas rule-authoring agent.

Audience and goal

You are implementing screens that:

  • Load a Data Product and its bound RuleSets
  • Let users edit SHACL Turtle (manually or via an agent)
  • Test rules against sample instances
  • Publish immutable versions back to CDF

Goal: keep multi-view products intact, avoid silent data loss on Publish, and never confuse “no recent validation records” with “rules or instances are gone”.

Three layers (do not conflate)

Layer What it is What the UI should show
Instances and views Operational DMS data Counts, deep links, schema
Data Product and RuleSets Published rules via quality.rules[] Rule Management, Publish, which ruleset each view uses
Validation Records Outputs from validation jobs in a Records stream Compliance scores, health cards

Empty compliance scores mean “no records in the selected lookback window”, not that the view, instances, or RuleSet are missing.

Happy path

  1. Resolve the latest published Data Product version (highest semantic version among published items — not raw API list order).
  2. Read quality.rules[]: each entry is { externalId, version } for a RuleSet.
  3. Load that RuleSet version’s Turtle into the editor as the published baseline.
  4. User edits a draft. Agent tools such as submit_rule update the draft only.
  5. Test runs validation against chosen instances. Test must not create a RuleSet or Data Product version.
  6. Publish:
  7. POST a new RuleSet version with the draft Turtle
  8. Build the next quality.rules list by bumping only the edited ruleset’s version and keeping every sibling
  9. POST a new Data Product version with that full list (and the same views / metadata as before)
  10. Reload the RuleSet version that quality.rules now references.

Always POST new versions. Do not PATCH an existing RuleSet or Data Product version in place. Do not treat CDF Files as the edit source of truth. Prefer the button label Publish over “Save”.

Multi-view products and sibling RuleSets

A single Data Product often binds several RuleSets — typically one per DMS view. That list lives on the Data Product as quality.rules[].

When the user Publishes edits for one ruleset:

  • Update that ruleset’s version in the list
  • Keep every other binding unchanged
  • Refuse to publish if the new list would be shorter than the previous one (thin Publish) or if the previous list was empty

Discovery and cards

When listing “all views” or workflow cards for a product:

  • Identify rows by stream + view (or equivalent view identity), not by rule_set_id alone. Distinct views can temporarily share a wrong rule_set_id after a bad Publish; deduping on ruleset collapses them.
  • After discovery, rebind each row’s ruleset id and version from the published Data Product quality.rules when a view match exists. Prefer that over stale ids on DMS triggers.
  • Keep cards for every product view even if some triggers are paused or thin.

App-side name matching is for editors and cards. The validation engine binds ruleset_references / Records ruleSetId by focus sh:targetClass (and SPARQL selects on non-stub shapes), skipping *ExistenceShape and other neighbour stubs so a relation stub in view A’s Turtle does not attach A’s RuleSet to view B. See Building a quality app — Engine focus binding.

Version selection and collision

Rule Why
List Data Product and RuleSet versions with an explicit high limit (for example 100) Default page size can miss higher versions after pruning, then Publish collides with “version already exists”
Choose “current” with highest published semantic version API order is not reliable
Next version = one patch above the maximum existing version string Using only “bound version + 1” fails when orphan higher versions still exist
When the product already has 10 versions, or the RuleSet already has 50, stop before any write and ask which versions to delete CDF rejects another version at those caps. Cancel deletes nothing and creates nothing
Never delete the current published Data Product version, or a RuleSet version a surviving Data Product still names Those versions are the publish base and the bindings that must keep working

Broken RuleSet versions

Load the RuleSet version the screen already resolved. If that version is not found:

  • Use another version only when the latest published quality.rules names the same external id at a different version.
  • Otherwise show the broken binding, including the external id and version, and ask the author to publish again.
  • Do not open the highest RuleSet version that happens to remain, and do not open a different external id.

When several bindings exist, match the view name first, then the row's own rule_set_id if it is in quality.rules. If neither matches, leave the row unchanged.

A publish of a ruleset starts one historic validation run. Do not start a second historic run because a status lookup was empty while the first run is still going.

Lookback and empty states

Compliance and overview UIs usually query Records with a time window (for example last N days). That window is an app filter, not a CDF limit.

  • Default and presets are product choices; persist the selection in the URL when useful so links are shareable.
  • Empty copy must not say “No data”. Prefer wording such as Not validated recently, no recent records, and mention the selected day count.
  • Widening lookback only includes older Records if they already exist — it does not start a validation job.
  • Point users to run or schedule validation, or widen lookback — not to “missing rules” or “deleted instances” when the window is empty.

Draft vs Test vs Publish

Action Creates CDF versions? Typical use
Edit / agent submit_rule No Update draft Turtle in the session
Test No Validate draft against instances
Publish Yes — new RuleSet version and new Data Product version Make the draft the bound production reference

After Test, do not discard a dirty draft when the UI refetches published Turtle. After Publish, load the newly bound RuleSet version — not an orphan draft file.

SHACL targeting

When generating or validating Turtle for a view, sh:targetClass must use the operational view space of the instances under validation — not the Data Product schema space used only for packaging metadata.

Both standard SPARQL placements are valid: nested under sh:property with sh:path, or pathless / cross-property constraints on the sh:NodeShape.

Common mistakes

What goes wrong Root cause Correct behavior
Publishing one view’s rules removes other views from the product Thin quality.rules (single binding) Merge siblings; refuse shrink
Multi-view product shows a single card Deduped on rule_set_id or trusted thin triggers Dedupe on view; rebind from Data Product
Publish fails with version already exists Incomplete version list (low page limit) List with high limit; next = max existing + patch
UI shows an old or wrong ruleset after Publish Reloaded File/draft instead of quality.rules target Reload the bound RuleSet version
Users think Test published No clear Publish gate Label Publish; disable until Test succeeds if that is your product rule
Empty scores look like deleted data Lookback window empty; vague copy Explicit “not validated recently” + lookback control
Fleet looks fully failed while listed rules pass Cards used passedValidation Count records that hit the rules on screen. See Building an app on the data quality engine
Immutable query returns 400 or a zero window lastUpdatedTime was omitted, changed mid-cursor, or a listed-rule or was placed inside the totals aggregate Send the time range on every filter and aggregate; count hits in a separate request
Editor shows the wrong rules after a missing version Highest leftover RuleSet, or a sibling id, was substituted Show the broken binding unless published quality.rules names another version of the same id
Rules never match instances Wrong space in sh:targetClass Operational view space

Checklist for a new editing surface

  1. Load latest published Data Product; fail closed if quality.rules is empty.
  2. Edit and Test never POST versions; only Publish does.
  3. Publish merges sibling bindings and POSTs RuleSet + Data Product versions.
  4. Version lists use a high limit; current = highest published semver.
  5. Discovery is view-keyed; rebind rules from the Data Product after load.
  6. Empty Records UI uses lookback-aware copy, never “No data”.
  7. After Publish, the workspace shows the newly referenced RuleSet version.
  8. Prove multi-view Publish on a product with at least two quality.rules bindings before shipping.

See also