feat(sdk)!: PlotMarkers builtin + unify markers/transforms into pj.data_processors.v1 (kind discriminator)#135
Merged
Merged
Conversation
New SDK builtin type for time-series plot markers (findings), the foundation of the anomaly-detection pipeline feature. - PlotMarkers (kPlotMarkers = 18): homogeneous, id-less marker records (Region / Event / ValueBand / Label) with shared semantic fields (status, severity, category, label, description, color, metadata) + a PlotMarkers list container. No id/source/scope by design — identity is owned by the host marker store; location carries provenance/reach. - Hand-written PJ.PlotMarkers wire codec following the image_annotations_codec idiom; PlotMarkers.proto contract. - Registered in C++ + C-ABI builtin enums, name()/parse()/typeOf(), and the ABI layout sentinel (static_assert id == 18). - Round-trip codec tests; full pj_base suite green (32/32) under ASAN. - Design (use-cases, architecture) + wire-format docs. ColorRGBA reused via image_annotations.hpp per the existing house convention (mesh3d / scene_entities do the same). abi/baseline.abi refresh + SDK MINOR version bump deferred to the release-cut step (additions-only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The plugin-facing contract for plot markers, mirroring the object
read/write host pattern.
- C vtable PJ_marker_store_host_vtable_t + fat pointer + an owning
PJ_marker_query_handle_t (plugin_data_api.h): add / remove / query /
query_bytes / release_query. Markers cross as serialized PJ.PlotMarkers
bytes; ids cross as uint64.
- C++ MarkerStoreHostView (plugin_data_api.hpp) wrapping the codec so
callers work in typed sdk::PlotMarker; add(single|batch) -> ids,
remove(id), query -> [{id, marker}]. + MarkerEntry.
- MarkerStoreService trait (service_traits.hpp), "pj.marker_store.v1".
ABI is additive (new service); abi/baseline.abi refresh deferred to the
release-cut step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ox slot Markers become a builtin object published to the ObjectStore via the generic object-write surface; the producer republishes the whole PlotMarkers set (last-writer-publish). The dedicated pj.marker_store.v1 service (vtable, MarkerStoreHostView, service trait) is removed. Add a tail-appended, idempotent toolbox-host slot register_object_topic_on_dataset(DatasetId, ...) so a toolbox can annotate an existing dataset (the object-write path was previously source-scoped). Add marker object-topic naming helpers (markerObjectTopicName / kGlobalMarkerTopic) to plot_markers.hpp. Update the ABI layout sentinel (88->96) and the two mock vtables. The PlotMarkers type + codec are unchanged. Docs (plot_markers_*) updated to the ObjectStore + republish model. MINOR bump + abi/baseline.abi refresh deferred to release. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…retention toolbox ABI slots Dialog protocol: add the MarkerTimeline custom widget (TimelineMark setters/event/parser/callback) for an editable multi-marker strip, with a single shared TimelineMark<->JSON codec used by all four sites. Toolbox host ABI (tail-appended, struct_size-gated): lookup_topic_on_dataset on the object-read vtable (dataset-scoped topic resolution; the name-only lookup is ambiguous across datasets) and set_object_topic_retention on the write vtable (keep-latest-N so a republishing producer's snapshots don't accumulate). Toolbox-host vtable 96->104; sentinels + mock vtables updated. Add sdk::markerSeriesKey (single-slash join tolerating a leading-'/' field) so producer and overlay build the per-series key identically. Doc refresh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Additive dialog-protocol surface (ABI-safe, no version bump): - setSaveFilePicker / isSaveFilePicker — a native "Save As" picker (getSaveFileName) so a not-yet-existing file can be created; reported via the existing onFileSelected. - ChartMarker + setChartMarkers / chartMarkers — overlay markers (events/regions/ value-bands) on a chart preview, alongside setChartSeries. - Fix a stale comment that referenced QLineSeries (the preview is Qwt, not Qt Charts). Cleanup: decodeMarker now drives the shared builtin_wire::parseFields loop like the sibling codecs, replacing the hand-rolled continue/break/skip control flow. Behavior and wire format are unchanged (codec round-trip tests pass). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RcygzPWUrHf9jHGGwzEzZo
Reconcile builtin-object ABI: VoxelGrid keeps slot 18 (released in 0.10.0), PlotMarkers moves to slot 19. SDK 0.8.0 -> 0.10.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BeoYQHmF6ixvYv9KXXgyVh
Adding the PlotMarkers canonical builtin (slot 19) is a MINOR addition over main's 0.11.0, so the merged SDK is a distinct 0.12.0 — not the upstream 0.11.0 (which lacks PlotMarkers). Keeps version identity clean for downstream pins. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BeoYQHmF6ixvYv9KXXgyVh
… ephemeral preview)
The whole-series analog of pj.data_processors.v1: a plugin submits a marker
generator by data (script + input series + an output marker-topic + params) and
the HOST runs it over the whole series and publishes the resulting PlotMarkers to
the ObjectStore. Nothing executable crosses the boundary.
- PJ_markers_host_vtable_t (plugin_data_api.h): create/remove/list/config +
tail-appended set_marker_preview / clear_marker_preview for a live, ephemeral
(non-persisted) preview generator read back through the object surface.
- MarkersHostView (sdk/plugin_data_api.hpp) + MarkersHostService trait.
- params_json {"scope":"all"} makes a global generator publish across every
dataset; absent = the active dataset only.
- markers_api_test.cpp: fake-host ABI test (forwarding, count-then-fill,
borrowed-string lifetime, binary-safe payload, preview slots).
Single output_marker_topic (not an outputs[] array): a generator writes exactly
one marker topic (global or per-series).
NOTE: MINOR version bump (conanfile.py / PJ_PACKAGE_VERSION / recipe.yaml) and the
abidiff baseline refresh are pending pre-merge — this commit adds API on a feature
branch (WIP) and does not bump the release version yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsY1SE2yBsX2qXvkgEnX8U
Bring v0.11.0 SDK work (voxel grid, data-processors API, RangeSlider markers) into the markers feature branch. Resolved widget_data.hpp additively: kept the feature's ChartMarker/TimelineMark + MarkerTimeline setters AND main's RangeSliderMarker + setRangeSliderMarkers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rh6GtzBNRVXXBUhFWhXEam
Unify the whole-series host-driven service under one SDK contract,
PJ_generators_host_vtable_t / GeneratorsHostService ("pj.generators.v1"),
with a string `kind` discriminator:
- kind="markers" (objects -> ObjectStore) is implemented; shared `language`
param, compile-only validate_script, ephemeral-preview flag, and out_topics
return apply across kinds.
- kind="transform" (per-sample timeseries -> DerivedEngine) is RESERVED, not
implemented; its end-state home is decided when the transform-editor work merges.
Removes PJ_markers_host_vtable_t / MarkersHostService (the old pj.markers.v1) and
markers_api_test; adds generators_api_test. A speculative kind="field" that briefly
existed on this branch was dropped (zero consumers, violated the object/timeseries
engine split); it changed no ABI struct since `kind` is a runtime string.
Versioning: API removal, normally MAJOR. Ships as 0.13.0 because no public tag ever
carried pj.markers.v1 (no released plugin breaks). The first public release carrying
pj.generators.v1 MUST be tagged 1.0.0 -- recorded in CHANGELOG.md + a comment by
`version` in conanfile.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013N1nzqco4Y9AQhnQgXo5tQ
…ta_processors.v1 Brings origin/main (#134 data-processors script API) into feature/plot-markers and collapses the two parallel whole-series host services into ONE: Pablo's `pj.data_processors.v1`, now carrying a string `kind` discriminator so a plugin chooses `markers`/`transform` (and future engines) from the SAME `create` call. SDK changes: - Generalized `PJ_data_processors_host_vtable_t.create_data_processor` with kind/language/flags/out_topics; ephemeral preview is now the PJ_DATA_PROCESSOR_FLAG_EPHEMERAL flag (removed the separate ephemeral slot); validate_data_processor_script gains `kind`. - Removed the interim `pj.generators.v1` entirely (vtable, host_t, GeneratorsHostService, GeneratorsHostView). - DataProcessorsHostView = unified `create(kind,…)` + thin shims createTransform/createEphemeralTransform (Status, signatures unchanged) / createMarkers. - Folded generators_api_test.cpp into data_processors_api_test.cpp (transform+markers). - CHANGELOG/conanfile reworded; stays 0.13.0 (no public tag carried the service yet; first public unified release MUST be 1.0.0). Builds green (RelWithDebInfo + debug/ASAN); 48/48 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHPK8VduUcoenjL8AtoSVd
Alvvalencia
changed the title
Applies the pre-commit clang-format pass that the merge commit (bf7eb0f) skipped via --no-verify. Formatting only — no semantic change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHPK8VduUcoenjL8AtoSVd
Completes the formatting pass over the full PR diff (PlotMarkers.proto column alignment + dialog_plugin_typed.hpp signature wrap) that the narrowly-scoped previous pass missed. Formatting only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHPK8VduUcoenjL8AtoSVd
Add WidgetData::setSaveFilePicker and the matching WidgetDataView accessors so a plugin can turn a button into a native "save as" chooser. The chosen path is delivered through the existing onFileSelected handler, so no new event type or typed handler is required. This is a backward-compatible API addition, so bump the SDK to 0.13.0 and bring recipe.yaml back in sync with the other version sources. Contents: - setSaveFilePicker(name, text, filter, title, default_suffix) -> "save_file_picker" action - WidgetDataView: isSaveFilePicker / saveFilePickerFilter / Title / DefaultSuffix - version 0.13.0 in conanfile.py, CMakeLists.txt, recipe.yaml + docstring example - docs: dialog-sdk-reference.md and dialog-plugin-guide.md
facontidavide
added a commit
that referenced
this pull request
Jul 6, 2026
The per-topic-pause work on this branch is strictly additive (tail slot + plugin extension, both struct_size/PJ_HAS_TAIL_SLOT-gated), and the last publicly tagged release (v0.14.0) already carries PR #135's pj.data_processors.v1 break -- git history confirms ca052ce is an ancestor of v0.14.0. So the delta since the last tag is additive-only: a MINOR bump is correct per CLAUDE.md's versioning rules, not the 1.0.0 the stale CHANGELOG/conanfile.py comments implied. Bump all three canonical version sources (conanfile.py, CMakeLists.txt PJ_PACKAGE_VERSION, recipe.yaml). Also fixes CHANGELOG.md: the pj.data_processors.v1 entry was still sitting under [Unreleased] with a "must ship as 1.0.0" note, even though it already shipped in v0.14.0 -- moved it to a backfilled [0.14.0] section with the versioning note corrected to past tense. packaging/conan-center-index/** intentionally untouched: it pins the real published v0.14.0 tag + its sha256 for eventual Conan Center submission, not one of the three synced version sources. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
facontidavide
added a commit
that referenced
this pull request
Jul 6, 2026
…tion.v1 (#140) * feat(sdk): DataSource per-topic pause — advertise + pj.topic_subscription.v1 Add a backward-compatible (additive) ABI so a streaming source can expose all its topics cheaply while only transmitting data for topics the host is actually displaying. - New capability PJ_DATA_SOURCE_CAPABILITY_PER_TOPIC_PAUSE (1<<6) + kCapabilityPerTopicPause mirror. - Plugin->host advertise: PJ_available_topic_t + notify_available_topics runtime-host tail slot, wrapped by DataSourceRuntimeHostView::notifyAvailableTopics (PJ_HAS_TAIL_SLOT-gated; errors on an old host so a new plugin can fall back). - Host->plugin control: get_plugin_extension("pj.topic_subscription.v1") -> PJ_topic_subscription_v1_t::set_active_topics (declarative full set), wrapped by DataSourceHandle::setActiveTopics (passes context() as ctx; safe no-op when the extension is absent). - Fix the stale "same thread as start()" runtime-host doc. ABI is additions-only: MIN_VTABLE_SIZE (128), PROTOCOL_VERSION (4) and PJ_ABI_VERSION (5) are unchanged; the runtime-host vtable deliberately grows 96->104. Sentinels updated. No version-source bump: recorded under CHANGELOG [Unreleased] (release number is a bundled, later decision). Tests: notify_available_topics_test (advertise round-trip, old-host degrade, extension round-trip, handle no-op) + updated helpers; 49/49 SDK tests pass. Docs + capability table updated; pj_base/CLAUDE.md versioning gotcha reconciled with Release Versioning. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pj_plugins): document per-topic pause in ARCHITECTURE/REQUIREMENTS REQUIREMENTS.md's DataSource capability table was missing kCapabilityPerTopicPause (data-source-guide.md had it, this table didn't). ARCHITECTURE.md had no mention of the new notify_available_topics tail slot or the pj.topic_subscription.v1 extension at all. Also fixes an unrelated pre-existing stale offset for push_message (documented as 96, actually 88 per the ABI sentinel test). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(pj_base): move non-template SDK method bodies out of headers DataSourcePluginBase::bind(), FileSourceBase/StreamSourceBase lifecycle methods, the DataSourceRuntimeHostView/ToolboxRuntimeHostView non-trivial methods, ToolboxPluginBase::bind(), and both families' ABI trampolines were inline in headers with no technical requirement to be (none are templates; pj_base already builds as a real STATIC lib that every plugin .so links privately, so these compile once into the archive instead of once per consumer TU). Declarations stay in the headers; bodies move to matching pj_base/src/*.cpp files. No behavior change — build clean, 49/49 tests pass, ABI layout sentinels unaffected. Templates (vtableWithCreate<CreateFn>, pushMessage<FetchMessageData>) and trivial one-line getters/forwards are left inline as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(sdk): bump version 0.14.0 -> 0.15.0 The per-topic-pause work on this branch is strictly additive (tail slot + plugin extension, both struct_size/PJ_HAS_TAIL_SLOT-gated), and the last publicly tagged release (v0.14.0) already carries PR #135's pj.data_processors.v1 break -- git history confirms ca052ce is an ancestor of v0.14.0. So the delta since the last tag is additive-only: a MINOR bump is correct per CLAUDE.md's versioning rules, not the 1.0.0 the stale CHANGELOG/conanfile.py comments implied. Bump all three canonical version sources (conanfile.py, CMakeLists.txt PJ_PACKAGE_VERSION, recipe.yaml). Also fixes CHANGELOG.md: the pj.data_processors.v1 entry was still sitting under [Unreleased] with a "must ship as 1.0.0" note, even though it already shipped in v0.14.0 -- moved it to a backfilled [0.14.0] section with the versioning note corrected to past tense. packaging/conan-center-index/** intentionally untouched: it pins the real published v0.14.0 tag + its sha256 for eventual Conan Center submission, not one of the three synced version sources. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Unifies the two parallel whole-series host services into one —
pj.data_processors.v1—with a string
kinddiscriminator, so a plugin choosesmarkers/transform(and futureengines) from the same
createcall. Also lands the canonical PlotMarkers builtinobject + codec.
This supersedes the interim
pj.generators.v1that earlier revisions of this branch added:that service is removed; its markers backend now lives under
kind="markers"on Pablo'sexisting
pj.data_processors.v1.ABI / API changes
PJ_data_processors_host_vtable_t.create_data_processorgeneralized withkind/language/flags/out_topics(the resolved sink names).PJ_DATA_PROCESSOR_FLAG_EPHEMERALflag oncreate_data_processor(the separate
create_data_processor_ephemeralslot is removed).validate_data_processor_scriptgains a leadingkindargument.pj.generators.v1entirely (vtable, host_t,GeneratorsHostService,GeneratorsHostView).DataProcessorsHostView= unifiedcreate(kind, …)+ thin convenience shimscreateTransform/createEphemeralTransform(Status return, signatures unchanged) /createMarkers.generators_api_test.cppfolded intodata_processors_api_test.cpp(transform + markers).Versioning
Stays
0.13.0: no public tag has carriedpj.data_processors.v1yet, so no released pluginbreaks. The first public release that carries the unified
pj.data_processors.v1must betagged
1.0.0(pre-1.0 break rule).This changes the
create_data_processor/validate_data_processor_scriptABI signatures, sothe host that IMPLEMENTS the service (
DataProcessorsRuntimeHost, PJ4 #234) must adopt thenew vtable fill. Callers keep working via the C++ shims; only
validateScriptcall-sitesgain a leading
kind("transform"). Merge this SDK first (it is the bottom layer); PJ4 / pluginsthen bump to it. The kind="transform" engine routing is owned by the transform-editor PRs.
Verification
./build.sh(RelWithDebInfo) +./build.sh --debug(ASAN) green;./test.sh48/48 pass.unified service (host + plugin both built against this ABI).
🤖 Generated with Claude Code