Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions aicontext/features/integrations/native-collector/feature.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ The wrapper is a **thin convenience layer**: it adds no wire behavior. Every reg
- **Version sensors emit on Start.** `AddAllModuleSensors(product_version)` and `AddCollectorMonitoringSensors()` don't just register `.module/Version` (the connected app's version, registered only when `product_version` is non-empty) and `.module/Collector version` (the collector **product** version `HSM_COLLECTOR_PRODUCT_VERSION` — the HSMDataCollector product line, kept in sync with the managed `HSMDataCollector.csproj <Version>`; **distinct from the C ABI version `HSM_COLLECTOR_VERSION` and from any host/agent version** — a host must NOT override it with its own version) — each posts a **one-shot value** (`Version.ToString()` shape + a `Start: <utc>` comment), mirroring managed `ProductVersionSensor.StartAsync` / `AddCollectorVersion`. Because the agent registers sensors while the collector is **Stopped** (where the data gate drops live values), the value is deferred (`deferred_start_values_`) and dispatched on the next `Start`, so it lands on the server on connect rather than being silently lost.
- **Collector self-monitoring emits values (#1198 follow-up).** `.module/Service alive` (heartbeat) and the `.module/Collector queue stats` bars (`Queue overflow`, `Items count in package`, `Package process time`, `Package content size`) were registered-but-empty in the native port. Handles are captured when `AddCollectorMonitoringSensors` / `AddAllQueueDiagnosticSensors` run; then a dedicated **self-monitor thread** posts the heartbeat (`true`, every collect period) plus the queue-overflow **delta** (only when non-zero, so the bar isn't all-zeros), and the **dispatch send path** posts per-package items/time/size from its UNLOCKED window — bars only aggregate (`AccumulateBar`), so there is no `queue_mutex_` re-entrancy. Overflow is counted lock-free (an atomic incremented in `Enqueue`); `AddBool` (which enqueues) only ever runs on the self-monitor thread, never under `queue_mutex_`. Guarded by `native_collector_self_monitoring_emits`.
- **Rate options parity (`RateSensorOptions`).** `RateOptions` carries the full registration surface (TTL / unit / display-unit / keep-history / self-destroy / statistics / aggregate / grafana / singleton / location), not just `post_period`, and lowers through the new C ABI `hsm_collector_create_rate_sensor_with_options`. The default cadence is **1 minute** (managed `RateSensorOptions.PostDataPeriod`, not the generic 15 s base); `CreateM1RateSensor` / `CreateM5RateSensor` mirror the managed convenience overloads (1 min / 5 min, always registering an **empty** Description vs `null` for a bare `CreateRateSensor`). When a field is left unset the two rate-specific defaults apply at the C ABI: `OriginalUnit` registers ValueInSecond (3000) and `DisplayUnit` registers 0 — both ALWAYS emitted (never null), matching `RateSensorOptions.ToApi()`; Description stays null. The bare `hsm_collector_create_rate_sensor` (post-period only) is unchanged, so the default rate registration stays byte-identical. Pinned cross-language by `rate_options_contract.hsmtest` + `native_rate_options_parity`.
- **TCP connection failure rate (admin rate metric, Windows; first built-in Rate consumer).** `EnableTcpConnectionFailureRateSensor(period)` (C ABI `hsm_collector_enable_tcp_connection_failure_rate_sensor`) starts an owned thread that each period reads the cumulative failed-connection-attempt counter (`MIB_TCPSTATS.dwAttemptFails`, IPv4+IPv6 summed via `GetTcpStatisticsEx`) and pushes the interval delta into a **Rate** sensor at `.computer/Network/Connection failures rate` (OriginalUnit ValueInSecond, **DisplayUnit per-minute** → failures/min). The first sample seeds the baseline; a quiet interval pushes nothing (posted rate 0). The sensor is created **synchronously at Start** (so it is deterministically registered); the thread is joined on Stop/Dispose. Call BEFORE `Start()`; Windows-only. The HsmAgent enables it by default on Windows (60 s). It carries a threshold **alert** (rides the registration): `Value > 1` failed attempt/sec (≈ 60/min), `AndSetSensorError` + notification, 5-minute confirmation period — so a sustained failure spike pages while transient blips don't. **Parity note:** managed has only `ConnectionFailuresCountSensor` (a delta count), no rate variant — native-only for now (like top-CPU), managed parity is a follow-up.
- **TCP connection failure rate (admin rate metric, Windows; first built-in Rate consumer).** `EnableTcpConnectionFailureRateSensor(period)` (C ABI `hsm_collector_enable_tcp_connection_failure_rate_sensor`) starts an owned thread that each period reads the cumulative failed-connection-attempt counter (`MIB_TCPSTATS.dwAttemptFails`, IPv4+IPv6 summed via `GetTcpStatisticsEx`) and pushes the interval delta into a **Rate** sensor at `.computer/Network/Connection failures rate` (OriginalUnit ValueInSecond, **DisplayUnit per-minute** → failures/min). The first sample seeds the baseline; a quiet interval pushes nothing (posted rate 0). The baseline/delta/wrap/quiet math lives in a portable, OS-free `TcpFailureRateAccumulator` (unit-tested by `tcp_failure_accumulator_*`); `WindowsTcpFailureSampler` is a thin `GetTcpStatisticsEx` wrapper over it. The sensor is created **synchronously at Start** (so it is deterministically registered); the thread is joined on Stop/Dispose. Call BEFORE `Start()`; Windows-only. The HsmAgent enables it by default on Windows (60 s). It carries a threshold **alert** (rides the registration): `Value > 1` failed attempt/sec (≈ 60/min), `AndSetSensorError` + notification, 5-minute confirmation period — so a sustained failure spike pages while transient blips don't. **Parity note:** managed has only `ConnectionFailuresCountSensor` (a delta count), no rate variant — native-only for now (like top-CPU), managed parity is a follow-up.
- **Live Windows service status (named service, Windows; wrapper-parity gap).** `EnableServiceStatusMonitoring(service_name, scan_period)` (C ABI `hsm_collector_enable_service_status_monitoring`) registers the `.module/Service status` **enum** sensor (catalog id, byte-identical to managed) at Start and starts an owned thread that every `scan_period` queries the named service via the Win32 SCM (`OpenSCManager`/`OpenService`/`QueryServiceStatus`) and posts its `ServiceControllerStatus` (Stopped=1…Paused=7 — the SCM `dwCurrentState` values are identical) to the sensor **on change**. A missing/unqueryable service posts **-1 with Error** ("Service not found!") once, then backs off for an hour (mirrors the managed `WindowsServiceStatusSensor` fault-state delay). The service name is the service NAME (case-insensitive), not the display name, and drives the query only — it is not part of the registration. Call BEFORE `Start()`; the thread is joined on Stop/Dispose. Replaces the C++/CLI wrapper's `AddServiceStateMonitoring(service_name)`. Smoke-tested by `native_service_status_sensor_emits_running_for_critical_service` + `native_service_status_sensor_reports_missing_service`.
- **Runtime alert attach to a built-in sensor (agent example).** The HsmAgent raises the built-in **Total CPU** sensor to **Error** on top of its default Warning: it grabs the sensor via `hsm_collector_add_default_sensor(TOTAL_CPU)` (idempotent by path — `AddAllComputerSensors` already created it), builds an `EmaMean > 50` alert with the fluent `CreateAlert(AlertKind::Bar).If(EmaMean, GreaterThan, "50").AsSensorError().ThenNotify(…).WithIcon(Error).WithConfirmationPeriod(5min)`, and `AttachAlert`s it before Start. `AttachAlert` rebuilds the sensor's registration in place, so the extra alert rides the AddOrUpdate. **No catalog/golden/managed change** — the shared default stays Warning@50%; this agent additionally Errors@50%. (`EmaMean`=213 is the bar's smoothed-mean property.) Pinned by `native_total_cpu_error_alert_attaches`.
- **Native wrapper DLL (`src/wrapper/`, drop-in replacement for the C++/CLI wrapper).** The legacy `hsm_wrapper::DataCollectorProxy` ABI is reimplemented over `hsm::collector`: `DataCollectorImpl` holds a `Collector`, lifecycle/`Initialize*Monitoring`/`Create*`/file/service-state forward to the native API, the static `AlertsBuilder` DSL accumulates plain data converted to a native alert (`Collector::CreateAlert` + `Sensor::AttachAlert`) at sensor creation, and the former managed func-sensor delegate bridge collapses (the native `Function`/`ValuesFunctionSensor` take a `std::function` directly). `Start`/`StartAsync` call `InstallWindowsMetricSources()` (live values, Windows) + `UseHttpTransport()` (real server) before the native `Start`. Built by `src/wrapper/CMakeLists.txt` (`add_subdirectory` of the native collector with `HSM_COLLECTOR_HTTP=ON`) into `HSMCppWrapper.dll`; `pch.h` force-defines `HSMWRAPPER_API=__declspec(dllexport)`. The public headers under `src/wrapper/include/` are unchanged, so a consumer relinks with zero source edits. Behavioral residue vs the managed wrapper is in `docs/native-collector-migration.md`; smoke-driven in-process by `WrapperConsole` (`src/wrapper/WrapperConsole/WrapperSmoke.cpp`), built + run in the `hsm-wrapper-build` CI lane. **Incremental-migration escape hatch:** `DataCollectorProxy::Native()` returns the wrapper's underlying `hsm::collector::Collector&`, so new sensors can be created directly against the native API on the same collector/connection while old sensors keep using the wrapper; the DLL re-exports the `extern "C"` C ABI (a CMake-generated `.def` from the frozen `hsm_collector.h`, `_t` typedefs filtered) so a consumer links only `HSMCppWrapper` — no second copy of the collector runtime. When the last wrapper call-site is gone, the proxy is swapped for a directly-owned `Collector` and the wrapper deleted (no big-bang). **Handoff:** the DLL is delivered to consumers as a versioned zip (both x64 configs + libcurl/zlib runtime + headers + a swap-recipe `MANIFEST.md`) attached to a `wrapper-v*` GitHub Release by the `hsm-wrapper-release` workflow — a stable cross-host download for the GitLab-hosted aggregator; layout mirrors their vendored tree so it unzips over the checkout. See `src/wrapper/packaging/` and the "Getting the native wrapper bundle" section of `docs/native-collector-migration.md`.
Expand Down Expand Up @@ -119,7 +119,8 @@ Tracked in full in `docs/native-collector-migration.md`. Headlines:
- `native_file_from_path_derives_name_extension_and_content` — `SendFile(path)` overrides the creation-time name/extension with the file's own stem/extension and posts the content; `native_file_from_path_missing_file_returns_not_found_and_sends_nothing` — a missing file returns `NOT_FOUND` and enqueues nothing.
- `rate_options_contract.hsmtest` — cross-language pin: the same options through `RateSensorOptions.ToApi()` (managed) and `hsm_collector_create_rate_sensor_with_options` (native) produce byte-identical registration text; includes the unset-`unit` ⇒ ValueInSecond(3000) default case.
- `bar_options_contract.hsmtest` — cross-language pin: the full bar registration surface (TTL/unit/keep-history/self-destroy/statistics/singleton/aggregate/grafana/description/`DefaultAlertsOptions`, `DisplayUnit:null`) through `BarSensorOptions.ToApi()` (managed) and `hsm_collector_create_double_bar_sensor_with_options` (native) is byte-identical.
- `native_tcp_failure_rate_sensor_registers` (Windows) — `EnableTcpConnectionFailureRateSensor` + Start registers a Rate sensor (type 9, OriginalUnit 3000, DisplayUnit 1) under `.computer/Network`. Live delta is non-deterministic host state, so only the registration is asserted (same approach as the network-speed/top-CPU owned-thread features).
- `native_tcp_failure_rate_sensor_registers` (Windows) — `EnableTcpConnectionFailureRateSensor` + Start registers a Rate sensor (type 9, OriginalUnit 3000, DisplayUnit 1) under `.computer/Network`. Live delta is non-deterministic host state, so only the registration is asserted (same approach as the network-speed/top-CPU owned-thread features). (Now actually wired into CTest — it previously existed only in the in-binary dispatch table and never ran in CI.)
- `tcp_failure_accumulator_*` (9 cases, cross-platform) — drive the portable `TcpFailureRateAccumulator` (the OS-free baseline/delta math extracted from `WindowsTcpFailureSampler`, fed synthetic per-family readings) through every branch: first-tick baseline seed → nullopt, growing counter → delta, IPv4+IPv6 deltas summed, quiet interval → nullopt, counter reset/wrap contributes 0 and re-baselines (no false spike), a reset in one family keeps the other's delta, an unreadable family contributes 0 and keeps its own baseline (resumes without a spike), both families unreadable → nullopt with baselines preserved, and independent per-family seeding. The Windows `GetTcpStatisticsEx` read is the only untested part.
- `native_service_status_sensor_emits_running_for_critical_service` / `native_service_status_sensor_reports_missing_service` (Windows) — the live SCM poller posts Running(4) for the always-present `RpcSs` service, and -1/Error ("Service not found!") for a missing service. Live host state, so asserted via the in-memory sent payloads.
- CI `install-consume` (Win+Linux) and `doxygen` lanes in `native-collector-conformance.yml`.
- Parity/process lanes (#1101): the per-PR `checklist-gate` job (strict checklist-disposition + unsupported-marker triage), the scheduled `native-collector-soak.yml` (heavier endurance fixtures on both drivers, drop-counter parity), and the scheduled alert-only `native-collector-benchmark.yml` (enqueue throughput + peak RSS vs `bench/baseline.json`).
14 changes: 14 additions & 0 deletions src/native/collector/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,15 @@ set(NATIVE_REGRESSION_TESTS
native_retry_meeting_full_queue_is_dropped_not_evicting_queued_values
native_retry_below_capacity_is_always_redelivered
cpu_select_top_n_selects_busiest_above_threshold
tcp_failure_accumulator_seeds_baseline_first_tick
tcp_failure_accumulator_returns_delta_after_seed
tcp_failure_accumulator_sums_both_families
tcp_failure_accumulator_quiet_interval_returns_nullopt
tcp_failure_accumulator_counter_reset_contributes_zero
tcp_failure_accumulator_reset_in_one_family_keeps_other_delta
tcp_failure_accumulator_unreadable_family_keeps_baseline
tcp_failure_accumulator_neither_readable_returns_nullopt
tcp_failure_accumulator_families_seed_independently
)

foreach(name IN LISTS NATIVE_REGRESSION_TESTS)
Expand Down Expand Up @@ -396,6 +405,11 @@ if(WIN32)
add_test(NAME native_service_status_sensor_reports_missing_service
COMMAND hsm_collector_tests native_service_status_sensor_reports_missing_service
)
# The opt-in TCP connection failure rate sensor registers a Rate sensor at Start (Windows-only, as
# EnableTcpConnectionFailureRateSensor is a no-op elsewhere).
add_test(NAME native_tcp_failure_rate_sensor_registers
COMMAND hsm_collector_tests native_tcp_failure_rate_sensor_registers
)
endif()

function(add_conformance_test name fixture)
Expand Down
90 changes: 56 additions & 34 deletions src/native/collector/src/tcp_connection_stats.cpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,54 @@
#include "tcp_connection_stats.hpp"

namespace hsm::collector
{
// --- Portable accumulator (no OS dependency; unit-tested directly) ---------------------------
//
// dwAttemptFails is the cumulative count of failed connection attempts — the same quantity the
// managed/PDH "TCPv4|TCPv6 / Connection Failures" counter reports. We keep a SEPARATE baseline per
// address family and sum the per-family deltas, so an intermittently-unreadable family just
// contributes 0 for that tick (and resumes cleanly from its own baseline) instead of poisoning a
// shared baseline.

void TcpFailureRateAccumulator::AccumulateFamily(FamilyBaseline& family, const FamilyReading& reading,
bool& any_readable, std::uint64_t& total_delta)
{
if (!reading.readable)
return; // unreadable this tick: contribute 0, keep the baseline so it resumes cleanly

any_readable = true;
const std::uint64_t current = reading.attempt_fails;

if (!family.have_baseline)
{
family.prev = current;
family.have_baseline = true;
return; // first reading of this family seeds its baseline, contributes nothing
}

if (current >= family.prev)
total_delta += current - family.prev;
// else: counter reset/wrap for this family — add nothing, just re-baseline below.
family.prev = current;
}

std::optional<double> TcpFailureRateAccumulator::Accumulate(const FamilyReading& v4, const FamilyReading& v6)
{
bool any_readable = false;
std::uint64_t total_delta = 0;

AccumulateFamily(v4_, v4, any_readable, total_delta);
AccumulateFamily(v6_, v6, any_readable, total_delta);

if (!any_readable)
return std::nullopt; // neither family readable this tick — skip, keep baselines
if (total_delta == 0)
return std::nullopt; // quiet interval (or baseline seeding) — nothing pushed, rate stays 0

return static_cast<double>(total_delta);
}
} // namespace hsm::collector

#ifdef _WIN32

// winsock2.h must precede windows.h so the WinSock2 types are available when iphlpapi.h pulls in the
Expand All @@ -18,44 +67,17 @@ namespace hsm::collector
{
std::optional<double> WindowsTcpFailureSampler::Sample()
{
// dwAttemptFails is the cumulative count of failed connection attempts — the same quantity the
// managed/PDH "TCPv4|TCPv6 / Connection Failures" counter reports. Track a SEPARATE baseline per
// address family and sum the per-family deltas, so an intermittently-unreadable family just
// contributes 0 for that tick (and resumes cleanly from its own baseline) instead of poisoning a
// shared baseline.
bool any_readable = false;
std::uint64_t total_delta = 0;

const auto sample_family = [&](FamilyBaseline& family, int address_family) {
// Read each family's cumulative dwAttemptFails; an unreadable family is reported as not-readable
// so the accumulator keeps its baseline. All baseline/delta/wrap/quiet math lives in the
// portable TcpFailureRateAccumulator (unit-tested), keeping this a thin OS wrapper.
const auto read_family = [](int address_family) -> TcpFailureRateAccumulator::FamilyReading {
MIB_TCPSTATS stats{};
if (GetTcpStatisticsEx(&stats, address_family) != NO_ERROR)
return; // unreadable this tick: contribute 0, keep the baseline so it resumes cleanly

any_readable = true;
const std::uint64_t current = stats.dwAttemptFails;

if (!family.have_baseline)
{
family.prev = current;
family.have_baseline = true;
return; // first reading of this family seeds its baseline, contributes nothing
}

if (current >= family.prev)
total_delta += current - family.prev;
// else: counter reset/wrap for this family — add nothing, just re-baseline below.
family.prev = current;
return { false, 0 };
return { true, stats.dwAttemptFails };
};

sample_family(v4_, AF_INET);
sample_family(v6_, AF_INET6);

if (!any_readable)
return std::nullopt; // neither family readable this tick — skip, keep baselines
if (total_delta == 0)
return std::nullopt; // quiet interval (or baseline seeding) — nothing pushed, rate stays 0

return static_cast<double>(total_delta);
return accumulator_.Accumulate(read_family(AF_INET), read_family(AF_INET6));
}
} // namespace hsm::collector

Expand Down
Loading
Loading