diff --git a/.github/workflows/firmware-build.yml b/.github/workflows/firmware-build.yml index 79ccd2f..46d55f0 100644 --- a/.github/workflows/firmware-build.yml +++ b/.github/workflows/firmware-build.yml @@ -46,13 +46,16 @@ jobs: # if-no-files-found: error only fires when ZERO files match, so a # partial match could silently upload an incomplete artifact set. - # Verify all three binaries exist before uploading. + # Verify all binaries exist before uploading. - name: Verify all binaries exist run: | test -f firmware/build/wateringsystem.bin test -f firmware/build/bootloader/bootloader.bin test -f firmware/build/partition_table/partition-table.bin test -f firmware/build/ota_data_initial.bin + # littlefs image for the `storage` partition, built by + # littlefs_create_partition_image() (feature 003, FR-007/SC-006). + test -f firmware/build/storage.bin - name: Upload binaries uses: actions/upload-artifact@v4 diff --git a/.specify/feature.json b/.specify/feature.json index d7c1352..229797a 100644 --- a/.specify/feature.json +++ b/.specify/feature.json @@ -1,3 +1,3 @@ { - "feature_directory": "specs/002-pump-gpio-board" + "feature_directory": "specs/003-nvs-littlefs-storage" } diff --git a/docs/parity-checklist.md b/docs/parity-checklist.md index 08071af..1ba811d 100644 --- a/docs/parity-checklist.md +++ b/docs/parity-checklist.md @@ -172,6 +172,50 @@ Route list extracted from `src/communication/WateringSystemWebServer.cpp:83-321` - [ ] `[HOST]` Pruning of old readings exists in the storage API but is **never called** by the application — history grows until reads start failing on the 16 KB JSON parse buffer (`src/storage/LittleFSStorage.cpp:374-440`, buffer `:237`, `:298`). Target: equivalent or better retention behavior with explicit bounding (new storage design may differ internally; behavior coverage = history endpoint keeps working over time) - [ ] `[HOST]` Storage stats (total/used bytes) available and reported in `/status` and serial status (`src/storage/LittleFSStorage.cpp:442-459`) +### Deliberate divergences in the ESP-IDF port (feature 003, PR-06) + +These are intentional behavior/format changes from the Arduino storage layer, +not parity targets. The master PRD authorizes a clean redesign (no migration); +each item below is the new contract behavior, host-tested in +`firmware/test_apps/host/`. + +- [ ] `[HOST]` **Configuration moves from `/config.json` to NVS** (namespace + `wscfg`, one typed entry per item). The legacy string-keyed JSON blob is + gone; defaults are compiled in and applied on missing/erased/out-of-range + entries (FR-002/FR-013). Factory reset = erase the `nvs` partition. The seven + watering items keep their legacy defaults and ranges (section 1); divergence: + `sensorReadInterval`/`dataLogInterval` are now first-class settable items + (legacy persisted them but exposed no setter) with new lower bounds (≥1 s / + ≥1 min) to prevent log-storm misconfiguration. +- [ ] `[HOST]` **WiFi "unconfigured" representation changes**: legacy used the + sentinel SSID `CONFIGURE_ME` in `/wifi_config.json`; the port stores + credentials in NVS and represents unconfigured as an **empty SSID string** + (factory state). PR-07 reads this for its AP-fallback decision; the legacy + password is never reused. +- [ ] `[HOST]` **Sensor history format redesigned and explicitly bounded**: + per-metric append-only chunk files of fixed 8-byte records + (`/storage/hist//.dat`, 8 KiB chunks, max 10 chunks per + metric, oldest-chunk eviction), replacing the unbounded JSON arrays. Resolves + the legacy defect at line 172 (history grew until the 16 KB parse buffer + failed). Retention target ≥30 days for all metrics at the default log + interval; max 10 distinct metrics (sized to the legacy metric set), an 11th + is rejected. Range-query behavior (inclusive filter, empty result on + no-data/error) is preserved. +- [ ] `[HOST]` **Event log is new surface** (no legacy equivalent): rotating + two-file log (`/storage/events/0.log`+`1.log`, 16 KiB each, oldest-half + rotation, newest always retained) for pump/fail-safe/connectivity/OTA/reset + events. Satisfies the constitution's "safety-relevant events MUST be + persisted"; producers are wired in PR-08. +- [ ] `[HOST]` **Interface split**: the legacy single `IDataStorage` + (config + history + stats, Arduino `String`) is redesigned into `IConfigStore` + + `IDataStorage` (history + events + stats, `std::string`). The two + never-called legacy methods (`getLastSensorReading`, `pruneOldReadings`) are + dropped; bounded retention is an internal guarantee, not a caller obligation. +- [ ] `[HOST]` Reservoir feature flags remain **not persisted** in this PR + (unchanged from legacy, line 171); the NVS-persistence decision is deferred to + PR-05 (reservoir board flag). The config store is extensible per-key so PR-05 + can add them without a contract change. + ## 7. WiFi / network / time - [ ] `[HIL]` STA connect at boot using saved credentials: STA mode, auto-reconnect off (handled manually), WiFi modem sleep disabled, clean disconnect first, **60 s** connect timeout with LED toggling every 500 ms (`src/main.cpp:46`, `168-212`) diff --git a/firmware/CLAUDE.md b/firmware/CLAUDE.md index 3e750e3..e074f3e 100644 --- a/firmware/CLAUDE.md +++ b/firmware/CLAUDE.md @@ -58,14 +58,23 @@ firmware/ │ ├── board/ # Board abstraction (header-only) │ │ └── include/board/board.h │ ├── interfaces/ # Header-only, NO IDF deps (host-includable) -│ │ └── include/interfaces/ # IActuator, IWaterPump, ITimeProvider -│ └── actuators/ # Pump drivers -│ ├── include/actuators/ # WaterPump (pure C++ logic), GpioWaterPump, -│ │ # EspTimeProvider (esp32-only header), -│ │ # testing/ (MockWaterPump, FakeTimeProvider) -│ └── src/ # GpioWaterPump.cpp excluded on linux target +│ │ └── include/interfaces/ # IActuator, IWaterPump, ITimeProvider, +│ │ # IConfigStore, IDataStorage +│ ├── actuators/ # Pump drivers +│ │ ├── include/actuators/ # WaterPump (pure C++ logic), GpioWaterPump, +│ │ │ # EspTimeProvider (esp32-only header), +│ │ │ # testing/ (MockWaterPump, FakeTimeProvider) +│ │ └── src/ # GpioWaterPump.cpp excluded on linux target +│ └── storage/ # Config + data persistence (feature 003) +│ ├── include/storage/ # NvsConfigStore, LittleFsDataStorage (POSIX, +│ │ # host-runnable), StorageMount (esp32-only), +│ │ # LockedConfigStore/LockedDataStorage, +│ │ # testing/ (MockConfigStore, MockDataStorage) +│ └── src/ # StorageMount.cpp + littlefs REQUIRES excluded +│ # on linux target (esp_littlefs has no port) └── test_apps/ - └── host/ # Host test app (linux preview target, Unity) + └── host/ # Host test app (linux preview target, Unity): + # pump + config store + data storage suites ``` Future components (drivers, controllers, web server) are added as siblings @@ -97,6 +106,46 @@ pump status pump status # both pumps ``` +Feature 003 adds `config` and `storage` subcommands (HIL verification path; the +handlers are thin interface calls, no logic). Credential values are never echoed +(FR-004): + +``` +config get | set | wifi | wifi-clear | factory-reset +storage stats | log | query [t0 t1] | event | events [n] +``` + +## Storage (config + data persistence) + +Feature 003 (PR-06). Two redesigned, host-includable interfaces in +`components/interfaces/`: + +- **`IConfigStore`** → `NvsConfigStore`: typed configuration in the `nvs` + partition (namespace `wscfg`, one entry per item), compiled-in factory + defaults applied on missing/erased/out-of-range entries (FR-013), explicit + `factoryReset()` (erases the partition). NVS runs natively on the linux + target, so the real store is host-tested — no mock skew. +- **`IDataStorage`** → `LittleFsDataStorage`: bounded sensor history + (per-metric 8-byte-record chunk files, ring eviction, ≥30-day retention, max + 10 metrics), a rotating event log (two 16 KiB files, newest always retained), + and filesystem usage stats. Implemented over **POSIX stdio with an injectable + base path**, so it builds and runs on the linux host; only `StorageMount` + (mount-or-format of the `storage` partition at `/storage`, `esp_littlefs_info` + stats) is esp32-only and excluded from the linux build via the component + CMakeLists `if(NOT ${IDF_TARGET} STREQUAL "linux")` guard. + +Concurrency: both base implementations are unsynchronized; anything accessed +from more than one task (main loop + console REPL) is wrapped in +`LockedConfigStore`/`LockedDataStorage` and accessed only through the wrapper — +the same pattern as `LockedWaterPump`. A committed seed directory +(`firmware/storage_image/`) feeds `littlefs_create_partition_image()`, which +emits `build/storage.bin` on every build (CI verifies it exists). No Arduino +data is migrated; on-disk formats diverge from legacy by design — see +`docs/parity-checklist.md` §6 "Deliberate divergences". + +The diagnostic console (`ws>`) exposes `config` and `storage` subcommands for +the HIL verification path (see below). + ## Board abstraction Two board revisions exist, selected via Kconfig (`main/Kconfig.projbuild`): diff --git a/firmware/components/interfaces/include/interfaces/IConfigStore.h b/firmware/components/interfaces/include/interfaces/IConfigStore.h new file mode 100644 index 0000000..ef4946b --- /dev/null +++ b/firmware/components/interfaces/include/interfaces/IConfigStore.h @@ -0,0 +1,167 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file IConfigStore.h + * @brief Typed, validated access to persisted configuration. + * + * Replaces the config half of the legacy IDataStorage (string-keyed + * storeConfig/getConfig over one JSON blob) — a deliberate contract + * redesign, same approach as PR-02's IWaterPump. Normative contract: + * specs/003-nvs-littlefs-storage/contracts/IConfigStore.md; item schema, + * defaults and ranges: specs/003-nvs-littlefs-storage/data-model.md. + * + * Invariants: + * 1. A getter never returns an out-of-range value — the compiled-in + * factory default shadows missing or invalid storage (FR-002). + * 2. A failed/rejected write never alters the stored value (FR-003): + * old-or-new, never torn. + * 3. No operation blocks on or is affected by network state. + * 4. WiFi credential values never appear in any diagnostic or log + * output (FR-004). + * + * Part of the header-only `interfaces` component: no IDF includes allowed + * (compiled on the host in the linux-target test suite). + */ + +#ifndef WATERINGSYSTEM_INTERFACES_ICONFIGSTORE_H +#define WATERINGSYSTEM_INTERFACES_ICONFIGSTORE_H + +#include +#include +#include + +/** + * @brief Typed configuration store with compiled-in factory defaults. + * + * Implementations are unsynchronized by design; cross-task consumers wrap + * them in the Locked* decorator (research.md D9, PR-02 CP3 precedent). + */ +class IConfigStore { +public: + // Factory defaults and valid ranges (data-model.md table) — the single + // source of truth shared by implementations, mocks and contract tests. + // Defaults mirror the frozen firmware (src/WateringController.cpp:16-21) + // except the documented divergences: the two interval items are settable + // and have new range floors (1 s / 1 min) against log-storm + // misconfiguration. + static constexpr float kMoistureThresholdMin = 0.0f; ///< % + static constexpr float kMoistureThresholdMax = 100.0f; ///< % + static constexpr float kDefaultMoistureThresholdLow = 30.0f; + static constexpr float kDefaultMoistureThresholdHigh = 55.0f; + + static constexpr uint32_t kWateringDurationMinS = 1; + static constexpr uint32_t kWateringDurationMaxS = 300; + static constexpr uint32_t kDefaultWateringDurationS = 20; + + static constexpr uint32_t kMinWateringIntervalFloorS = 1; + static constexpr uint32_t kDefaultMinWateringIntervalS = 300; + + static constexpr bool kDefaultWateringEnabled = true; + + static constexpr uint32_t kSensorReadIntervalFloorMs = 1000; + static constexpr uint32_t kDefaultSensorReadIntervalMs = 5000; + + static constexpr uint32_t kDataLogIntervalFloorMs = 60000; + static constexpr uint32_t kDefaultDataLogIntervalMs = 300000; + + static constexpr std::size_t kWifiSsidMaxLen = 32; ///< bytes + static constexpr std::size_t kWifiPasswordMaxLen = 64; ///< bytes + + virtual ~IConfigStore() = default; + + /** + * @brief Lower soil-moisture threshold in percent (0–100). + * + * Never fails: returns the stored value if present and in range, + * otherwise kDefaultMoistureThresholdLow. + */ + virtual float getMoistureThresholdLow() const = 0; + + /** + * @brief Set the lower soil-moisture threshold. + * + * In range (0–100, not NaN): persisted atomically, survives a power + * cycle, returns true. Out of range: returns false, stored value + * untouched. May also return false on a persistence failure. + */ + virtual bool setMoistureThresholdLow(float percent) = 0; + + /// Upper soil-moisture threshold in percent (0–100); same semantics. + virtual float getMoistureThresholdHigh() const = 0; + + /// Set the upper soil-moisture threshold; same semantics as the low one. + virtual bool setMoistureThresholdHigh(float percent) = 0; + + /// Watering run duration in seconds (1–300). + virtual uint32_t getWateringDurationS() const = 0; + + /// Set the watering run duration; rejects values outside 1–300 s. + virtual bool setWateringDurationS(uint32_t seconds) = 0; + + /// Minimum interval between watering runs (soak pause) in seconds (>= 1). + virtual uint32_t getMinWateringIntervalS() const = 0; + + /// Set the soak pause; rejects 0. + virtual bool setMinWateringIntervalS(uint32_t seconds) = 0; + + /// Whether automatic watering is enabled. + virtual bool getWateringEnabled() const = 0; + + /// Enable/disable automatic watering (no range to violate; false only + /// on a persistence failure). + virtual bool setWateringEnabled(bool enabled) = 0; + + /// Sensor read interval in milliseconds (>= 1000). + virtual uint32_t getSensorReadIntervalMs() const = 0; + + /// Set the sensor read interval; rejects values below 1000 ms. + virtual bool setSensorReadIntervalMs(uint32_t ms) = 0; + + /// Data log interval in milliseconds (>= 60000). + virtual uint32_t getDataLogIntervalMs() const = 0; + + /// Set the data log interval; rejects values below 60000 ms. + virtual bool setDataLogIntervalMs(uint32_t ms) = 0; + + /** + * @brief Stored WiFi SSID; empty string = unconfigured (factory state). + * + * Divergence from legacy: the Arduino firmware marked the unconfigured + * state with a CONFIGURE_ME sentinel in /wifi_config.json. + */ + virtual std::string getWifiSsid() const = 0; + + /// Stored WiFi password; empty string in the factory state. + virtual std::string getWifiPassword() const = 0; + + /** + * @brief Store WiFi credentials as one pair. + * + * Length-validated (SSID <= 32 bytes, password <= 64 bytes); over-long + * input is rejected with false, stored values untouched. + * Implementations MUST NOT log the values (FR-004). + */ + virtual bool setWifiCredentials(const std::string& ssid, + const std::string& password) = 0; + + /// Return both credential items to the factory (empty) state. + virtual bool clearWifiCredentials() = 0; + + /** + * @brief Factory reset: erase the underlying config storage. + * + * Afterwards every item reads its factory default and the credentials + * are removed (FR-005). The store remains usable without + * re-construction. + * + * WARNING: the NvsConfigStore implementation erases the ENTIRE default + * NVS partition (nvs_flash_erase_partition), not just the `wscfg` + * namespace. Any other namespace's data in that partition (future + * provisioning/OTA state, etc.) is therefore also cleared. This + * whole-partition erase is the documented data-model decision + * (data-model.md "Factory reset"), not an implementation accident. + */ + virtual bool factoryReset() = 0; +}; + +#endif /* WATERINGSYSTEM_INTERFACES_ICONFIGSTORE_H */ diff --git a/firmware/components/interfaces/include/interfaces/IDataStorage.h b/firmware/components/interfaces/include/interfaces/IDataStorage.h new file mode 100644 index 0000000..b660d74 --- /dev/null +++ b/firmware/components/interfaces/include/interfaces/IDataStorage.h @@ -0,0 +1,134 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file IDataStorage.h + * @brief Bounded sensor history, rotating event log, storage statistics. + * + * Replaces the data half of the legacy IDataStorage — a deliberate + * contract redesign, same approach as IConfigStore. The two legacy + * methods with no callers (getLastSensorReading, pruneOldReadings) are + * dropped; retention is an internal bounded-storage guarantee instead of + * a caller obligation. Normative contract: + * specs/003-nvs-littlefs-storage/contracts/IDataStorage.md; on-disk + * formats and budgets: specs/003-nvs-littlefs-storage/data-model.md. + * + * Invariants: + * 1. History and event storage never exceed their documented budgets; + * writes at the bound evict oldest data, never fail the append. + * 2. A power loss during append loses at most the in-flight record; + * previously stored records remain readable (torn tails detected + * and skipped on read). + * 3. Reads are side-effect free except torn-tail truncation/skip. + * 4. Timestamps are caller-supplied epoch seconds, stored verbatim + * (time correctness is the caller's concern, parity checklist 184). + * + * Part of the header-only `interfaces` component: no IDF includes allowed + * (compiled on the host in the linux-target test suite). + */ + +#ifndef WATERINGSYSTEM_INTERFACES_IDATASTORAGE_H +#define WATERINGSYSTEM_INTERFACES_IDATASTORAGE_H + +#include +#include +#include +#include + +/// One sensor reading; `metric` follows the legacy naming +/// (env_temperature, soil_moisture, ...) but the set is open. +struct SensorReading { + std::string metric; + uint32_t epoch = 0; ///< epoch seconds, caller-supplied + float value = 0.0f; +}; + +/// One persisted safety-relevant event. +struct EventRecord { + uint32_t epoch = 0; ///< epoch seconds, caller-supplied + uint8_t category = 0; ///< IDataStorage::kCategory* (open set) + std::string detail; ///< <= kEventDetailMaxLen bytes +}; + +/// Total/used bytes of the data filesystem (FR-008). +struct StorageStats { + uint32_t totalBytes = 0; + uint32_t usedBytes = 0; +}; + +/** + * @brief Bounded data storage with internal retention guarantees. + * + * Implementations are unsynchronized by design; cross-task consumers wrap + * them in the Locked* decorator (research.md D9, PR-02 CP3 precedent). + */ +class IDataStorage { +public: + // Contract-level bounds (data-model.md) — the single source of truth + // shared by implementations, mocks and contract tests. + + /// Budget guard: at most this many distinct metrics; storing an + /// extra distinct metric is rejected (prevents a buggy caller from + /// silently destroying history or blowing the budget). + static constexpr std::size_t kMaxMetrics = 10; + + /// Longer event details are silently truncated to this many bytes on + /// store — the event itself is always recorded, never rejected. + static constexpr std::size_t kEventDetailMaxLen = 120; + + // Event categories (FR-011). PR-08 may extend the set — unknown + // values are stored and returned verbatim. + static constexpr uint8_t kCategoryPump = 1; + static constexpr uint8_t kCategoryFailsafe = 2; + static constexpr uint8_t kCategoryConnectivity = 3; + static constexpr uint8_t kCategoryOta = 4; + static constexpr uint8_t kCategoryReset = 5; + + virtual ~IDataStorage() = default; + + /** + * @brief Append one sensor reading. + * + * Durable once true is returned (survives power loss). An unknown + * metric is accepted up to kMaxMetrics distinct metrics; one more + * distinct metric is rejected with false. Bounding/eviction is + * internal: >= 30-day retention at the default log interval, + * oldest-first eviction, never failing the append at the bound. + */ + virtual bool storeSensorReading(const std::string& metric, + uint32_t epoch, float value) = 0; + + /** + * @brief Readings for `metric` with epoch in [t0, t1] (inclusive). + * + * Chronological order. Empty vector on no data, unknown metric, + * t0 > t1, or read error — never throws/fails (legacy parity). + */ + virtual std::vector getSensorReadings( + const std::string& metric, uint32_t t0, uint32_t t1) const = 0; + + /** + * @brief Append one event record. + * + * `detail` longer than kEventDetailMaxLen bytes is silently + * truncated (the event is always recorded, never rejected for + * length). Rotation keeps total event storage within its budget and + * always retains the newest records. + */ + virtual bool storeEvent(uint32_t epoch, uint8_t category, + const std::string& detail) = 0; + + /** + * @brief Newest-first events, at most maxCount. + * + * Empty vector on no data or read error. "Newest-first" assumes + * monotonic caller-supplied epochs; under a non-monotonic clock the + * ordering degrades to active-file-first (time correctness is the + * caller's concern, parity checklist 184). + */ + virtual std::vector getEvents(std::size_t maxCount) const = 0; + + /// Total/used bytes of the data filesystem. + virtual StorageStats getStorageStats() const = 0; +}; + +#endif /* WATERINGSYSTEM_INTERFACES_IDATASTORAGE_H */ diff --git a/firmware/components/storage/CMakeLists.txt b/firmware/components/storage/CMakeLists.txt new file mode 100644 index 0000000..9b8e0f9 --- /dev/null +++ b/firmware/components/storage/CMakeLists.txt @@ -0,0 +1,31 @@ +# storage — persisted configuration (NVS) and data storage (littlefs). +# +# All configuration/data logic builds for every target including the linux +# preview target: host tests run against the real nvs_flash implementation +# (research.md D3) and plain POSIX file I/O (research.md D4). +# +# esp_littlefs has NO linux port, so the littlefs dependency and the +# target-only mount wrapper (src/StorageMount.cpp, user story 4) are +# confined to the non-linux branch below (research.md D4 mechanism). +if(${IDF_TARGET} STREQUAL "linux") + idf_component_register( + SRCS "src/NvsConfigStore.cpp" + "src/LittleFsDataStorage.cpp" + INCLUDE_DIRS "include" + REQUIRES nvs_flash interfaces + ) +else() + # Target build only: the littlefs mount/format/stats wrapper. littlefs + # is the managed joltwallet/littlefs component (pinned in + # main/idf_component.yml + dependencies.lock; registered under its + # namespaced component name). PRIV: littlefs headers appear only in + # src/StorageMount.cpp, never in this component's public headers. + idf_component_register( + SRCS "src/NvsConfigStore.cpp" + "src/LittleFsDataStorage.cpp" + "src/StorageMount.cpp" + INCLUDE_DIRS "include" + REQUIRES nvs_flash interfaces + PRIV_REQUIRES joltwallet__littlefs + ) +endif() diff --git a/firmware/components/storage/include/storage/LittleFsDataStorage.h b/firmware/components/storage/include/storage/LittleFsDataStorage.h new file mode 100644 index 0000000..9fd8180 --- /dev/null +++ b/firmware/components/storage/include/storage/LittleFsDataStorage.h @@ -0,0 +1,102 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file LittleFsDataStorage.h + * @brief IDataStorage over POSIX file I/O against an injectable base path. + * + * Written against plain POSIX stdio only — NO esp_littlefs includes — so + * the same code runs under the /storage littlefs VFS mount on target and + * against a temp directory in the linux-target host tests (research.md + * D4). Storage statistics come from an injected provider + * (esp_littlefs_info on target — wired in user story 4; a fake on host). + * + * On-disk formats per specs/003-nvs-littlefs-storage/data-model.md: + * - History: /hist//.dat append-only chunks of + * 8-byte little-endian records {uint32 epoch, float value}; chunks + * sealed at 8 KiB, at most 10 chunks per metric (ring eviction), at + * most 10 distinct metrics (11th rejected). + * - Events: /events/0.log + 1.log, 0xE7-framed records + * {marker, uint32 epoch, uint8 category, uint8 detail_len, detail}; + * 16 KiB per file, truncate-and-switch rotation (newest always kept). + * + * Durability (research.md D5): fflush+fsync per appended record; chunk + * eviction via remove(); no in-place overwrites of committed data. Torn + * tails: history = file size % 8 truncated logically on read; events = + * marker/length framing, invalid tail skipped. The write path repairs a + * torn tail (truncate to the valid prefix) before appending so committed + * records always stay parseable. + * + * Stateless with respect to the filesystem: every operation derives its + * state (active chunk, active event file) from the files themselves, so + * a restart needs no recovery step. Unsynchronized by design — + * cross-task consumers wrap the storage in the Locked* decorator + * (research.md D9, PR-02 CP3 precedent). + */ + +#ifndef WATERINGSYSTEM_STORAGE_LITTLEFSDATASTORAGE_H +#define WATERINGSYSTEM_STORAGE_LITTLEFSDATASTORAGE_H + +#include +#include +#include +#include +#include + +#include "interfaces/IDataStorage.h" + +/** + * @brief File-backed data storage (target littlefs VFS + host POSIX). + */ +class LittleFsDataStorage : public IDataStorage { +public: + /// Filesystem statistics callback; returns false when unavailable. + /// Target: esp_littlefs_info (user story 4). Host tests: a fake. + using StatsProvider = + std::function; + + // On-disk format constants (data-model.md) — exposed for the + // contract tests; the contract-level bounds live in IDataStorage. + static constexpr std::size_t kHistoryRecordBytes = 8; + static constexpr std::size_t kHistoryChunkMaxBytes = 8192; + static constexpr std::size_t kHistoryMaxChunksPerMetric = 10; + static constexpr std::size_t kEventFileMaxBytes = 16384; + static constexpr std::size_t kEventHeaderBytes = 7; ///< marker..detail_len + static constexpr uint8_t kEventMarker = 0xE7; + + /** + * @param basePath storage root without trailing slash ("/storage" on + * target, a temp directory in host tests) + * @param statsProvider filesystem statistics source; getStorageStats() + * reports zeros when absent or failing + */ + explicit LittleFsDataStorage(std::string basePath, + StatsProvider statsProvider = nullptr); + + // IDataStorage + bool storeSensorReading(const std::string& metric, uint32_t epoch, + float value) override; + std::vector getSensorReadings(const std::string& metric, + uint32_t t0, + uint32_t t1) const override; + bool storeEvent(uint32_t epoch, uint8_t category, + const std::string& detail) override; + std::vector getEvents(std::size_t maxCount) const override; + StorageStats getStorageStats() const override; + +private: + std::string histDir() const; + std::string metricDir(const std::string& metric) const; + std::string eventsDir() const; + std::string eventPath(int index) const; + + /// Which of the two event files appends are directed to, derived + /// from the files alone: the file whose last valid record has the + /// newest epoch (ties broken toward the smaller file; both empty + /// -> 0). Full rationale at the definition. + int activeEventIndex() const; + + std::string basePath_; + StatsProvider statsProvider_; +}; + +#endif /* WATERINGSYSTEM_STORAGE_LITTLEFSDATASTORAGE_H */ diff --git a/firmware/components/storage/include/storage/LockedConfigStore.h b/firmware/components/storage/include/storage/LockedConfigStore.h new file mode 100644 index 0000000..35dcf00 --- /dev/null +++ b/firmware/components/storage/include/storage/LockedConfigStore.h @@ -0,0 +1,178 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file LockedConfigStore.h + * @brief Mutex-serializing IConfigStore decorator (header-only). + * + * WHY THIS EXISTS: the config store is reached from more than one FreeRTOS + * task — the diag console REPL task issues get/set/factory-reset commands + * while application tasks (watering controller, web server in later PRs) + * read configuration. NvsConfigStore itself is deliberately unsynchronized + * (host-testable pure logic over per-operation NVS handles), so concurrent + * setters could interleave — e.g. setWifiCredentials() writes two NVS + * entries and a concurrent factoryReset() erasing the partition in between + * would leave a half-written credential pair. This decorator wraps an + * IConfigStore and takes a mutex around every interface call, serializing + * all access (FR-013; research.md D9, PR-02 CP3 precedent — + * actuators/LockedWaterPump.h). + * + * USAGE RULE: once a store is wrapped, the underlying store must ONLY be + * accessed through the wrapper — every call site (boot wiring, console + * registration, controllers, ...) goes through the LockedConfigStore, + * never through the wrapped object directly. + * + * SCOPE: this decorator provides PER-CALL atomicity only, not cross-call. + * A read-modify-write sequence spanning two calls (e.g. get* then set*) is + * NOT protected against an interleaving writer between the two — another + * task may change the value in between. Such sequences need higher-level + * coordination (a caller-held lock or single-owner task). + * + * Pure C++ ( is available via pthread on ESP-IDF and on the linux + * preview target), so the decorator is host-testable. + */ + +#ifndef WATERINGSYSTEM_STORAGE_LOCKEDCONFIGSTORE_H +#define WATERINGSYSTEM_STORAGE_LOCKEDCONFIGSTORE_H + +#include +#include +#include + +#include "interfaces/IConfigStore.h" + +/** + * @brief IConfigStore decorator that serializes every call with a mutex. + * + * Composition, not inheritance from a concrete store: the base class stays + * pure (no locking) and the existing host tests are unchanged. The wrapped + * store must outlive this object. + */ +class LockedConfigStore : public IConfigStore { +public: + /// Wrap @p store; the wrapped store must outlive this object. + explicit LockedConfigStore(IConfigStore& store) : store_(store) {} + + LockedConfigStore(const LockedConfigStore&) = delete; + LockedConfigStore& operator=(const LockedConfigStore&) = delete; + + float getMoistureThresholdLow() const override + { + std::lock_guard lock(mutex_); + return store_.getMoistureThresholdLow(); + } + + bool setMoistureThresholdLow(float percent) override + { + std::lock_guard lock(mutex_); + return store_.setMoistureThresholdLow(percent); + } + + float getMoistureThresholdHigh() const override + { + std::lock_guard lock(mutex_); + return store_.getMoistureThresholdHigh(); + } + + bool setMoistureThresholdHigh(float percent) override + { + std::lock_guard lock(mutex_); + return store_.setMoistureThresholdHigh(percent); + } + + uint32_t getWateringDurationS() const override + { + std::lock_guard lock(mutex_); + return store_.getWateringDurationS(); + } + + bool setWateringDurationS(uint32_t seconds) override + { + std::lock_guard lock(mutex_); + return store_.setWateringDurationS(seconds); + } + + uint32_t getMinWateringIntervalS() const override + { + std::lock_guard lock(mutex_); + return store_.getMinWateringIntervalS(); + } + + bool setMinWateringIntervalS(uint32_t seconds) override + { + std::lock_guard lock(mutex_); + return store_.setMinWateringIntervalS(seconds); + } + + bool getWateringEnabled() const override + { + std::lock_guard lock(mutex_); + return store_.getWateringEnabled(); + } + + bool setWateringEnabled(bool enabled) override + { + std::lock_guard lock(mutex_); + return store_.setWateringEnabled(enabled); + } + + uint32_t getSensorReadIntervalMs() const override + { + std::lock_guard lock(mutex_); + return store_.getSensorReadIntervalMs(); + } + + bool setSensorReadIntervalMs(uint32_t ms) override + { + std::lock_guard lock(mutex_); + return store_.setSensorReadIntervalMs(ms); + } + + uint32_t getDataLogIntervalMs() const override + { + std::lock_guard lock(mutex_); + return store_.getDataLogIntervalMs(); + } + + bool setDataLogIntervalMs(uint32_t ms) override + { + std::lock_guard lock(mutex_); + return store_.setDataLogIntervalMs(ms); + } + + std::string getWifiSsid() const override + { + std::lock_guard lock(mutex_); + return store_.getWifiSsid(); + } + + std::string getWifiPassword() const override + { + std::lock_guard lock(mutex_); + return store_.getWifiPassword(); + } + + bool setWifiCredentials(const std::string& ssid, + const std::string& password) override + { + std::lock_guard lock(mutex_); + return store_.setWifiCredentials(ssid, password); + } + + bool clearWifiCredentials() override + { + std::lock_guard lock(mutex_); + return store_.clearWifiCredentials(); + } + + bool factoryReset() override + { + std::lock_guard lock(mutex_); + return store_.factoryReset(); + } + +private: + IConfigStore& store_; + mutable std::mutex mutex_; +}; + +#endif /* WATERINGSYSTEM_STORAGE_LOCKEDCONFIGSTORE_H */ diff --git a/firmware/components/storage/include/storage/LockedDataStorage.h b/firmware/components/storage/include/storage/LockedDataStorage.h new file mode 100644 index 0000000..54e42e4 --- /dev/null +++ b/firmware/components/storage/include/storage/LockedDataStorage.h @@ -0,0 +1,99 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file LockedDataStorage.h + * @brief Mutex-serializing IDataStorage decorator (header-only). + * + * WHY THIS EXISTS: the data storage is reached from more than one FreeRTOS + * task — the diag console REPL task issues store/query/stats commands + * while application tasks (data logger, event sources, web server in later + * PRs) append and read. LittleFsDataStorage itself is deliberately + * unsynchronized (host-testable POSIX file logic) and derives its state + * from the files on every call, so concurrent appends could interleave — + * e.g. two storeEvent() calls deciding on the same active file offset, or + * a rotation truncating a file mid-append from another task. This + * decorator wraps an IDataStorage and takes a mutex around every interface + * call, serializing all access (FR-013; research.md D9, PR-02 CP3 + * precedent — actuators/LockedWaterPump.h). + * + * USAGE RULE: once a storage is wrapped, the underlying storage must ONLY + * be accessed through the wrapper — every call site (boot wiring, console + * registration, loggers, ...) goes through the LockedDataStorage, never + * through the wrapped object directly. + * + * SCOPE: this decorator provides PER-CALL atomicity only, not cross-call. + * A read-modify-write sequence spanning two calls (e.g. getSensorReadings + * then storeSensorReading) is NOT protected against an interleaving writer + * between the two — another task may append in between. Such sequences need + * higher-level coordination (a caller-held lock or single-owner task). + * + * Pure C++ ( is available via pthread on ESP-IDF and on the linux + * preview target), so the decorator is host-testable. + */ + +#ifndef WATERINGSYSTEM_STORAGE_LOCKEDDATASTORAGE_H +#define WATERINGSYSTEM_STORAGE_LOCKEDDATASTORAGE_H + +#include +#include +#include +#include +#include + +#include "interfaces/IDataStorage.h" + +/** + * @brief IDataStorage decorator that serializes every call with a mutex. + * + * Composition, not inheritance from a concrete storage: the base class + * stays pure (no locking) and the existing host tests are unchanged. The + * wrapped storage must outlive this object. + */ +class LockedDataStorage : public IDataStorage { +public: + /// Wrap @p storage; the wrapped storage must outlive this object. + explicit LockedDataStorage(IDataStorage& storage) : storage_(storage) {} + + LockedDataStorage(const LockedDataStorage&) = delete; + LockedDataStorage& operator=(const LockedDataStorage&) = delete; + + bool storeSensorReading(const std::string& metric, uint32_t epoch, + float value) override + { + std::lock_guard lock(mutex_); + return storage_.storeSensorReading(metric, epoch, value); + } + + std::vector getSensorReadings(const std::string& metric, + uint32_t t0, + uint32_t t1) const override + { + std::lock_guard lock(mutex_); + return storage_.getSensorReadings(metric, t0, t1); + } + + bool storeEvent(uint32_t epoch, uint8_t category, + const std::string& detail) override + { + std::lock_guard lock(mutex_); + return storage_.storeEvent(epoch, category, detail); + } + + std::vector getEvents(std::size_t maxCount) const override + { + std::lock_guard lock(mutex_); + return storage_.getEvents(maxCount); + } + + StorageStats getStorageStats() const override + { + std::lock_guard lock(mutex_); + return storage_.getStorageStats(); + } + +private: + IDataStorage& storage_; + mutable std::mutex mutex_; +}; + +#endif /* WATERINGSYSTEM_STORAGE_LOCKEDDATASTORAGE_H */ diff --git a/firmware/components/storage/include/storage/NvsConfigStore.h b/firmware/components/storage/include/storage/NvsConfigStore.h new file mode 100644 index 0000000..6228abd --- /dev/null +++ b/firmware/components/storage/include/storage/NvsConfigStore.h @@ -0,0 +1,74 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file NvsConfigStore.h + * @brief IConfigStore backed by NVS (namespace `wscfg`). + * + * Schema per specs/003-nvs-littlefs-storage/data-model.md: one NVS entry + * per item, floats stored as u32 bit patterns (NVS has no float type). + * Requires nvs_flash to be initialized before use (boot wiring in user + * story 4; the host test fixture initializes it explicitly). + * + * NVS handles are opened per operation, never held: factoryReset() erases + * the whole partition, which would invalidate any long-lived handle. + * + * Unsynchronized by design — cross-task consumers wrap the store in the + * Locked* decorator (research.md D9, PR-02 CP3 precedent). + */ + +#ifndef WATERINGSYSTEM_STORAGE_NVSCONFIGSTORE_H +#define WATERINGSYSTEM_STORAGE_NVSCONFIGSTORE_H + +#include +#include +#include + +#include "interfaces/IConfigStore.h" + +/** + * @brief NVS-backed configuration store (target + linux-target emulation). + */ +class NvsConfigStore : public IConfigStore { +public: + NvsConfigStore() = default; + + // IConfigStore + float getMoistureThresholdLow() const override; + bool setMoistureThresholdLow(float percent) override; + float getMoistureThresholdHigh() const override; + bool setMoistureThresholdHigh(float percent) override; + uint32_t getWateringDurationS() const override; + bool setWateringDurationS(uint32_t seconds) override; + uint32_t getMinWateringIntervalS() const override; + bool setMinWateringIntervalS(uint32_t seconds) override; + bool getWateringEnabled() const override; + bool setWateringEnabled(bool enabled) override; + uint32_t getSensorReadIntervalMs() const override; + bool setSensorReadIntervalMs(uint32_t ms) override; + uint32_t getDataLogIntervalMs() const override; + bool setDataLogIntervalMs(uint32_t ms) override; + std::string getWifiSsid() const override; + std::string getWifiPassword() const override; + bool setWifiCredentials(const std::string& ssid, + const std::string& password) override; + bool clearWifiCredentials() override; + bool factoryReset() override; + +private: + // Typed helpers over per-operation NVS handles. Getters shadow a + // missing or out-of-range stored value with the default (FR-002); + // setters reject out-of-range input without touching storage (FR-003). + float getFloat(const char* key, float defaultValue, float minValue, + float maxValue) const; + bool setFloat(const char* key, float value, float minValue, + float maxValue); + uint32_t getU32(const char* key, uint32_t defaultValue, uint32_t minValue, + uint32_t maxValue) const; + bool setU32(const char* key, uint32_t value, uint32_t minValue, + uint32_t maxValue); + bool getBool(const char* key, bool defaultValue) const; + bool setBool(const char* key, bool value); + std::string getString(const char* key, std::size_t maxLen) const; +}; + +#endif /* WATERINGSYSTEM_STORAGE_NVSCONFIGSTORE_H */ diff --git a/firmware/components/storage/include/storage/StorageMount.h b/firmware/components/storage/include/storage/StorageMount.h new file mode 100644 index 0000000..5c035a9 --- /dev/null +++ b/firmware/components/storage/include/storage/StorageMount.h @@ -0,0 +1,58 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file StorageMount.h + * @brief Target-only littlefs mount/format/stats wrapper (FR-007, FR-008). + * + * The ONLY littlefs-specific code in the storage component (research.md + * D2/D4): mounts the `storage` partition at /storage with + * format-if-mount-failed (legacy `LittleFS.begin(true)` parity — a + * corrupted filesystem is reformatted, never bricks the unit) and exposes + * the esp_littlefs_info-based statistics provider that boot wiring + * injects into LittleFsDataStorage. + * + * NOT built for the linux preview target — esp_littlefs has no linux + * port; the host tests exercise LittleFsDataStorage over a POSIX temp + * directory with a fake stats provider instead (see the storage + * component CMakeLists). Contains no logic beyond the IDF calls. + */ + +#ifndef WATERINGSYSTEM_STORAGE_STORAGEMOUNT_H +#define WATERINGSYSTEM_STORAGE_STORAGEMOUNT_H + +#include "esp_err.h" + +#include "storage/LittleFsDataStorage.h" + +/** + * @brief Mount-or-format of the `storage` partition (static-only helper). + */ +class StorageMount { +public: + /// Partition name in firmware/partitions.csv (littlefs is the subtype). + static constexpr const char* kPartitionLabel = "storage"; + + /// VFS mount point; base path for LittleFsDataStorage on target. + static constexpr const char* kBasePath = "/storage"; + + StorageMount() = delete; + + /** + * @brief Register the littlefs VFS: mount `storage` at /storage. + * + * format_if_mount_failed is set, so a fresh or corrupted partition is + * formatted and mounted instead of failing (FR-007; data loss + * accepted, bricking not). + * + * @return ESP_OK on success, the failing esp_err_t otherwise. + */ + static esp_err_t mount(void); + + /** + * @brief Stats provider over esp_littlefs_info for the mounted + * partition; inject into LittleFsDataStorage (FR-008). + */ + static LittleFsDataStorage::StatsProvider statsProvider(void); +}; + +#endif /* WATERINGSYSTEM_STORAGE_STORAGEMOUNT_H */ diff --git a/firmware/components/storage/include/storage/testing/MockConfigStore.h b/firmware/components/storage/include/storage/testing/MockConfigStore.h new file mode 100644 index 0000000..de43859 --- /dev/null +++ b/firmware/components/storage/include/storage/testing/MockConfigStore.h @@ -0,0 +1,255 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file MockConfigStore.h + * @brief In-memory IConfigStore test double (header-only). + * + * Holds the same contract invariants as the real store (FR-012): + * range-validated setters, default-shadowed getters, factory reset, + * credential length limits. `stored` is public so tests can inject + * out-of-range raw values and exercise the shadowing path; `failWrites` + * simulates a persistence failure. For consumer tests in later PRs + * (PR-07, PR-09, PR-11). Never compiled into target builds (only + * included from test code). No IDF includes. + */ + +#ifndef WATERINGSYSTEM_STORAGE_TESTING_MOCKCONFIGSTORE_H +#define WATERINGSYSTEM_STORAGE_TESTING_MOCKCONFIGSTORE_H + +#include +#include +#include +#include + +#include "interfaces/IConfigStore.h" + +/** + * @brief IConfigStore over plain members, instrumented for tests. + */ +class MockConfigStore : public IConfigStore { +public: + /** + * @brief Raw stored state, one optional per NVS entry (data-model.md). + * + * The bool item is a u8 — exactly like its NVS entry — so tests can + * inject an invalid stored value (neither 0 nor 1). + */ + struct Stored { + std::optional moistureThresholdLow; + std::optional moistureThresholdHigh; + std::optional wateringDurationS; + std::optional minWateringIntervalS; + std::optional wateringEnabled; + std::optional sensorReadIntervalMs; + std::optional dataLogIntervalMs; + std::optional wifiSsid; + std::optional wifiPassword; + }; + + Stored stored; + + // Instrumentation. + int acceptedWrites = 0; ///< setters that persisted a value + int rejectedWrites = 0; ///< setters rejected (validation or failWrites) + int factoryResets = 0; ///< successful factoryReset() calls + bool failWrites = false; ///< true: every write fails, state untouched + + // IConfigStore — getters (defaults shadow missing/invalid storage) + float getMoistureThresholdLow() const override + { + return shadowFloat(stored.moistureThresholdLow, + kDefaultMoistureThresholdLow); + } + + float getMoistureThresholdHigh() const override + { + return shadowFloat(stored.moistureThresholdHigh, + kDefaultMoistureThresholdHigh); + } + + uint32_t getWateringDurationS() const override + { + return shadowU32(stored.wateringDurationS, kDefaultWateringDurationS, + kWateringDurationMinS, kWateringDurationMaxS); + } + + uint32_t getMinWateringIntervalS() const override + { + return shadowU32(stored.minWateringIntervalS, + kDefaultMinWateringIntervalS, + kMinWateringIntervalFloorS, kNoUpperBound); + } + + bool getWateringEnabled() const override + { + if (!stored.wateringEnabled.has_value() || + (*stored.wateringEnabled != 0 && *stored.wateringEnabled != 1)) { + return kDefaultWateringEnabled; + } + return *stored.wateringEnabled == 1; + } + + uint32_t getSensorReadIntervalMs() const override + { + return shadowU32(stored.sensorReadIntervalMs, + kDefaultSensorReadIntervalMs, + kSensorReadIntervalFloorMs, kNoUpperBound); + } + + uint32_t getDataLogIntervalMs() const override + { + return shadowU32(stored.dataLogIntervalMs, kDefaultDataLogIntervalMs, + kDataLogIntervalFloorMs, kNoUpperBound); + } + + std::string getWifiSsid() const override + { + return shadowString(stored.wifiSsid, kWifiSsidMaxLen); + } + + std::string getWifiPassword() const override + { + return shadowString(stored.wifiPassword, kWifiPasswordMaxLen); + } + + // IConfigStore — setters (reject out-of-range, leave state untouched) + bool setMoistureThresholdLow(float percent) override + { + return writeFloat(stored.moistureThresholdLow, percent); + } + + bool setMoistureThresholdHigh(float percent) override + { + return writeFloat(stored.moistureThresholdHigh, percent); + } + + bool setWateringDurationS(uint32_t seconds) override + { + return writeU32(stored.wateringDurationS, seconds, + kWateringDurationMinS, kWateringDurationMaxS); + } + + bool setMinWateringIntervalS(uint32_t seconds) override + { + return writeU32(stored.minWateringIntervalS, seconds, + kMinWateringIntervalFloorS, kNoUpperBound); + } + + bool setWateringEnabled(bool enabled) override + { + if (failWrites) { + ++rejectedWrites; + return false; + } + stored.wateringEnabled = enabled ? 1 : 0; + ++acceptedWrites; + return true; + } + + bool setSensorReadIntervalMs(uint32_t ms) override + { + return writeU32(stored.sensorReadIntervalMs, ms, + kSensorReadIntervalFloorMs, kNoUpperBound); + } + + bool setDataLogIntervalMs(uint32_t ms) override + { + return writeU32(stored.dataLogIntervalMs, ms, kDataLogIntervalFloorMs, + kNoUpperBound); + } + + bool setWifiCredentials(const std::string& ssid, + const std::string& password) override + { + if (failWrites || ssid.size() > kWifiSsidMaxLen || + password.size() > kWifiPasswordMaxLen) { + ++rejectedWrites; + return false; + } + stored.wifiSsid = ssid; + stored.wifiPassword = password; + ++acceptedWrites; + return true; + } + + bool clearWifiCredentials() override + { + if (failWrites) { + ++rejectedWrites; + return false; + } + stored.wifiSsid.reset(); + stored.wifiPassword.reset(); + ++acceptedWrites; + return true; + } + + bool factoryReset() override + { + if (failWrites) { + return false; + } + stored = Stored{}; + ++factoryResets; + return true; + } + +private: + static constexpr uint32_t kNoUpperBound = UINT32_MAX; + + static float shadowFloat(const std::optional& value, + float defaultValue) + { + if (!value.has_value() || std::isnan(*value) || + *value < kMoistureThresholdMin || *value > kMoistureThresholdMax) { + return defaultValue; + } + return *value; + } + + static uint32_t shadowU32(const std::optional& value, + uint32_t defaultValue, uint32_t minValue, + uint32_t maxValue) + { + if (!value.has_value() || *value < minValue || *value > maxValue) { + return defaultValue; + } + return *value; + } + + static std::string shadowString(const std::optional& value, + std::size_t maxLen) + { + // Factory default for both credential items is the empty string. + if (!value.has_value() || value->size() > maxLen) { + return ""; + } + return *value; + } + + bool writeFloat(std::optional& slot, float value) + { + if (failWrites || std::isnan(value) || + value < kMoistureThresholdMin || value > kMoistureThresholdMax) { + ++rejectedWrites; + return false; + } + slot = value; + ++acceptedWrites; + return true; + } + + bool writeU32(std::optional& slot, uint32_t value, + uint32_t minValue, uint32_t maxValue) + { + if (failWrites || value < minValue || value > maxValue) { + ++rejectedWrites; + return false; + } + slot = value; + ++acceptedWrites; + return true; + } +}; + +#endif /* WATERINGSYSTEM_STORAGE_TESTING_MOCKCONFIGSTORE_H */ diff --git a/firmware/components/storage/include/storage/testing/MockDataStorage.h b/firmware/components/storage/include/storage/testing/MockDataStorage.h new file mode 100644 index 0000000..b1484c7 --- /dev/null +++ b/firmware/components/storage/include/storage/testing/MockDataStorage.h @@ -0,0 +1,139 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file MockDataStorage.h + * @brief In-memory IDataStorage test double (header-only). + * + * Holds the same contract invariants as the real storage (FR-012): + * inclusive chronological range queries that never fail, the + * kMaxMetrics distinct-metric cap, event detail truncation at + * kEventDetailMaxLen, newest-first event retrieval, and internal + * bounding that keeps the newest data. `history`, `events` and `stats` + * are public so tests can inject state directly; `failWrites` simulates + * a persistence failure. For consumer tests in later PRs (PR-08, PR-09). + * Never compiled into target builds (only included from test code). + * No IDF includes. + */ + +#ifndef WATERINGSYSTEM_STORAGE_TESTING_MOCKDATASTORAGE_H +#define WATERINGSYSTEM_STORAGE_TESTING_MOCKDATASTORAGE_H + +#include +#include +#include +#include +#include + +#include "interfaces/IDataStorage.h" + +/** + * @brief IDataStorage over plain containers, instrumented for tests. + */ +class MockDataStorage : public IDataStorage { +public: + // In-memory bounds mirroring the real store's budget granularity + // (data-model.md: 10 chunks x 1024 records per metric; events keep + // the newest, oldest evicted at the bound). + static constexpr std::size_t kMaxRecordsPerMetric = 10240; + static constexpr std::size_t kMaxEvents = 512; + + /// Per-metric history in append order (the real store's + /// "chronological by construction"). + std::map> history; + + /// Events in append order, oldest first (retrieval reverses). + std::vector events; + + /// Returned verbatim by getStorageStats(). + StorageStats stats{}; + + // Instrumentation. + int acceptedWrites = 0; ///< appends that were stored + int rejectedWrites = 0; ///< appends rejected (cap or failWrites) + bool failWrites = false; ///< true: every write fails, state untouched + + /// Defensive metric-name rule shared with the real store: names become + /// directory names there, so empty, '/' or ".." are rejected. + static bool isValidMetricName(const std::string& metric) + { + return !metric.empty() && metric.find('/') == std::string::npos && + metric.find("..") == std::string::npos; + } + + // IDataStorage + bool storeSensorReading(const std::string& metric, uint32_t epoch, + float value) override + { + if (failWrites || !isValidMetricName(metric)) { + ++rejectedWrites; + return false; + } + auto it = history.find(metric); + if (it == history.end()) { + if (history.size() >= kMaxMetrics) { + // Contract: one more distinct metric is rejected. + ++rejectedWrites; + return false; + } + it = history.emplace(metric, std::vector{}).first; + } + it->second.push_back(SensorReading{metric, epoch, value}); + if (it->second.size() > kMaxRecordsPerMetric) { + // Invariant 1: oldest-first eviction, never failing the append. + it->second.erase(it->second.begin()); + } + ++acceptedWrites; + return true; + } + + std::vector getSensorReadings(const std::string& metric, + uint32_t t0, + uint32_t t1) const override + { + std::vector result; + if (t0 > t1) { + return result; // contract: empty, not an error + } + const auto it = history.find(metric); + if (it == history.end()) { + return result; + } + for (const auto& reading : it->second) { + if (reading.epoch >= t0 && reading.epoch <= t1) { + result.push_back(reading); + } + } + return result; + } + + bool storeEvent(uint32_t epoch, uint8_t category, + const std::string& detail) override + { + if (failWrites) { + ++rejectedWrites; + return false; + } + // Over-long detail is truncated, never rejected (contract). + events.push_back( + EventRecord{epoch, category, detail.substr(0, kEventDetailMaxLen)}); + if (events.size() > kMaxEvents) { + events.erase(events.begin()); // newest always retained + } + ++acceptedWrites; + return true; + } + + std::vector getEvents(std::size_t maxCount) const override + { + std::vector result; + for (auto it = events.rbegin(); + it != events.rend() && result.size() < maxCount; ++it) { + result.push_back(*it); + } + return result; + } + + StorageStats getStorageStats() const override { return stats; } +}; + +#endif /* WATERINGSYSTEM_STORAGE_TESTING_MOCKDATASTORAGE_H */ diff --git a/firmware/components/storage/src/LittleFsDataStorage.cpp b/firmware/components/storage/src/LittleFsDataStorage.cpp new file mode 100644 index 0000000..3a52bb8 --- /dev/null +++ b/firmware/components/storage/src/LittleFsDataStorage.cpp @@ -0,0 +1,463 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file LittleFsDataStorage.cpp + * @brief IDataStorage over POSIX file I/O (sensor history + event log). + * + * POSIX stdio only, no esp_littlefs/IDF includes: the identical code runs + * against the /storage littlefs VFS mount on target and a temp directory + * in the linux-target host tests (research.md D4). On-disk formats per + * specs/003-nvs-littlefs-storage/data-model.md; durability per research D5 + * (fflush+fsync per append, remove() for eviction, torn tails repaired by + * truncating to the valid prefix before appending). + */ + +#include "storage/LittleFsDataStorage.h" + +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +namespace { + +/// One history chunk file, identified by its parsed filename epoch. +struct ChunkRef { + uint32_t firstEpoch = 0; + std::string name; +}; + +/// Metric names become directory names; empty, '/' or ".." are unsafe +/// (defensive — a buggy caller must not escape the history tree). +bool isValidMetricName(const std::string& metric) +{ + return !metric.empty() && metric.find('/') == std::string::npos && + metric.find("..") == std::string::npos; +} + +/// mkdir that treats an already existing directory as success. +bool ensureDir(const std::string& path) +{ + return ::mkdir(path.c_str(), 0775) == 0 || errno == EEXIST; +} + +bool isDir(const std::string& path) +{ + struct stat st {}; + return ::stat(path.c_str(), &st) == 0 && S_ISDIR(st.st_mode); +} + +/// File size in bytes, or -1 when the file cannot be stat'ed. +long fileSize(const std::string& path) +{ + struct stat st {}; + if (::stat(path.c_str(), &st) != 0) { + return -1; + } + return static_cast(st.st_size); +} + +/// Parse ".dat"; anything else is not a chunk file. +bool parseChunkName(const char* name, uint32_t& firstEpochOut) +{ + const char* dot = std::strchr(name, '.'); + if (dot == nullptr || dot == name || std::strcmp(dot, ".dat") != 0) { + return false; + } + unsigned long long value = 0; + for (const char* p = name; p != dot; ++p) { + if (*p < '0' || *p > '9') { + return false; + } + value = value * 10 + static_cast(*p - '0'); + if (value > UINT32_MAX) { + return false; + } + } + firstEpochOut = static_cast(value); + return true; +} + +/// Chunk files of one metric directory, sorted ascending by filename +/// epoch (oldest first). Empty when the directory does not exist. +std::vector listChunks(const std::string& dir) +{ + std::vector chunks; + if (DIR* d = ::opendir(dir.c_str())) { + while (const dirent* entry = ::readdir(d)) { + uint32_t firstEpoch = 0; + if (parseChunkName(entry->d_name, firstEpoch)) { + chunks.push_back(ChunkRef{firstEpoch, entry->d_name}); + } + } + ::closedir(d); + } + std::sort(chunks.begin(), chunks.end(), + [](const ChunkRef& a, const ChunkRef& b) { + return a.firstEpoch < b.firstEpoch; + }); + return chunks; +} + +/// Number of subdirectories (metric directories) under `dir`. +std::size_t countSubdirs(const std::string& dir) +{ + std::size_t count = 0; + if (DIR* d = ::opendir(dir.c_str())) { + while (const dirent* entry = ::readdir(d)) { + if (std::strcmp(entry->d_name, ".") == 0 || + std::strcmp(entry->d_name, "..") == 0) { + continue; + } + if (isDir(dir + "/" + entry->d_name)) { + ++count; + } + } + ::closedir(d); + } + return count; +} + +// Record codec: 8 bytes, explicitly little-endian {uint32 epoch, +// float value-bits} so files are portable between host and target. + +void encodeRecord(uint8_t out[LittleFsDataStorage::kHistoryRecordBytes], + uint32_t epoch, float value) +{ + uint32_t valueBits = 0; + std::memcpy(&valueBits, &value, sizeof(valueBits)); + for (int i = 0; i < 4; ++i) { + out[i] = static_cast((epoch >> (8 * i)) & 0xFF); + out[4 + i] = static_cast((valueBits >> (8 * i)) & 0xFF); + } +} + +uint32_t decodeU32Le(const uint8_t* bytes) +{ + return static_cast(bytes[0]) | + (static_cast(bytes[1]) << 8) | + (static_cast(bytes[2]) << 16) | + (static_cast(bytes[3]) << 24); +} + +float decodeFloatLe(const uint8_t* bytes) +{ + const uint32_t valueBits = decodeU32Le(bytes); + float value = 0.0f; + std::memcpy(&value, &valueBits, sizeof(value)); + return value; +} + +// Event-log codec: 0xE7-framed records {marker, uint32 LE epoch, +// uint8 category, uint8 detail_len, detail bytes} (data-model.md). + +/// Valid framed prefix of one event file: records in append order plus +/// the byte length of the parseable prefix. A torn tail — bad marker or +/// a frame shorter than its declared length, i.e. a power loss +/// mid-append — ends the prefix; everything before it stays usable +/// (contract invariant 2). An absent file is an empty log. +struct ParsedEventFile { + std::vector records; + long validBytes = 0; +}; + +ParsedEventFile parseEventFile(const std::string& path) +{ + ParsedEventFile parsed; + FILE* file = std::fopen(path.c_str(), "rb"); + if (file == nullptr) { + return parsed; + } + uint8_t header[LittleFsDataStorage::kEventHeaderBytes]; + char detail[UINT8_MAX]; // detail_len is one byte + for (;;) { + if (std::fread(header, 1, sizeof(header), file) != sizeof(header) || + header[0] != LittleFsDataStorage::kEventMarker) { + break; // torn/absent frame header: end of the valid prefix + } + const std::size_t detailLen = header[6]; + if (std::fread(detail, 1, detailLen, file) != detailLen) { + break; // declared length exceeds the file: torn detail + } + parsed.records.push_back(EventRecord{ + decodeU32Le(header + 1), header[5], std::string(detail, detailLen)}); + parsed.validBytes += static_cast(sizeof(header) + detailLen); + } + std::fclose(file); + return parsed; +} + +/// Create-or-empty a file: rotation truncates the standby event file +/// before switching appends to it (::truncate cannot create). +bool truncateToEmpty(const std::string& path) +{ + FILE* file = std::fopen(path.c_str(), "wb"); + if (file == nullptr) { + return false; + } + return std::fclose(file) == 0; +} + +} // namespace + +LittleFsDataStorage::LittleFsDataStorage(std::string basePath, + StatsProvider statsProvider) + : basePath_(std::move(basePath)), statsProvider_(std::move(statsProvider)) +{ +} + +bool LittleFsDataStorage::storeSensorReading(const std::string& metric, + uint32_t epoch, float value) +{ + if (!isValidMetricName(metric)) { + return false; + } + const std::string hist = histDir(); + const std::string dir = metricDir(metric); + if (!isDir(dir)) { + if (!ensureDir(basePath_) || !ensureDir(hist)) { + return false; + } + if (countSubdirs(hist) >= kMaxMetrics) { + return false; // budget guard: 11th distinct metric rejected + } + if (!ensureDir(dir)) { + return false; + } + } + + // Derive the active chunk from the files (stateless across restarts). + std::vector chunks = listChunks(dir); + std::string activePath; + if (!chunks.empty()) { + activePath = dir + "/" + chunks.back().name; + long size = fileSize(activePath); + if (size < 0) { + return false; + } + const long torn = size % static_cast(kHistoryRecordBytes); + if (torn != 0) { + // Repair a torn tail (power loss mid-append) so committed + // records stay 8-byte aligned and parseable. + if (::truncate(activePath.c_str(), size - torn) != 0) { + return false; + } + size -= torn; + } + if (static_cast(size) >= kHistoryChunkMaxBytes) { + activePath.clear(); // sealed at 8 KiB — start a successor + } + } + if (activePath.empty()) { + if (chunks.size() >= kHistoryMaxChunksPerMetric) { + // Ring bound: creating chunk #11 deletes the oldest. + if (std::remove((dir + "/" + chunks.front().name).c_str()) != 0) { + return false; + } + chunks.erase(chunks.begin()); + } + // Filename = first record's epoch. A non-monotonic epoch that + // does not sort after the sealed chunk is bumped past it: chunk + // names must stay strictly increasing for ordering/eviction + // (time correctness is the caller's concern, parity 184). + uint32_t chunkEpoch = epoch; + if (!chunks.empty() && chunkEpoch <= chunks.back().firstEpoch) { + chunkEpoch = chunks.back().firstEpoch + 1; + } + activePath = dir + "/" + std::to_string(chunkEpoch) + ".dat"; + } + + FILE* file = std::fopen(activePath.c_str(), "ab"); + if (file == nullptr) { + return false; + } + uint8_t record[kHistoryRecordBytes]; + encodeRecord(record, epoch, value); + // Durable once true is returned: flush stdio, then sync to flash. + bool ok = std::fwrite(record, 1, sizeof(record), file) == sizeof(record) && + std::fflush(file) == 0 && ::fsync(fileno(file)) == 0; + ok = (std::fclose(file) == 0) && ok; + return ok; +} + +std::vector LittleFsDataStorage::getSensorReadings( + const std::string& metric, uint32_t t0, uint32_t t1) const +{ + std::vector result; + if (t0 > t1 || !isValidMetricName(metric)) { + return result; // contract: empty, never an error + } + const std::string dir = metricDir(metric); + // At most 10 chunks (80 KiB) per metric: scanning every chunk and + // filtering per record is cheap and stays correct for any input. + for (const ChunkRef& chunk : listChunks(dir)) { + FILE* file = std::fopen((dir + "/" + chunk.name).c_str(), "rb"); + if (file == nullptr) { + continue; // unreadable chunk: skip, never fail the query + } + uint8_t record[kHistoryRecordBytes]; + // A short final read is a torn tail — logically truncated here. + while (std::fread(record, 1, sizeof(record), file) == sizeof(record)) { + const uint32_t epoch = decodeU32Le(record); + if (epoch >= t0 && epoch <= t1) { + result.push_back( + SensorReading{metric, epoch, decodeFloatLe(record + 4)}); + } + } + std::fclose(file); + } + return result; +} + +bool LittleFsDataStorage::storeEvent(uint32_t epoch, uint8_t category, + const std::string& detail) +{ + if (!ensureDir(basePath_) || !ensureDir(eventsDir())) { + return false; + } + // Contract: an over-long detail is silently truncated — the event + // itself is always recorded, never rejected for length. + const std::size_t detailLen = std::min(detail.size(), kEventDetailMaxLen); + const std::size_t recordBytes = kEventHeaderBytes + detailLen; + + int active = activeEventIndex(); + std::string path = eventPath(active); + const ParsedEventFile parsed = parseEventFile(path); + const long size = fileSize(path); + // An absent file (size < 0, validBytes 0) is a not-yet-created event + // log, not an error: the append below creates it. A stat failure on a + // file that does hold valid bytes is a real error — append anyway would + // corrupt the frame boundary, so refuse it (mirrors the history path). + if (size < 0 && parsed.validBytes > 0) { + return false; + } + if (size > parsed.validBytes) { + // Repair a torn tail (power loss mid-append) so the new record + // lands on a frame boundary and the whole file stays parseable. + if (::truncate(path.c_str(), parsed.validBytes) != 0) { + return false; + } + } + if (parsed.validBytes + static_cast(recordBytes) > + static_cast(kEventFileMaxBytes)) { + // Rotation (data-model.md): the append would exceed the 16 KiB + // cap, so truncate the standby file and switch appends to it — + // the oldest half is dropped, the newest records always kept. + active = 1 - active; + path = eventPath(active); + if (!truncateToEmpty(path)) { + return false; + } + } + + uint8_t header[kEventHeaderBytes]; + header[0] = kEventMarker; + for (int i = 0; i < 4; ++i) { + header[1 + i] = static_cast((epoch >> (8 * i)) & 0xFF); + } + header[5] = category; + header[6] = static_cast(detailLen); + + FILE* file = std::fopen(path.c_str(), "ab"); + if (file == nullptr) { + return false; + } + // Durable once true is returned: flush stdio, then sync to flash. + bool ok = std::fwrite(header, 1, sizeof(header), file) == sizeof(header) && + std::fwrite(detail.data(), 1, detailLen, file) == detailLen && + std::fflush(file) == 0 && ::fsync(fileno(file)) == 0; + ok = (std::fclose(file) == 0) && ok; + return ok; +} + +std::vector LittleFsDataStorage::getEvents( + std::size_t maxCount) const +{ + // Each file holds records in append order; the active file's records + // are all newer than the standby's (rotation empties the file it + // switches to). Newest-first therefore = active reversed, then + // standby reversed. Files are <= 16 KiB and event rates are low, so + // re-parsing per call keeps this stateless across restarts. + const int active = activeEventIndex(); + const ParsedEventFile newer = parseEventFile(eventPath(active)); + const ParsedEventFile older = parseEventFile(eventPath(1 - active)); + + std::vector result; + result.reserve( + std::min(maxCount, newer.records.size() + older.records.size())); + for (const ParsedEventFile* file : {&newer, &older}) { + for (auto it = file->records.rbegin(); + it != file->records.rend() && result.size() < maxCount; ++it) { + result.push_back(*it); + } + } + return result; +} + +StorageStats LittleFsDataStorage::getStorageStats() const +{ + StorageStats stats{}; + if (statsProvider_) { + uint32_t totalBytes = 0; + uint32_t usedBytes = 0; + if (statsProvider_(totalBytes, usedBytes)) { + stats.totalBytes = totalBytes; + stats.usedBytes = usedBytes; + } + } + return stats; +} + +std::string LittleFsDataStorage::histDir() const { return basePath_ + "/hist"; } + +std::string LittleFsDataStorage::metricDir(const std::string& metric) const +{ + return histDir() + "/" + metric; +} + +std::string LittleFsDataStorage::eventsDir() const +{ + return basePath_ + "/events"; +} + +std::string LittleFsDataStorage::eventPath(int index) const +{ + return eventsDir() + "/" + std::to_string(index) + ".log"; +} + +int LittleFsDataStorage::activeEventIndex() const +{ + // Restart-recovery rule, derived from the files alone (stateless): + // the file whose last valid record carries the newest epoch is the + // active one — appends always extend the newest end of the log. A + // file without valid records cannot hold the newest half; both + // empty/absent means a fresh log starting at 0. Equal last epochs + // (several events within one second around a rotation) break the + // tie toward the smaller file: right after a rotation the active + // file is the freshly truncated, smaller one while the sealed + // standby sits near the 16 KiB cap. A full file picked as active + // self-corrects on the next append (rotation switches away from + // it), so a wrong pick cannot wedge the log. Epoch ordering assumes + // the caller's clock — time correctness is the caller's concern + // (parity checklist 184). + const ParsedEventFile file0 = parseEventFile(eventPath(0)); + const ParsedEventFile file1 = parseEventFile(eventPath(1)); + if (file1.records.empty()) { + return 0; // covers both-empty: a fresh log starts at 0 + } + if (file0.records.empty()) { + return 1; + } + const uint32_t last0 = file0.records.back().epoch; + const uint32_t last1 = file1.records.back().epoch; + if (last0 != last1) { + return last0 > last1 ? 0 : 1; + } + return file0.validBytes <= file1.validBytes ? 0 : 1; +} diff --git a/firmware/components/storage/src/NvsConfigStore.cpp b/firmware/components/storage/src/NvsConfigStore.cpp new file mode 100644 index 0000000..3ae4c0a --- /dev/null +++ b/firmware/components/storage/src/NvsConfigStore.cpp @@ -0,0 +1,405 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file NvsConfigStore.cpp + * @brief NVS-backed IConfigStore implementation. + * + * Builds for every target including the linux preview target, where the + * host test suite runs it against the real nvs_flash implementation + * (research.md D3). + * + * FR-004: WiFi credential VALUES must never reach any log output — only + * key names and esp_err codes are logged in this file. + */ + +#include "storage/NvsConfigStore.h" + +#include +#include + +#include "esp_log.h" +#include "nvs.h" +#include "nvs_flash.h" + +namespace { + +const char* TAG = "nvsconfigstore"; + +// Namespace and keys per data-model.md (NVS key limit: 15 chars). +constexpr const char* kNamespace = "wscfg"; +constexpr const char* kKeyMoistLow = "moist_low"; +constexpr const char* kKeyMoistHigh = "moist_high"; +constexpr const char* kKeyWaterDur = "water_dur"; +constexpr const char* kKeySoakPause = "soak_pause"; +constexpr const char* kKeyWaterEn = "water_en"; +constexpr const char* kKeyReadIv = "read_iv"; +constexpr const char* kKeyLogIv = "log_iv"; +constexpr const char* kKeyWifiSsid = "wifi_ssid"; +constexpr const char* kKeyWifiPass = "wifi_pass"; + +constexpr uint32_t kNoUpperBound = UINT32_MAX; + +/// RAII guard for a C-API NVS handle (opened per operation, see header). +class NvsHandleGuard { +public: + NvsHandleGuard(const char* ns, nvs_open_mode_t mode) + { + err_ = nvs_open(ns, mode, &handle_); + } + + ~NvsHandleGuard() + { + if (err_ == ESP_OK) { + nvs_close(handle_); + } + } + + NvsHandleGuard(const NvsHandleGuard&) = delete; + NvsHandleGuard& operator=(const NvsHandleGuard&) = delete; + + bool ok() const { return err_ == ESP_OK; } + esp_err_t error() const { return err_; } + nvs_handle_t get() const { return handle_; } + +private: + nvs_handle_t handle_ = 0; + esp_err_t err_ = ESP_FAIL; +}; + +// NVS has no float type: floats are stored as u32 bit patterns (data-model). +uint32_t floatToBits(float value) +{ + static_assert(sizeof(uint32_t) == sizeof(float)); + uint32_t bits = 0; + std::memcpy(&bits, &value, sizeof(bits)); + return bits; +} + +float bitsToFloat(uint32_t bits) +{ + float value = 0.0f; + std::memcpy(&value, &bits, sizeof(value)); + return value; +} + +bool commitValue(nvs_handle_t handle, const char* key, esp_err_t setErr) +{ + esp_err_t err = setErr; + if (err == ESP_OK) { + err = nvs_commit(handle); + } + if (err != ESP_OK) { + ESP_LOGE(TAG, "persisting %s failed: %s", key, esp_err_to_name(err)); + return false; + } + return true; +} + +} // namespace + +// --------------------------------------------------------------------------- +// Typed helpers +// --------------------------------------------------------------------------- + +float NvsConfigStore::getFloat(const char* key, float defaultValue, + float minValue, float maxValue) const +{ + NvsHandleGuard handle(kNamespace, NVS_READONLY); + if (!handle.ok()) { + // Namespace absent (factory state) or NVS unavailable: default. + return defaultValue; + } + uint32_t bits = 0; + if (nvs_get_u32(handle.get(), key, &bits) != ESP_OK) { + return defaultValue; + } + const float value = bitsToFloat(bits); + if (std::isnan(value) || value < minValue || value > maxValue) { + // FR-002: shadow with the default; the invalid entry stays in + // place until the next valid write. + ESP_LOGW(TAG, "stored %s out of range, using default", key); + return defaultValue; + } + return value; +} + +bool NvsConfigStore::setFloat(const char* key, float value, float minValue, + float maxValue) +{ + if (std::isnan(value) || value < minValue || value > maxValue) { + return false; // FR-003: rejected, stored value untouched + } + NvsHandleGuard handle(kNamespace, NVS_READWRITE); + if (!handle.ok()) { + ESP_LOGE(TAG, "nvs_open for %s failed: %s", key, + esp_err_to_name(handle.error())); + return false; + } + return commitValue(handle.get(), key, + nvs_set_u32(handle.get(), key, floatToBits(value))); +} + +uint32_t NvsConfigStore::getU32(const char* key, uint32_t defaultValue, + uint32_t minValue, uint32_t maxValue) const +{ + NvsHandleGuard handle(kNamespace, NVS_READONLY); + if (!handle.ok()) { + return defaultValue; + } + uint32_t value = 0; + if (nvs_get_u32(handle.get(), key, &value) != ESP_OK) { + return defaultValue; + } + if (value < minValue || value > maxValue) { + ESP_LOGW(TAG, "stored %s out of range, using default", key); + return defaultValue; + } + return value; +} + +bool NvsConfigStore::setU32(const char* key, uint32_t value, + uint32_t minValue, uint32_t maxValue) +{ + if (value < minValue || value > maxValue) { + return false; + } + NvsHandleGuard handle(kNamespace, NVS_READWRITE); + if (!handle.ok()) { + ESP_LOGE(TAG, "nvs_open for %s failed: %s", key, + esp_err_to_name(handle.error())); + return false; + } + return commitValue(handle.get(), key, + nvs_set_u32(handle.get(), key, value)); +} + +bool NvsConfigStore::getBool(const char* key, bool defaultValue) const +{ + NvsHandleGuard handle(kNamespace, NVS_READONLY); + if (!handle.ok()) { + return defaultValue; + } + uint8_t value = 0; + if (nvs_get_u8(handle.get(), key, &value) != ESP_OK) { + return defaultValue; + } + if (value != 0 && value != 1) { + // Valid range for the bool item is 0/1 (data-model). + ESP_LOGW(TAG, "stored %s out of range, using default", key); + return defaultValue; + } + return value == 1; +} + +bool NvsConfigStore::setBool(const char* key, bool value) +{ + NvsHandleGuard handle(kNamespace, NVS_READWRITE); + if (!handle.ok()) { + ESP_LOGE(TAG, "nvs_open for %s failed: %s", key, + esp_err_to_name(handle.error())); + return false; + } + return commitValue(handle.get(), key, + nvs_set_u8(handle.get(), key, value ? 1 : 0)); +} + +std::string NvsConfigStore::getString(const char* key, + std::size_t maxLen) const +{ + NvsHandleGuard handle(kNamespace, NVS_READONLY); + if (!handle.ok()) { + return ""; + } + size_t len = 0; // includes the NUL terminator + if (nvs_get_str(handle.get(), key, nullptr, &len) != ESP_OK || len == 0) { + return ""; + } + std::string value(len, '\0'); + if (nvs_get_str(handle.get(), key, value.data(), &len) != ESP_OK) { + return ""; + } + value.resize(len - 1); // drop the NUL terminator + if (value.size() > maxLen) { + // FR-002 analogue: an over-long stored credential is shadowed by + // the factory (empty) state. Value intentionally not logged. + ESP_LOGW(TAG, "stored %s exceeds length limit, using default", key); + return ""; + } + return value; +} + +// --------------------------------------------------------------------------- +// IConfigStore +// --------------------------------------------------------------------------- + +float NvsConfigStore::getMoistureThresholdLow() const +{ + return getFloat(kKeyMoistLow, kDefaultMoistureThresholdLow, + kMoistureThresholdMin, kMoistureThresholdMax); +} + +bool NvsConfigStore::setMoistureThresholdLow(float percent) +{ + return setFloat(kKeyMoistLow, percent, kMoistureThresholdMin, + kMoistureThresholdMax); +} + +float NvsConfigStore::getMoistureThresholdHigh() const +{ + return getFloat(kKeyMoistHigh, kDefaultMoistureThresholdHigh, + kMoistureThresholdMin, kMoistureThresholdMax); +} + +bool NvsConfigStore::setMoistureThresholdHigh(float percent) +{ + return setFloat(kKeyMoistHigh, percent, kMoistureThresholdMin, + kMoistureThresholdMax); +} + +uint32_t NvsConfigStore::getWateringDurationS() const +{ + return getU32(kKeyWaterDur, kDefaultWateringDurationS, + kWateringDurationMinS, kWateringDurationMaxS); +} + +bool NvsConfigStore::setWateringDurationS(uint32_t seconds) +{ + return setU32(kKeyWaterDur, seconds, kWateringDurationMinS, + kWateringDurationMaxS); +} + +uint32_t NvsConfigStore::getMinWateringIntervalS() const +{ + return getU32(kKeySoakPause, kDefaultMinWateringIntervalS, + kMinWateringIntervalFloorS, kNoUpperBound); +} + +bool NvsConfigStore::setMinWateringIntervalS(uint32_t seconds) +{ + return setU32(kKeySoakPause, seconds, kMinWateringIntervalFloorS, + kNoUpperBound); +} + +bool NvsConfigStore::getWateringEnabled() const +{ + return getBool(kKeyWaterEn, kDefaultWateringEnabled); +} + +bool NvsConfigStore::setWateringEnabled(bool enabled) +{ + return setBool(kKeyWaterEn, enabled); +} + +uint32_t NvsConfigStore::getSensorReadIntervalMs() const +{ + return getU32(kKeyReadIv, kDefaultSensorReadIntervalMs, + kSensorReadIntervalFloorMs, kNoUpperBound); +} + +bool NvsConfigStore::setSensorReadIntervalMs(uint32_t ms) +{ + return setU32(kKeyReadIv, ms, kSensorReadIntervalFloorMs, kNoUpperBound); +} + +uint32_t NvsConfigStore::getDataLogIntervalMs() const +{ + return getU32(kKeyLogIv, kDefaultDataLogIntervalMs, + kDataLogIntervalFloorMs, kNoUpperBound); +} + +bool NvsConfigStore::setDataLogIntervalMs(uint32_t ms) +{ + return setU32(kKeyLogIv, ms, kDataLogIntervalFloorMs, kNoUpperBound); +} + +std::string NvsConfigStore::getWifiSsid() const +{ + return getString(kKeyWifiSsid, kWifiSsidMaxLen); +} + +std::string NvsConfigStore::getWifiPassword() const +{ + return getString(kKeyWifiPass, kWifiPasswordMaxLen); +} + +bool NvsConfigStore::setWifiCredentials(const std::string& ssid, + const std::string& password) +{ + if (ssid.size() > kWifiSsidMaxLen || + password.size() > kWifiPasswordMaxLen) { + // Rejected without logging anything about the input (FR-004). + return false; + } + NvsHandleGuard handle(kNamespace, NVS_READWRITE); + if (!handle.ok()) { + ESP_LOGE(TAG, "nvs_open for wifi credentials failed: %s", + esp_err_to_name(handle.error())); + return false; + } + esp_err_t err = nvs_set_str(handle.get(), kKeyWifiSsid, ssid.c_str()); + if (err == ESP_OK) { + err = nvs_set_str(handle.get(), kKeyWifiPass, password.c_str()); + } + if (err == ESP_OK) { + err = nvs_commit(handle.get()); + } + if (err != ESP_OK) { + ESP_LOGE(TAG, "persisting wifi credentials failed: %s", + esp_err_to_name(err)); + return false; + } + return true; +} + +bool NvsConfigStore::clearWifiCredentials() +{ + NvsHandleGuard handle(kNamespace, NVS_READWRITE); + if (!handle.ok()) { + // Namespace never created = already in the factory (empty) state. + if (handle.error() == ESP_ERR_NVS_NOT_FOUND) { + return true; + } + ESP_LOGE(TAG, "nvs_open for wifi credentials failed: %s", + esp_err_to_name(handle.error())); + return false; + } + esp_err_t err = nvs_erase_key(handle.get(), kKeyWifiSsid); + if (err == ESP_ERR_NVS_NOT_FOUND) { + err = ESP_OK; // already absent + } + if (err == ESP_OK) { + err = nvs_erase_key(handle.get(), kKeyWifiPass); + if (err == ESP_ERR_NVS_NOT_FOUND) { + err = ESP_OK; + } + } + if (err == ESP_OK) { + err = nvs_commit(handle.get()); + } + if (err != ESP_OK) { + ESP_LOGE(TAG, "clearing wifi credentials failed: %s", + esp_err_to_name(err)); + return false; + } + return true; +} + +bool NvsConfigStore::factoryReset() +{ + // Standard factory-reset sequence (research.md D5/D8). The erase call + // deinitializes an initialized partition first, so no handle is open + // when the erase happens (handles here are per-operation anyway). + esp_err_t err = nvs_flash_erase_partition(NVS_DEFAULT_PART_NAME); + if (err != ESP_OK) { + ESP_LOGE(TAG, "nvs partition erase failed: %s", esp_err_to_name(err)); + return false; + } + err = nvs_flash_init(); + if (err != ESP_OK) { + ESP_LOGE(TAG, "nvs re-init after factory reset failed: %s", + esp_err_to_name(err)); + return false; + } + ESP_LOGI(TAG, "factory reset: all configuration restored to defaults"); + return true; +} diff --git a/firmware/components/storage/src/StorageMount.cpp b/firmware/components/storage/src/StorageMount.cpp new file mode 100644 index 0000000..8677377 --- /dev/null +++ b/firmware/components/storage/src/StorageMount.cpp @@ -0,0 +1,44 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file StorageMount.cpp + * @brief Target-only littlefs mount/format/stats wrapper implementation. + * + * See StorageMount.h. Excluded from the linux preview target build + * (storage component CMakeLists) — esp_littlefs has no linux port. + */ + +#include "storage/StorageMount.h" + +#include +#include + +#include "esp_littlefs.h" + +esp_err_t StorageMount::mount(void) +{ + // Member-wise init (not designated init): esp_vfs_littlefs_conf_t + // carries Kconfig-dependent fields; zero-init + explicit members is + // robust against field-order differences across component versions. + esp_vfs_littlefs_conf_t conf = {}; + conf.base_path = kBasePath; + conf.partition_label = kPartitionLabel; + conf.format_if_mount_failed = true; + conf.read_only = false; + + return esp_vfs_littlefs_register(&conf); +} + +LittleFsDataStorage::StatsProvider StorageMount::statsProvider(void) +{ + return [](uint32_t& totalBytes, uint32_t& usedBytes) { + std::size_t total = 0; + std::size_t used = 0; + if (esp_littlefs_info(kPartitionLabel, &total, &used) != ESP_OK) { + return false; + } + totalBytes = static_cast(total); + usedBytes = static_cast(used); + return true; + }; +} diff --git a/firmware/main/CMakeLists.txt b/firmware/main/CMakeLists.txt index 5c53f49..6aac901 100644 --- a/firmware/main/CMakeLists.txt +++ b/firmware/main/CMakeLists.txt @@ -2,4 +2,16 @@ idf_component_register( SRCS "app_main.cpp" "diag_console.cpp" PRIV_REQUIRES board esp_driver_gpio esp_app_format actuators interfaces console esp_timer + storage nvs_flash ) + +# Build-time littlefs image of the committed seed directory +# (firmware/storage_image), flashed to the `storage` partition — research.md +# D1. The function comes from the managed joltwallet/littlefs component's +# project_include.cmake and registers an ALL target, so build/storage.bin is +# produced on every `idf.py build` (CI verifies the file exists); +# FLASH_IN_PROJECT attaches the image to `idf.py flash` for a deterministic +# fresh-flash filesystem. Canonical placement per the component's example: +# a component CMakeLists, after idf_component_register (the relative path +# resolves against this directory). +littlefs_create_partition_image(storage ../storage_image FLASH_IN_PROJECT) diff --git a/firmware/main/app_main.cpp b/firmware/main/app_main.cpp index 666f785..687addc 100644 --- a/firmware/main/app_main.cpp +++ b/firmware/main/app_main.cpp @@ -25,10 +25,16 @@ #include "driver/gpio.h" #include "freertos/FreeRTOS.h" #include "freertos/task.h" +#include "nvs_flash.h" #include "actuators/EspTimeProvider.h" #include "actuators/GpioWaterPump.h" #include "actuators/LockedWaterPump.h" +#include "storage/LittleFsDataStorage.h" +#include "storage/LockedConfigStore.h" +#include "storage/LockedDataStorage.h" +#include "storage/NvsConfigStore.h" +#include "storage/StorageMount.h" #include "diag_console.h" @@ -116,8 +122,57 @@ extern "C" void app_main(void) abort(); } + // Persistent storage. Not safety-critical: any failure below is logged + // and the system keeps running — config reads fall back to compiled-in + // defaults and data-storage operations fail gracefully (the pump + // safety loop never depends on storage). + // + // NVS init with the standard recovery: a full page-less partition or a + // newer NVS format version is erased and re-initialized (factory-state + // config, by design old-or-new never torn — research.md D5). + esp_err_t nvs_err = nvs_flash_init(); + if (nvs_err == ESP_ERR_NVS_NO_FREE_PAGES || + nvs_err == ESP_ERR_NVS_NEW_VERSION_FOUND) { + ESP_LOGW(TAG, "NVS needs erase (%s), recovering", + esp_err_to_name(nvs_err)); + nvs_err = nvs_flash_erase(); + if (nvs_err == ESP_OK) { + nvs_err = nvs_flash_init(); + } + } + if (nvs_err != ESP_OK) { + ESP_LOGE(TAG, "NVS init failed: %s (config falls back to defaults)", + esp_err_to_name(nvs_err)); + } + + // littlefs mount-or-format of the `storage` partition at /storage + // (FR-007; a corrupted filesystem is reformatted, never bricks). + const esp_err_t mount_err = StorageMount::mount(); + if (mount_err != ESP_OK) { + ESP_LOGE(TAG, "storage mount failed: %s (data storage unavailable)", + esp_err_to_name(mount_err)); + } + + // Storage instances — function-local statics after pumps_force_off() + // (boot fail-safe rule), wrapped in the mutex-serializing decorators: + // accessed from this task and the console REPL task, so EVERY access + // from here on goes through the wrappers (FR-013). + static NvsConfigStore config_store; + static LittleFsDataStorage data_storage(StorageMount::kBasePath, + StorageMount::statsProvider()); + static LockedConfigStore config(config_store); + static LockedDataStorage storage(data_storage); + + // One-line usage report (parity: storage usage in the serial status + // block; FR-008). + const StorageStats stats = storage.getStorageStats(); + ESP_LOGI(TAG, "Storage: %lu/%lu KiB used", + static_cast(stats.usedBytes / 1024), + static_cast(stats.totalBytes / 1024)); + // Serial diagnostic REPL (rig testing; contracts/serial-diagnostic.md). diag_console_register_pumps(plant, reservoir); + diag_console_register_storage(config, storage); esp_err_t err = diag_console_start(); if (err != ESP_OK) { // Console is a diagnostic aid, not a safety function: log and keep diff --git a/firmware/main/diag_console.cpp b/firmware/main/diag_console.cpp index 42a55ef..c1f3c39 100644 --- a/firmware/main/diag_console.cpp +++ b/firmware/main/diag_console.cpp @@ -12,6 +12,21 @@ * pump status * pump status # both pumps * + * Storage commands (HIL verification path for feature 003, quickstart.md + * steps 2-4; thin wrappers — every handler is a direct interface call): + * + * config get # all items; credentials shown + * # only as (un)configured (FR-004) + * config set # item = NVS key (data-model.md) + * config wifi # values never echoed (FR-004) + * config wifi-clear + * config factory-reset + * storage stats + * storage log # reading at the current epoch + * storage query [t0 t1] # count + newest records in range + * storage event # category = u8 (1..255) + * storage events [n] # newest-first, default 10 + * * Handler exit codes follow the esp_console convention: 0 on OK, 1 on ERR. * * State is plain pointers/PODs set from app_main — no non-trivial static @@ -20,12 +35,18 @@ #include "diag_console.h" +#include +#include #include #include #include +#include +#include #include "esp_console.h" +#include "interfaces/IConfigStore.h" +#include "interfaces/IDataStorage.h" #include "interfaces/IWaterPump.h" namespace { @@ -44,6 +65,12 @@ PumpSlot s_slots[2] = { {"reservoir", nullptr, 0}, }; +// Storage instances (set from app_main; expected to be the Locked* +// decorators — the handlers run on the REPL task, FR-013). Trivially +// initialized pointers, same rule as s_slots. +IConfigStore *s_config = nullptr; +IDataStorage *s_storage = nullptr; + const char *stop_reason_str(StopReason reason) { switch (reason) { @@ -157,6 +184,252 @@ int pump_cmd(int argc, char **argv) return print_usage(); } +// --- config / storage commands (feature 003 HIL verification path) ------ + +/// Strict decimal u32 parse; false on garbage/overflow. +bool parse_u32(const char *arg, uint32_t &out) +{ + char *end = nullptr; + const unsigned long value = strtoul(arg, &end, 10); + if (end == arg || *end != '\0' || value > UINT32_MAX) { + return false; + } + out = static_cast(value); + return true; +} + +/// Strict float parse; false on garbage (range checks are the store's job). +bool parse_float(const char *arg, float &out) +{ + char *end = nullptr; + out = strtof(arg, &end); + return end != arg && *end == '\0'; +} + +void print_config(const IConfigStore &config) +{ + printf("moist_low=%.2f %%\n", + static_cast(config.getMoistureThresholdLow())); + printf("moist_high=%.2f %%\n", + static_cast(config.getMoistureThresholdHigh())); + printf("water_dur=%lu s\n", + static_cast(config.getWateringDurationS())); + printf("soak_pause=%lu s\n", + static_cast(config.getMinWateringIntervalS())); + printf("water_en=%d\n", config.getWateringEnabled() ? 1 : 0); + printf("read_iv=%lu ms\n", + static_cast(config.getSensorReadIntervalMs())); + printf("log_iv=%lu ms\n", + static_cast(config.getDataLogIntervalMs())); + // Credential VALUES never appear in diagnostic output (FR-004). + printf("wifi=%s\n", + config.getWifiSsid().empty() ? "unconfigured" : "configured"); +} + +int print_config_usage(void) +{ + printf("ERR usage: config |wifi " + "|wifi-clear|factory-reset>\n"); + return 1; +} + +/// `config set `: items are the NVS keys (data-model.md); +/// the store validates and persists — rejection means out-of-range input +/// or a persistence failure. +int cmd_config_set(IConfigStore &config, const char *item, const char *value) +{ + bool ok = false; + if (strcmp(item, "moist_low") == 0 || strcmp(item, "moist_high") == 0) { + float parsed = 0.0f; + if (!parse_float(value, parsed)) { + printf("ERR %s: not a number\n", item); + return 1; + } + ok = (strcmp(item, "moist_low") == 0) + ? config.setMoistureThresholdLow(parsed) + : config.setMoistureThresholdHigh(parsed); + } else if (strcmp(item, "water_en") == 0) { + if (strcmp(value, "0") != 0 && strcmp(value, "1") != 0) { + printf("ERR water_en: value must be 0 or 1\n"); + return 1; + } + ok = config.setWateringEnabled(strcmp(value, "1") == 0); + } else { + uint32_t parsed = 0; + if (!parse_u32(value, parsed)) { + printf("ERR %s: not an unsigned integer\n", item); + return 1; + } + if (strcmp(item, "water_dur") == 0) { + ok = config.setWateringDurationS(parsed); + } else if (strcmp(item, "soak_pause") == 0) { + ok = config.setMinWateringIntervalS(parsed); + } else if (strcmp(item, "read_iv") == 0) { + ok = config.setSensorReadIntervalMs(parsed); + } else if (strcmp(item, "log_iv") == 0) { + ok = config.setDataLogIntervalMs(parsed); + } else { + printf("ERR unknown item '%s' (moist_low moist_high water_dur " + "soak_pause water_en read_iv log_iv)\n", + item); + return 1; + } + } + if (!ok) { + printf("ERR %s rejected (out of range or storage failure)\n", item); + return 1; + } + printf("OK %s=%s\n", item, value); + return 0; +} + +int config_cmd(int argc, char **argv) +{ + if (s_config == nullptr) { + printf("ERR config store not available\n"); + return 1; + } + if (argc == 2 && strcmp(argv[1], "get") == 0) { + print_config(*s_config); + return 0; + } + if (argc == 4 && strcmp(argv[1], "set") == 0) { + return cmd_config_set(*s_config, argv[2], argv[3]); + } + if (argc == 4 && strcmp(argv[1], "wifi") == 0) { + // Never echo the values back (FR-004). + if (!s_config->setWifiCredentials(argv[2], argv[3])) { + printf("ERR credentials rejected (ssid <= 32, password <= 64 " + "bytes, or storage failure)\n"); + return 1; + } + printf("OK wifi credentials stored\n"); + return 0; + } + if (argc == 2 && strcmp(argv[1], "wifi-clear") == 0) { + if (!s_config->clearWifiCredentials()) { + printf("ERR wifi clear failed\n"); + return 1; + } + printf("OK wifi credentials cleared\n"); + return 0; + } + if (argc == 2 && strcmp(argv[1], "factory-reset") == 0) { + if (!s_config->factoryReset()) { + printf("ERR factory reset failed\n"); + return 1; + } + printf("OK factory defaults restored\n"); + return 0; + } + return print_config_usage(); +} + +int print_storage_usage(void) +{ + printf("ERR usage: storage |query " + "[t0 t1]|event |events [n]>\n"); + return 1; +} + +int storage_cmd(int argc, char **argv) +{ + if (s_storage == nullptr) { + printf("ERR data storage not available\n"); + return 1; + } + if (argc == 2 && strcmp(argv[1], "stats") == 0) { + const StorageStats stats = s_storage->getStorageStats(); + printf("OK total=%lu B used=%lu B (%lu%%)\n", + static_cast(stats.totalBytes), + static_cast(stats.usedBytes), + static_cast( + stats.totalBytes != 0 + ? (100ULL * stats.usedBytes) / stats.totalBytes + : 0)); + return 0; + } + if (argc == 4 && strcmp(argv[1], "log") == 0) { + float value = 0.0f; + if (!parse_float(argv[3], value)) { + printf("ERR value: not a number\n"); + return 1; + } + const uint32_t epoch = static_cast(time(nullptr)); + if (!s_storage->storeSensorReading(argv[2], epoch, value)) { + printf("ERR reading rejected (metric cap or storage failure)\n"); + return 1; + } + printf("OK %s %lu %.3f\n", argv[2], static_cast(epoch), + static_cast(value)); + return 0; + } + if ((argc == 3 || argc == 5) && strcmp(argv[1], "query") == 0) { + uint32_t t0 = 0; + uint32_t t1 = UINT32_MAX; + if (argc == 5 && + (!parse_u32(argv[3], t0) || !parse_u32(argv[4], t1))) { + printf("ERR t0/t1: not unsigned integers\n"); + return 1; + } + const std::vector readings = + s_storage->getSensorReadings(argv[2], t0, t1); + printf("OK %u readings\n", static_cast(readings.size())); + // Bounded output: the newest 10 are enough for the HIL checks. + const std::size_t first = + readings.size() > 10 ? readings.size() - 10 : 0; + for (std::size_t i = first; i < readings.size(); ++i) { + printf("%lu %.3f\n", + static_cast(readings[i].epoch), + static_cast(readings[i].value)); + } + return 0; + } + if (argc >= 4 && strcmp(argv[1], "event") == 0) { + uint32_t category = 0; + if (!parse_u32(argv[2], category) || category == 0 || + category > UINT8_MAX) { + printf("ERR category must be 1..255\n"); + return 1; + } + // Re-join the detail words (the console splits on spaces). + std::string detail; + for (int i = 3; i < argc; ++i) { + if (!detail.empty()) { + detail += ' '; + } + detail += argv[i]; + } + const uint32_t epoch = static_cast(time(nullptr)); + if (!s_storage->storeEvent(epoch, static_cast(category), + detail)) { + printf("ERR event rejected (storage failure)\n"); + return 1; + } + printf("OK event %lu cat=%lu\n", static_cast(epoch), + static_cast(category)); + return 0; + } + if ((argc == 2 || argc == 3) && strcmp(argv[1], "events") == 0) { + uint32_t maxCount = 10; + if (argc == 3 && (!parse_u32(argv[2], maxCount) || maxCount == 0)) { + printf("ERR n must be a positive integer\n"); + return 1; + } + const std::vector events = s_storage->getEvents(maxCount); + printf("OK %u events (newest first)\n", + static_cast(events.size())); + for (const EventRecord &event : events) { + printf("%lu cat=%u %s\n", + static_cast(event.epoch), + static_cast(event.category), + event.detail.c_str()); + } + return 0; + } + return print_storage_usage(); +} + } // namespace void diag_console_register_pumps(IWaterPump& plant, IWaterPump& reservoir) @@ -165,6 +438,12 @@ void diag_console_register_pumps(IWaterPump& plant, IWaterPump& reservoir) s_slots[1].pump = &reservoir; } +void diag_console_register_storage(IConfigStore& config, IDataStorage& storage) +{ + s_config = &config; + s_storage = &storage; +} + esp_err_t diag_console_start(void) { esp_console_repl_t *repl = nullptr; @@ -195,5 +474,35 @@ esp_err_t diag_console_start(void) return err; } + const esp_console_cmd_t cmd_config = { + .command = "config", + .help = "config |wifi " + "|wifi-clear|factory-reset>", + .hint = nullptr, + .func = &config_cmd, + .argtable = nullptr, + .func_w_context = nullptr, + .context = nullptr, + }; + err = esp_console_cmd_register(&cmd_config); + if (err != ESP_OK) { + return err; + } + + const esp_console_cmd_t cmd_storage = { + .command = "storage", + .help = "storage |query [t0 t1]" + "|event |events [n]>", + .hint = nullptr, + .func = &storage_cmd, + .argtable = nullptr, + .func_w_context = nullptr, + .context = nullptr, + }; + err = esp_console_cmd_register(&cmd_storage); + if (err != ESP_OK) { + return err; + } + return esp_console_start_repl(repl); } diff --git a/firmware/main/diag_console.h b/firmware/main/diag_console.h index 2505d27..fe6343a 100644 --- a/firmware/main/diag_console.h +++ b/firmware/main/diag_console.h @@ -12,6 +12,8 @@ #define WATERINGSYSTEM_MAIN_DIAG_CONSOLE_H #include "esp_err.h" +#include "interfaces/IConfigStore.h" +#include "interfaces/IDataStorage.h" #include "interfaces/IWaterPump.h" /** @@ -23,7 +25,18 @@ void diag_console_register_pumps(IWaterPump& plant, IWaterPump& reservoir); /** - * @brief Start the UART REPL (prompt "ws>") and register the pump command. + * @brief Register the storage instances the `config`/`storage` commands + * operate on (HIL verification path for feature 003). + * + * Pass the Locked* decorators, never the raw stores — the console handlers + * run on the REPL task, concurrently with the main task (FR-013). Must be + * called before diag_console_start(); plain pointer registration. + */ +void diag_console_register_storage(IConfigStore& config, + IDataStorage& storage); + +/** + * @brief Start the UART REPL (prompt "ws>") and register the commands. * * @return ESP_OK on success, the failing esp_err_t otherwise. */ diff --git a/firmware/storage_image/README b/firmware/storage_image/README new file mode 100644 index 0000000..de21d47 --- /dev/null +++ b/firmware/storage_image/README @@ -0,0 +1,13 @@ +This directory is the committed seed for the littlefs image of the +`storage` partition (0x310000, 960 KiB — see firmware/partitions.csv). + +`littlefs_create_partition_image(storage ../storage_image FLASH_IN_PROJECT)` +in firmware/main/CMakeLists.txt (user story 4 of feature +003-nvs-littlefs-storage) packs the contents of this directory into +build/storage.bin on every build, so a fresh flash boots with a +deterministic, already-formatted filesystem (research.md D1; CI verifies +the file exists). + +The firmware creates its own directories (/storage/hist/, /storage/events/) +at runtime — no seed content is required beyond this README, which also +ends up in the image and documents its origin. diff --git a/firmware/test_apps/host/CMakeLists.txt b/firmware/test_apps/host/CMakeLists.txt index 8b768c1..2f27ae0 100644 --- a/firmware/test_apps/host/CMakeLists.txt +++ b/firmware/test_apps/host/CMakeLists.txt @@ -1,11 +1,16 @@ -# Host test app for the pump actuator layer (IDF linux preview target). +# Host test app for the firmware components (IDF linux preview target). +# Suites: pump actuator layer, config store (real linux-target NVS), +# data storage. # # Built and run with no ESP32 attached: # idf.py --preview set-target linux && idf.py build && ./build/pump_host_tests.elf # The executable's exit code equals the Unity failure count (CI gate). +# The project (and elf) name predates the storage suites and is kept so the +# CI host-test job keeps working unchanged. cmake_minimum_required(VERSION 3.22) -# Reuse the firmware components (interfaces, actuators) without copying. +# Reuse the firmware components (interfaces, actuators, storage) without +# copying. set(EXTRA_COMPONENT_DIRS "${CMAKE_CURRENT_LIST_DIR}/../../components") # Component isolation: only main + its requirements are built, so the diff --git a/firmware/test_apps/host/main/CMakeLists.txt b/firmware/test_apps/host/main/CMakeLists.txt index 4a6fa99..7785cc2 100644 --- a/firmware/test_apps/host/main/CMakeLists.txt +++ b/firmware/test_apps/host/main/CMakeLists.txt @@ -1,4 +1,7 @@ idf_component_register( - SRCS "test_water_pump.cpp" - REQUIRES unity actuators interfaces + SRCS "test_main.cpp" + "test_water_pump.cpp" + "test_config_store.cpp" + "test_data_storage.cpp" + REQUIRES unity actuators interfaces storage nvs_flash ) diff --git a/firmware/test_apps/host/main/test_config_store.cpp b/firmware/test_apps/host/main/test_config_store.cpp new file mode 100644 index 0000000..7f9e9db --- /dev/null +++ b/firmware/test_apps/host/main/test_config_store.cpp @@ -0,0 +1,572 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file test_config_store.cpp + * @brief Host tests for the IConfigStore contract (linux preview target). + * + * NvsConfigStore runs against the REAL nvs_flash implementation: the linux + * esp_partition emulation provides the `nvs` partition from + * partitions_host.csv (research.md D3). Every NVS test starts from an + * erased partition (per-test isolation). MockConfigStore is held to the + * same contract invariants (FR-012). + * + * Coverage maps to specs/003-nvs-littlefs-storage/contracts/IConfigStore.md + * and the data-model.md item table (tasks T009-T012). + */ + +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "unity.h" + +#include "nvs.h" +#include "nvs_flash.h" + +#include "interfaces/IConfigStore.h" +#include "storage/LockedConfigStore.h" +#include "storage/NvsConfigStore.h" +#include "storage/testing/MockConfigStore.h" + +namespace { + +constexpr const char* kNamespace = "wscfg"; + +/// Erase + re-init the default NVS partition (per-test isolation). +/// nvs_flash_erase() deinitializes an initialized partition before erasing. +void resetNvs() +{ + TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_erase()); + TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_init()); +} + +uint32_t floatBits(float value) +{ + uint32_t bits = 0; + std::memcpy(&bits, &value, sizeof(bits)); + return bits; +} + +// Raw writes bypass the store's validation to plant out-of-range STORED +// values (US1 scenario 3: shadowing, not write rejection). +void rawSetU32(const char* key, uint32_t value) +{ + nvs_handle_t handle = 0; + TEST_ASSERT_EQUAL(ESP_OK, nvs_open(kNamespace, NVS_READWRITE, &handle)); + TEST_ASSERT_EQUAL(ESP_OK, nvs_set_u32(handle, key, value)); + TEST_ASSERT_EQUAL(ESP_OK, nvs_commit(handle)); + nvs_close(handle); +} + +void rawSetU8(const char* key, uint8_t value) +{ + nvs_handle_t handle = 0; + TEST_ASSERT_EQUAL(ESP_OK, nvs_open(kNamespace, NVS_READWRITE, &handle)); + TEST_ASSERT_EQUAL(ESP_OK, nvs_set_u8(handle, key, value)); + TEST_ASSERT_EQUAL(ESP_OK, nvs_commit(handle)); + nvs_close(handle); +} + +void rawSetStr(const char* key, const std::string& value) +{ + nvs_handle_t handle = 0; + TEST_ASSERT_EQUAL(ESP_OK, nvs_open(kNamespace, NVS_READWRITE, &handle)); + TEST_ASSERT_EQUAL(ESP_OK, nvs_set_str(handle, key, value.c_str())); + TEST_ASSERT_EQUAL(ESP_OK, nvs_commit(handle)); + nvs_close(handle); +} + +/// Assert that every item reads its compiled-in factory default. +void assertAllDefaults(const IConfigStore& store) +{ + TEST_ASSERT_EQUAL_FLOAT(IConfigStore::kDefaultMoistureThresholdLow, + store.getMoistureThresholdLow()); + TEST_ASSERT_EQUAL_FLOAT(IConfigStore::kDefaultMoistureThresholdHigh, + store.getMoistureThresholdHigh()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultWateringDurationS, + store.getWateringDurationS()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultMinWateringIntervalS, + store.getMinWateringIntervalS()); + TEST_ASSERT_EQUAL(IConfigStore::kDefaultWateringEnabled, + store.getWateringEnabled()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultSensorReadIntervalMs, + store.getSensorReadIntervalMs()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultDataLogIntervalMs, + store.getDataLogIntervalMs()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiSsid().c_str()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiPassword().c_str()); +} + +/// Write one non-default valid value per item. +void setAllNonDefaults(IConfigStore& store) +{ + TEST_ASSERT_TRUE(store.setMoistureThresholdLow(33.5f)); + TEST_ASSERT_TRUE(store.setMoistureThresholdHigh(66.25f)); + TEST_ASSERT_TRUE(store.setWateringDurationS(45)); + TEST_ASSERT_TRUE(store.setMinWateringIntervalS(600)); + TEST_ASSERT_TRUE(store.setWateringEnabled(false)); + TEST_ASSERT_TRUE(store.setSensorReadIntervalMs(2000)); + TEST_ASSERT_TRUE(store.setDataLogIntervalMs(120000)); + TEST_ASSERT_TRUE(store.setWifiCredentials("greenhouse", "round-trip-pw")); +} + +/// Assert the values written by setAllNonDefaults(). +void assertAllNonDefaults(const IConfigStore& store) +{ + TEST_ASSERT_EQUAL_FLOAT(33.5f, store.getMoistureThresholdLow()); + TEST_ASSERT_EQUAL_FLOAT(66.25f, store.getMoistureThresholdHigh()); + TEST_ASSERT_EQUAL_UINT32(45, store.getWateringDurationS()); + TEST_ASSERT_EQUAL_UINT32(600, store.getMinWateringIntervalS()); + TEST_ASSERT_FALSE(store.getWateringEnabled()); + TEST_ASSERT_EQUAL_UINT32(2000, store.getSensorReadIntervalMs()); + TEST_ASSERT_EQUAL_UINT32(120000, store.getDataLogIntervalMs()); + TEST_ASSERT_EQUAL_STRING("greenhouse", store.getWifiSsid().c_str()); + TEST_ASSERT_EQUAL_STRING("round-trip-pw", + store.getWifiPassword().c_str()); +} + +} // namespace + +// --------------------------------------------------------------------------- +// T009 — erased NVS: every item reads its factory default (FR-001/FR-002) +// --------------------------------------------------------------------------- +static void test_defaults_on_erased_nvs(void) +{ + resetNvs(); + NvsConfigStore store; + assertAllDefaults(store); +} + +// --------------------------------------------------------------------------- +// T010 — set/get round-trip per item (FR-003) +// --------------------------------------------------------------------------- +static void test_round_trip_all_items(void) +{ + resetNvs(); + NvsConfigStore store; + setAllNonDefaults(store); + assertAllNonDefaults(store); +} + +// --------------------------------------------------------------------------- +// T010 — values survive an NVS deinit/init cycle (simulated restart) and +// are visible to a fresh store instance (US1 scenario 2) +// --------------------------------------------------------------------------- +static void test_persistence_across_reinit(void) +{ + resetNvs(); + { + NvsConfigStore store; + setAllNonDefaults(store); + } + + TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_deinit()); + TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_init()); + + NvsConfigStore store; + assertAllNonDefaults(store); +} + +// --------------------------------------------------------------------------- +// T011 — out-of-range writes rejected at every documented bound (FR-003) +// --------------------------------------------------------------------------- +static void test_set_rejects_out_of_range(void) +{ + resetNvs(); + NvsConfigStore store; + + // Moisture thresholds: 0–100 %, NaN never accepted. + TEST_ASSERT_FALSE(store.setMoistureThresholdLow(-0.5f)); + TEST_ASSERT_FALSE(store.setMoistureThresholdLow(100.5f)); + TEST_ASSERT_FALSE(store.setMoistureThresholdLow(NAN)); + TEST_ASSERT_FALSE(store.setMoistureThresholdHigh(-0.5f)); + TEST_ASSERT_FALSE(store.setMoistureThresholdHigh(100.5f)); + TEST_ASSERT_FALSE(store.setMoistureThresholdHigh(NAN)); + + // Watering duration: 1–300 s. + TEST_ASSERT_FALSE(store.setWateringDurationS(0)); + TEST_ASSERT_FALSE(store.setWateringDurationS(301)); + + // Soak pause: >= 1 s. + TEST_ASSERT_FALSE(store.setMinWateringIntervalS(0)); + + // Sensor read interval: >= 1000 ms. + TEST_ASSERT_FALSE(store.setSensorReadIntervalMs(999)); + + // Data log interval: >= 60000 ms. + TEST_ASSERT_FALSE(store.setDataLogIntervalMs(59999)); + + // Credentials: SSID <= 32 bytes, password <= 64 bytes. + const std::string longSsid(33, 's'); + const std::string longPass(65, 'p'); + TEST_ASSERT_FALSE(store.setWifiCredentials(longSsid, "ok")); + TEST_ASSERT_FALSE(store.setWifiCredentials("ok", longPass)); + + // Nothing was stored: every item still reads its default. + assertAllDefaults(store); +} + +// --------------------------------------------------------------------------- +// T011 — boundary values are in range and accepted +// --------------------------------------------------------------------------- +static void test_boundary_values_accepted(void) +{ + resetNvs(); + NvsConfigStore store; + + TEST_ASSERT_TRUE(store.setMoistureThresholdLow(0.0f)); + TEST_ASSERT_EQUAL_FLOAT(0.0f, store.getMoistureThresholdLow()); + TEST_ASSERT_TRUE(store.setMoistureThresholdHigh(100.0f)); + TEST_ASSERT_EQUAL_FLOAT(100.0f, store.getMoistureThresholdHigh()); + + TEST_ASSERT_TRUE(store.setWateringDurationS(1)); + TEST_ASSERT_EQUAL_UINT32(1, store.getWateringDurationS()); + TEST_ASSERT_TRUE(store.setWateringDurationS(300)); + TEST_ASSERT_EQUAL_UINT32(300, store.getWateringDurationS()); + + TEST_ASSERT_TRUE(store.setMinWateringIntervalS(1)); + TEST_ASSERT_EQUAL_UINT32(1, store.getMinWateringIntervalS()); + + TEST_ASSERT_TRUE(store.setSensorReadIntervalMs(1000)); + TEST_ASSERT_EQUAL_UINT32(1000, store.getSensorReadIntervalMs()); + + TEST_ASSERT_TRUE(store.setDataLogIntervalMs(60000)); + TEST_ASSERT_EQUAL_UINT32(60000, store.getDataLogIntervalMs()); + + // Exact-length credentials are accepted. + const std::string maxSsid(32, 's'); + const std::string maxPass(64, 'p'); + TEST_ASSERT_TRUE(store.setWifiCredentials(maxSsid, maxPass)); + TEST_ASSERT_EQUAL_STRING(maxSsid.c_str(), store.getWifiSsid().c_str()); + TEST_ASSERT_EQUAL_STRING(maxPass.c_str(), + store.getWifiPassword().c_str()); +} + +// --------------------------------------------------------------------------- +// T011 — a rejected write leaves the previously stored value untouched +// --------------------------------------------------------------------------- +static void test_rejected_write_leaves_stored_value(void) +{ + resetNvs(); + NvsConfigStore store; + + TEST_ASSERT_TRUE(store.setMoistureThresholdLow(40.0f)); + TEST_ASSERT_FALSE(store.setMoistureThresholdLow(150.0f)); + TEST_ASSERT_EQUAL_FLOAT(40.0f, store.getMoistureThresholdLow()); + + TEST_ASSERT_TRUE(store.setWateringDurationS(60)); + TEST_ASSERT_FALSE(store.setWateringDurationS(0)); + TEST_ASSERT_EQUAL_UINT32(60, store.getWateringDurationS()); + + TEST_ASSERT_TRUE(store.setWifiCredentials("kept-ssid", "kept-pass")); + TEST_ASSERT_FALSE(store.setWifiCredentials(std::string(33, 'x'), "new")); + TEST_ASSERT_EQUAL_STRING("kept-ssid", store.getWifiSsid().c_str()); + TEST_ASSERT_EQUAL_STRING("kept-pass", store.getWifiPassword().c_str()); +} + +// --------------------------------------------------------------------------- +// T011 — out-of-range STORED values are shadowed by the defaults +// (US1 scenario 3, FR-002); a later valid write replaces the invalid entry +// --------------------------------------------------------------------------- +static void test_out_of_range_stored_values_shadowed(void) +{ + resetNvs(); + + rawSetU32("moist_low", floatBits(150.0f)); // > 100 % + rawSetU32("moist_high", floatBits(-1.0f)); // < 0 % + rawSetU32("water_dur", 301); // > 300 s + rawSetU32("soak_pause", 0); // < 1 s + rawSetU8("water_en", 7); // neither 0 nor 1 + rawSetU32("read_iv", 999); // < 1000 ms + rawSetU32("log_iv", 59999); // < 60000 ms + rawSetStr("wifi_ssid", std::string(33, 'a')); // > 32 bytes + + NvsConfigStore store; + TEST_ASSERT_EQUAL_FLOAT(IConfigStore::kDefaultMoistureThresholdLow, + store.getMoistureThresholdLow()); + TEST_ASSERT_EQUAL_FLOAT(IConfigStore::kDefaultMoistureThresholdHigh, + store.getMoistureThresholdHigh()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultWateringDurationS, + store.getWateringDurationS()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultMinWateringIntervalS, + store.getMinWateringIntervalS()); + TEST_ASSERT_EQUAL(IConfigStore::kDefaultWateringEnabled, + store.getWateringEnabled()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultSensorReadIntervalMs, + store.getSensorReadIntervalMs()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultDataLogIntervalMs, + store.getDataLogIntervalMs()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiSsid().c_str()); + + // A NaN bit pattern is also shadowed, never returned. + rawSetU32("moist_low", floatBits(NAN)); + TEST_ASSERT_EQUAL_FLOAT(IConfigStore::kDefaultMoistureThresholdLow, + store.getMoistureThresholdLow()); + + // The invalid entry stays in place until the next VALID write replaces + // it (data-model rule). + TEST_ASSERT_TRUE(store.setMoistureThresholdLow(42.0f)); + TEST_ASSERT_EQUAL_FLOAT(42.0f, store.getMoistureThresholdLow()); +} + +// --------------------------------------------------------------------------- +// T012 — factory reset restores every default and removes credentials +// (FR-005, SC-003) +// --------------------------------------------------------------------------- +static void test_factory_reset_restores_defaults(void) +{ + resetNvs(); + NvsConfigStore store; + setAllNonDefaults(store); + + TEST_ASSERT_TRUE(store.factoryReset()); + + // The same instance stays usable; everything reads defaults again. + assertAllDefaults(store); + + // The store accepts new writes after the reset. + TEST_ASSERT_TRUE(store.setWateringDurationS(30)); + TEST_ASSERT_EQUAL_UINT32(30, store.getWateringDurationS()); +} + +// --------------------------------------------------------------------------- +// T012 — credential set/clear semantics (FR-004 storage side) +// --------------------------------------------------------------------------- +static void test_credential_set_and_clear(void) +{ + resetNvs(); + NvsConfigStore store; + + // Factory state: unconfigured = empty strings. + TEST_ASSERT_EQUAL_STRING("", store.getWifiSsid().c_str()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiPassword().c_str()); + + TEST_ASSERT_TRUE(store.setWifiCredentials("my-network", "my-password")); + TEST_ASSERT_EQUAL_STRING("my-network", store.getWifiSsid().c_str()); + TEST_ASSERT_EQUAL_STRING("my-password", store.getWifiPassword().c_str()); + + TEST_ASSERT_TRUE(store.clearWifiCredentials()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiSsid().c_str()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiPassword().c_str()); + + // Clearing the factory state is a successful no-op. + TEST_ASSERT_TRUE(store.clearWifiCredentials()); +} + +// --------------------------------------------------------------------------- +// T012 — credential values never appear in log/diagnostic output (FR-004). +// stdout+stderr are redirected to a file around the credential operations; +// the captured bytes must not contain the secret values. +// --------------------------------------------------------------------------- +static void test_credentials_never_logged(void) +{ + resetNvs(); + NvsConfigStore store; + + char path[] = "/tmp/ws_log_capture_XXXXXX"; + const int captureFd = mkstemp(path); + TEST_ASSERT_TRUE(captureFd >= 0); + + fflush(stdout); + fflush(stderr); + const int savedOut = dup(STDOUT_FILENO); + const int savedErr = dup(STDERR_FILENO); + TEST_ASSERT_TRUE(savedOut >= 0 && savedErr >= 0); + dup2(captureFd, STDOUT_FILENO); + dup2(captureFd, STDERR_FILENO); + + const std::string ssid = "SECRET-SSID-FOR-LOG-TEST"; + const std::string password = "secret-password-for-log-test"; + const bool setOk = store.setWifiCredentials(ssid, password); + const std::string readSsid = store.getWifiSsid(); + const std::string readPass = store.getWifiPassword(); + // An over-long rejection must not log the input either. + const bool rejected = store.setWifiCredentials(std::string(33, 'Z'), + password); + const bool clearOk = store.clearWifiCredentials(); + + fflush(stdout); + fflush(stderr); + dup2(savedOut, STDOUT_FILENO); + dup2(savedErr, STDERR_FILENO); + close(savedOut); + close(savedErr); + + // Assertions only after stdout is restored (Unity output must be seen). + TEST_ASSERT_TRUE(setOk); + TEST_ASSERT_EQUAL_STRING(ssid.c_str(), readSsid.c_str()); + TEST_ASSERT_EQUAL_STRING(password.c_str(), readPass.c_str()); + TEST_ASSERT_FALSE(rejected); + TEST_ASSERT_TRUE(clearOk); + + std::string captured; + TEST_ASSERT_TRUE(lseek(captureFd, 0, SEEK_SET) == 0); + char buf[256]; + ssize_t n = 0; + while ((n = read(captureFd, buf, sizeof(buf))) > 0) { + captured.append(buf, static_cast(n)); + } + close(captureFd); + unlink(path); + + TEST_ASSERT_TRUE(captured.find(ssid) == std::string::npos); + TEST_ASSERT_TRUE(captured.find(password) == std::string::npos); + TEST_ASSERT_TRUE(captured.find("ZZZZ") == std::string::npos); +} + +// --------------------------------------------------------------------------- +// T012 — MockConfigStore holds the same contract invariants (FR-012) +// --------------------------------------------------------------------------- +static void test_mock_defaults_roundtrip_rejection(void) +{ + MockConfigStore store; + + // Fresh mock = factory state. + assertAllDefaults(store); + + // Round-trip. + setAllNonDefaults(store); + assertAllNonDefaults(store); + + // Every documented bound rejects; stored values stay untouched. + TEST_ASSERT_FALSE(store.setMoistureThresholdLow(-0.5f)); + TEST_ASSERT_FALSE(store.setMoistureThresholdLow(100.5f)); + TEST_ASSERT_FALSE(store.setMoistureThresholdLow(NAN)); + TEST_ASSERT_FALSE(store.setMoistureThresholdHigh(100.5f)); + TEST_ASSERT_FALSE(store.setWateringDurationS(0)); + TEST_ASSERT_FALSE(store.setWateringDurationS(301)); + TEST_ASSERT_FALSE(store.setMinWateringIntervalS(0)); + TEST_ASSERT_FALSE(store.setSensorReadIntervalMs(999)); + TEST_ASSERT_FALSE(store.setDataLogIntervalMs(59999)); + TEST_ASSERT_FALSE(store.setWifiCredentials(std::string(33, 's'), "ok")); + TEST_ASSERT_FALSE(store.setWifiCredentials("ok", std::string(65, 'p'))); + assertAllNonDefaults(store); + + // Instrumentation counted the rejections. + TEST_ASSERT_EQUAL(11, store.rejectedWrites); +} + +static void test_mock_shadowing_factory_reset_and_fail_writes(void) +{ + MockConfigStore store; + + // Injected out-of-range stored values are shadowed by defaults + // (same invariant as the NVS store). + store.stored.moistureThresholdLow = 150.0f; + store.stored.wateringDurationS = 301; + store.stored.wateringEnabled = 7; + store.stored.sensorReadIntervalMs = 999; + store.stored.wifiSsid = std::string(33, 'a'); + TEST_ASSERT_EQUAL_FLOAT(IConfigStore::kDefaultMoistureThresholdLow, + store.getMoistureThresholdLow()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultWateringDurationS, + store.getWateringDurationS()); + TEST_ASSERT_EQUAL(IConfigStore::kDefaultWateringEnabled, + store.getWateringEnabled()); + TEST_ASSERT_EQUAL_UINT32(IConfigStore::kDefaultSensorReadIntervalMs, + store.getSensorReadIntervalMs()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiSsid().c_str()); + + // Factory reset restores the factory state. + setAllNonDefaults(store); + TEST_ASSERT_TRUE(store.factoryReset()); + TEST_ASSERT_EQUAL(1, store.factoryResets); + assertAllDefaults(store); + + // Simulated persistence failure: write rejected, state untouched. + TEST_ASSERT_TRUE(store.setWateringDurationS(45)); + store.failWrites = true; + TEST_ASSERT_FALSE(store.setWateringDurationS(60)); + TEST_ASSERT_FALSE(store.setWateringEnabled(false)); + TEST_ASSERT_FALSE(store.setWifiCredentials("ssid", "pass")); + TEST_ASSERT_FALSE(store.clearWifiCredentials()); + TEST_ASSERT_FALSE(store.factoryReset()); + store.failWrites = false; + TEST_ASSERT_EQUAL_UINT32(45, store.getWateringDurationS()); +} + +// --------------------------------------------------------------------------- +// T028 — LockedConfigStore decorator delegates the full contract path +// unchanged (the wrapper adds task-level mutex serialization; see +// LockedConfigStore.h — same mechanism PR-02's LockedWaterPump test +// established). The instrumented mock proves every call reached the +// wrapped store. +// --------------------------------------------------------------------------- +static void test_locked_config_store_delegates_full_contract(void) +{ + MockConfigStore inner; + LockedConfigStore store(inner); + + // Factory state and every accepted write pass through unchanged. + assertAllDefaults(store); + setAllNonDefaults(store); + assertAllNonDefaults(store); + TEST_ASSERT_EQUAL(8, inner.acceptedWrites); // setAllNonDefaults calls + + // Validation rejections behave identically through the wrapper and + // leave the stored values untouched. + TEST_ASSERT_FALSE(store.setMoistureThresholdLow(150.0f)); + TEST_ASSERT_FALSE(store.setMoistureThresholdHigh(NAN)); + TEST_ASSERT_FALSE(store.setWateringDurationS(0)); + TEST_ASSERT_FALSE(store.setMinWateringIntervalS(0)); + TEST_ASSERT_FALSE(store.setSensorReadIntervalMs(999)); + TEST_ASSERT_FALSE(store.setDataLogIntervalMs(59999)); + TEST_ASSERT_FALSE(store.setWifiCredentials(std::string(33, 's'), "ok")); + TEST_ASSERT_EQUAL(7, inner.rejectedWrites); + assertAllNonDefaults(store); + + // Credential clear and factory reset delegate too. + TEST_ASSERT_TRUE(store.clearWifiCredentials()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiSsid().c_str()); + TEST_ASSERT_EQUAL_STRING("", store.getWifiPassword().c_str()); + TEST_ASSERT_TRUE(store.factoryReset()); + TEST_ASSERT_EQUAL(1, inner.factoryResets); + assertAllDefaults(store); + + // A persistence failure surfaces through the wrapper unchanged. + inner.failWrites = true; + TEST_ASSERT_FALSE(store.setWateringEnabled(false)); + TEST_ASSERT_FALSE(store.factoryReset()); +} + +// --------------------------------------------------------------------------- +// T028 — the wrapped NvsConfigStore keeps its contract behind the wrapper +// (real-store spot check: round-trip + factory reset through the mutex). +// --------------------------------------------------------------------------- +static void test_locked_config_store_over_real_store(void) +{ + resetNvs(); + NvsConfigStore inner; + LockedConfigStore store(inner); + + assertAllDefaults(store); + setAllNonDefaults(store); + assertAllNonDefaults(store); + TEST_ASSERT_TRUE(store.factoryReset()); + assertAllDefaults(store); +} + +void run_config_store_tests(void) +{ + RUN_TEST(test_defaults_on_erased_nvs); + RUN_TEST(test_round_trip_all_items); + RUN_TEST(test_persistence_across_reinit); + RUN_TEST(test_set_rejects_out_of_range); + RUN_TEST(test_boundary_values_accepted); + RUN_TEST(test_rejected_write_leaves_stored_value); + RUN_TEST(test_out_of_range_stored_values_shadowed); + RUN_TEST(test_factory_reset_restores_defaults); + RUN_TEST(test_credential_set_and_clear); + RUN_TEST(test_credentials_never_logged); + RUN_TEST(test_mock_defaults_roundtrip_rejection); + RUN_TEST(test_mock_shadowing_factory_reset_and_fail_writes); + // T028 — Locked* decorator (FR-013). + RUN_TEST(test_locked_config_store_delegates_full_contract); + RUN_TEST(test_locked_config_store_over_real_store); +} diff --git a/firmware/test_apps/host/main/test_data_storage.cpp b/firmware/test_apps/host/main/test_data_storage.cpp new file mode 100644 index 0000000..9c145ef --- /dev/null +++ b/firmware/test_apps/host/main/test_data_storage.cpp @@ -0,0 +1,966 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file test_data_storage.cpp + * @brief Host tests for the IDataStorage contract (linux preview target). + * + * LittleFsDataStorage runs against plain POSIX file I/O in a per-test + * temp directory (research.md D4) — the identical code path used under + * the /storage littlefs VFS mount on target. MockDataStorage is held to + * the same contract invariants (FR-012). + * + * Coverage maps to specs/003-nvs-littlefs-storage/contracts/IDataStorage.md + * and the data-model.md history and event layouts (tasks T018-T020, T023). + */ + +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "unity.h" + +#include "interfaces/IDataStorage.h" +#include "storage/LittleFsDataStorage.h" +#include "storage/LockedDataStorage.h" +#include "storage/testing/MockDataStorage.h" + +namespace { + +/// Default data-log interval in seconds (data-model.md: 300000 ms). +constexpr uint32_t kLogIntervalS = 300; + +constexpr std::size_t kRecordsPerChunk = + LittleFsDataStorage::kHistoryChunkMaxBytes / + LittleFsDataStorage::kHistoryRecordBytes; // 1024 + +constexpr std::size_t kMetricCapacity = + kRecordsPerChunk * LittleFsDataStorage::kHistoryMaxChunksPerMetric; // 10240 + +/// Per-test temp directory. Cleanup is RAII (scope exit) instead of the +/// Unity tearDown, which is shared by all suites in test_main.cpp. +class TempDir { +public: + TempDir() + { + char templ[] = "/tmp/ws_datastore_XXXXXX"; + char* dir = ::mkdtemp(templ); + TEST_ASSERT_NOT_NULL_MESSAGE(dir, "mkdtemp failed"); + path_ = dir; + } + ~TempDir() { removeTree(path_); } + TempDir(const TempDir&) = delete; + TempDir& operator=(const TempDir&) = delete; + + const std::string& path() const { return path_; } + +private: + static void removeTree(const std::string& path) + { + if (DIR* dir = ::opendir(path.c_str())) { + while (const dirent* entry = ::readdir(dir)) { + if (std::strcmp(entry->d_name, ".") == 0 || + std::strcmp(entry->d_name, "..") == 0) { + continue; + } + removeTree(path + "/" + entry->d_name); + } + ::closedir(dir); + } + std::remove(path.c_str()); // file, or directory now empty + } + + std::string path_; +}; + +/// Names of the regular entries in `dir` (no "."/".."); empty if absent. +std::vector listDir(const std::string& dir) +{ + std::vector names; + if (DIR* d = ::opendir(dir.c_str())) { + while (const dirent* entry = ::readdir(d)) { + if (std::strcmp(entry->d_name, ".") != 0 && + std::strcmp(entry->d_name, "..") != 0) { + names.emplace_back(entry->d_name); + } + } + ::closedir(d); + } + return names; +} + +std::string metricDirOf(const TempDir& dir, const std::string& metric) +{ + return dir.path() + "/hist/" + metric; +} + +long sizeOf(const std::string& path) +{ + struct stat st {}; + TEST_ASSERT_EQUAL_INT_MESSAGE(0, ::stat(path.c_str(), &st), path.c_str()); + return static_cast(st.st_size); +} + +void appendGarbage(const std::string& path, std::size_t bytes) +{ + FILE* file = std::fopen(path.c_str(), "ab"); + TEST_ASSERT_NOT_NULL(file); + const std::string garbage(bytes, 'X'); + TEST_ASSERT_EQUAL_size_t(bytes, + std::fwrite(garbage.data(), 1, bytes, file)); + TEST_ASSERT_EQUAL_INT(0, std::fclose(file)); +} + +// --- T018: range-query semantics (FR-009) ------------------------------- + +void test_query_is_chronological_and_inclusive(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + TEST_ASSERT_TRUE(storage.storeSensorReading("soil_moisture", 100, 1.0f)); + TEST_ASSERT_TRUE(storage.storeSensorReading("soil_moisture", 200, 2.0f)); + TEST_ASSERT_TRUE(storage.storeSensorReading("soil_moisture", 300, 3.0f)); + TEST_ASSERT_TRUE(storage.storeSensorReading("soil_moisture", 400, 4.0f)); + + // Both bounds inclusive, results in chronological order. + const auto mid = storage.getSensorReadings("soil_moisture", 200, 300); + TEST_ASSERT_EQUAL_size_t(2, mid.size()); + TEST_ASSERT_EQUAL_UINT32(200, mid[0].epoch); + TEST_ASSERT_EQUAL_FLOAT(2.0f, mid[0].value); + TEST_ASSERT_EQUAL_STRING("soil_moisture", mid[0].metric.c_str()); + TEST_ASSERT_EQUAL_UINT32(300, mid[1].epoch); + TEST_ASSERT_EQUAL_FLOAT(3.0f, mid[1].value); + + // Degenerate single-point range still matches inclusively. + const auto point = storage.getSensorReadings("soil_moisture", 400, 400); + TEST_ASSERT_EQUAL_size_t(1, point.size()); + TEST_ASSERT_EQUAL_UINT32(400, point[0].epoch); + + const auto all = storage.getSensorReadings("soil_moisture", 0, UINT32_MAX); + TEST_ASSERT_EQUAL_size_t(4, all.size()); + for (std::size_t i = 1; i < all.size(); ++i) { + TEST_ASSERT_TRUE(all[i - 1].epoch < all[i].epoch); + } +} + +void test_query_empty_never_an_error(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + // No data at all (not even the /hist directory). + TEST_ASSERT_TRUE(storage.getSensorReadings("soil_moisture", 0, 100).empty()); + + TEST_ASSERT_TRUE(storage.storeSensorReading("soil_moisture", 100, 1.0f)); + + // Unknown metric. + TEST_ASSERT_TRUE( + storage.getSensorReadings("env_temperature", 0, 1000).empty()); + // t0 > t1. + TEST_ASSERT_TRUE(storage.getSensorReadings("soil_moisture", 200, 100).empty()); + // Valid range containing no records. + TEST_ASSERT_TRUE(storage.getSensorReadings("soil_moisture", 101, 999).empty()); +} + +void test_unsafe_metric_names_rejected(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + // Metric names become directory names: empty, '/' and ".." are unsafe. + const char* bad[] = {"", "a/b", "..", "a..b", "/abs"}; + for (const char* name : bad) { + TEST_ASSERT_FALSE_MESSAGE(storage.storeSensorReading(name, 100, 1.0f), + name); + TEST_ASSERT_TRUE_MESSAGE( + storage.getSensorReadings(name, 0, UINT32_MAX).empty(), name); + } + // A rejected name must not consume a metric-directory slot. + TEST_ASSERT_TRUE(listDir(dir.path() + "/hist").empty()); +} + +// --- T018: MockDataStorage contract conformance (FR-012) ---------------- + +void test_mock_holds_range_query_contract(void) +{ + MockDataStorage mock; + + TEST_ASSERT_TRUE(mock.storeSensorReading("soil_moisture", 100, 1.0f)); + TEST_ASSERT_TRUE(mock.storeSensorReading("soil_moisture", 200, 2.0f)); + TEST_ASSERT_TRUE(mock.storeSensorReading("soil_moisture", 300, 3.0f)); + + const auto mid = mock.getSensorReadings("soil_moisture", 100, 200); + TEST_ASSERT_EQUAL_size_t(2, mid.size()); + TEST_ASSERT_EQUAL_UINT32(100, mid[0].epoch); + TEST_ASSERT_EQUAL_UINT32(200, mid[1].epoch); + + TEST_ASSERT_TRUE(mock.getSensorReadings("env_humidity", 0, 1000).empty()); + TEST_ASSERT_TRUE(mock.getSensorReadings("soil_moisture", 300, 100).empty()); + TEST_ASSERT_FALSE(mock.storeSensorReading("a/b", 100, 1.0f)); + TEST_ASSERT_FALSE(mock.storeSensorReading("", 100, 1.0f)); +} + +void test_mock_holds_bounds_and_event_contract(void) +{ + MockDataStorage mock; + + // kMaxMetrics distinct metrics accepted, one more rejected. + for (std::size_t i = 0; i < IDataStorage::kMaxMetrics; ++i) { + TEST_ASSERT_TRUE( + mock.storeSensorReading("metric_" + std::to_string(i), 100, 1.0f)); + } + TEST_ASSERT_FALSE(mock.storeSensorReading("metric_extra", 100, 1.0f)); + TEST_ASSERT_TRUE(mock.storeSensorReading("metric_0", 200, 2.0f)); + + // Over-long event detail truncated, never rejected; newest-first reads. + const std::string longDetail(IDataStorage::kEventDetailMaxLen + 80, 'x'); + TEST_ASSERT_TRUE( + mock.storeEvent(10, IDataStorage::kCategoryPump, longDetail)); + TEST_ASSERT_TRUE(mock.storeEvent(20, IDataStorage::kCategoryFailsafe, "b")); + const auto events = mock.getEvents(10); + TEST_ASSERT_EQUAL_size_t(2, events.size()); + TEST_ASSERT_EQUAL_UINT32(20, events[0].epoch); + TEST_ASSERT_EQUAL_size_t(IDataStorage::kEventDetailMaxLen, + events[1].detail.size()); + TEST_ASSERT_EQUAL_size_t(1, mock.getEvents(1).size()); + + // Injected stats returned verbatim; failWrites fails both stores. + mock.stats = StorageStats{4096, 1024}; + TEST_ASSERT_EQUAL_UINT32(4096, mock.getStorageStats().totalBytes); + TEST_ASSERT_EQUAL_UINT32(1024, mock.getStorageStats().usedBytes); + mock.failWrites = true; + TEST_ASSERT_FALSE(mock.storeSensorReading("metric_0", 300, 3.0f)); + TEST_ASSERT_FALSE(mock.storeEvent(30, IDataStorage::kCategoryReset, "c")); +} + +// --- getStorageStats provider paths (FR-008) ----------------------------- + +void test_storage_stats_provider_paths(void) +{ + TempDir dir; + + // No provider injected -> zeros (the absent/failing-provider contract). + LittleFsDataStorage noProvider(dir.path()); + const StorageStats none = noProvider.getStorageStats(); + TEST_ASSERT_EQUAL_UINT32(0, none.totalBytes); + TEST_ASSERT_EQUAL_UINT32(0, none.usedBytes); + + // Provider returning known total/used -> those values verbatim. + LittleFsDataStorage withProvider( + dir.path(), [](uint32_t& total, uint32_t& used) { + total = 983040; + used = 12288; + return true; + }); + const StorageStats stats = withProvider.getStorageStats(); + TEST_ASSERT_EQUAL_UINT32(983040, stats.totalBytes); + TEST_ASSERT_EQUAL_UINT32(12288, stats.usedBytes); +} + +// --- T019: bounding (FR-010, SC-004) ------------------------------------- + +/// Append `count` records spaced `stepS` seconds from `firstEpoch`, +/// value = record index. Asserts once on the aggregate failure count +/// (per-append asserts would dominate the endurance runtime). +void appendSeries(LittleFsDataStorage& storage, const std::string& metric, + uint32_t firstEpoch, std::size_t count, uint32_t stepS) +{ + std::size_t failures = 0; + for (std::size_t i = 0; i < count; ++i) { + if (!storage.storeSensorReading( + metric, firstEpoch + static_cast(i) * stepS, + static_cast(i))) { + ++failures; + } + } + TEST_ASSERT_EQUAL_size_t(0, failures); +} + +void test_chunk_seals_at_1024_records(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const std::string metric = "soil_moisture"; + const uint32_t base = 1000; + + appendSeries(storage, metric, base, kRecordsPerChunk, 1); + TEST_ASSERT_EQUAL_size_t(1, listDir(metricDirOf(dir, metric)).size()); + + // Record 1025 seals the chunk at 8 KiB and opens a successor. + TEST_ASSERT_TRUE(storage.storeSensorReading( + metric, base + static_cast(kRecordsPerChunk), 9.0f)); + TEST_ASSERT_EQUAL_size_t(2, listDir(metricDirOf(dir, metric)).size()); + + // All records remain retrievable across the chunk boundary. + const auto all = storage.getSensorReadings(metric, 0, UINT32_MAX); + TEST_ASSERT_EQUAL_size_t(kRecordsPerChunk + 1, all.size()); + TEST_ASSERT_EQUAL_FLOAT(9.0f, all.back().value); +} + +void test_non_monotonic_epoch_bumps_chunk_name(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const std::string metric = "soil_moisture"; + const uint32_t base = 1000; + + // Seal the first chunk (1024 records) so a second chunk is opened by + // the next append; the sealed chunk's first epoch is `base`. + appendSeries(storage, metric, base, kRecordsPerChunk, 1); + TEST_ASSERT_EQUAL_size_t(1, listDir(metricDirOf(dir, metric)).size()); + + // The successor chunk's name = its first record's epoch. Feed an epoch + // EARLIER than the sealed chunk's first epoch: the chunk name must be + // bumped past the sealed one (LittleFsDataStorage.cpp:268-272) so names + // stay strictly increasing — no collision with the existing chunk. + const uint32_t earlier = base - 500; + TEST_ASSERT_TRUE(storage.storeSensorReading(metric, earlier, 7.0f)); + + // Exactly two distinct chunk files exist (no name collision). + const auto chunks = listDir(metricDirOf(dir, metric)); + TEST_ASSERT_EQUAL_size_t(2, chunks.size()); + TEST_ASSERT_TRUE(chunks[0] != chunks[1]); + + // The earlier-epoch record is still retrievable over a range covering it. + const auto hit = storage.getSensorReadings(metric, 0, base - 1); + TEST_ASSERT_EQUAL_size_t(1, hit.size()); + TEST_ASSERT_EQUAL_UINT32(earlier, hit[0].epoch); + TEST_ASSERT_EQUAL_FLOAT(7.0f, hit[0].value); +} + +void test_ring_evicts_oldest_chunk(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const std::string metric = "soil_moisture"; + const uint32_t base = 1000000; + + // Fill the ring exactly: 10 chunks x 1024 records, nothing evicted. + appendSeries(storage, metric, base, kMetricCapacity, 1); + TEST_ASSERT_EQUAL_size_t(LittleFsDataStorage::kHistoryMaxChunksPerMetric, + listDir(metricDirOf(dir, metric)).size()); + + // One more record needs an 11th chunk -> the oldest chunk is removed. + const uint32_t next = base + static_cast(kMetricCapacity); + TEST_ASSERT_TRUE(storage.storeSensorReading(metric, next, -1.0f)); + TEST_ASSERT_EQUAL_size_t(LittleFsDataStorage::kHistoryMaxChunksPerMetric, + listDir(metricDirOf(dir, metric)).size()); + + // The first chunk's records are gone, everything newer is intact. + TEST_ASSERT_TRUE( + storage + .getSensorReadings(metric, base, + base + static_cast(kRecordsPerChunk) - 1) + .empty()); + const auto rest = storage.getSensorReadings(metric, 0, UINT32_MAX); + TEST_ASSERT_EQUAL_size_t(kMetricCapacity - kRecordsPerChunk + 1, + rest.size()); + TEST_ASSERT_EQUAL_UINT32(base + static_cast(kRecordsPerChunk), + rest.front().epoch); + TEST_ASSERT_EQUAL_UINT32(next, rest.back().epoch); +} + +void test_retention_30_days_at_default_interval(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const std::string metric = "env_temperature"; + const uint32_t base = 1700000000; + + // 30 days at the 5-min default log interval = 8640 records; the + // 10-chunk ring (10240 records) must hold them all (FR-010). + constexpr std::size_t k30Days = 30u * 24u * 3600u / kLogIntervalS; + appendSeries(storage, metric, base, k30Days, kLogIntervalS); + + const auto all = storage.getSensorReadings(metric, 0, UINT32_MAX); + TEST_ASSERT_EQUAL_size_t(k30Days, all.size()); + TEST_ASSERT_EQUAL_UINT32(base, all.front().epoch); // nothing evicted +} + +void test_endurance_ten_times_the_bound(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const std::string metric = "soil_moisture"; + const uint32_t base = 1000; + + // SC-004: 10x the per-metric bound (100 chunk-fulls) without a write + // failure or budget overrun. + constexpr std::size_t kAppends = 10 * kMetricCapacity; + appendSeries(storage, metric, base, kAppends, 1); + + TEST_ASSERT_EQUAL_size_t(LittleFsDataStorage::kHistoryMaxChunksPerMetric, + listDir(metricDirOf(dir, metric)).size()); + + // The newest records are intact after ~90 evictions. + const uint32_t last = base + static_cast(kAppends) - 1; + const auto tail = storage.getSensorReadings(metric, last - 99, last); + TEST_ASSERT_EQUAL_size_t(100, tail.size()); + TEST_ASSERT_EQUAL_UINT32(last, tail.back().epoch); + TEST_ASSERT_EQUAL_FLOAT(static_cast(kAppends - 1), + tail.back().value); +} + +void test_eleventh_distinct_metric_rejected(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + for (std::size_t i = 0; i < IDataStorage::kMaxMetrics; ++i) { + TEST_ASSERT_TRUE( + storage.storeSensorReading("metric_" + std::to_string(i), 100, 1.0f)); + } + TEST_ASSERT_FALSE(storage.storeSensorReading("metric_extra", 100, 1.0f)); + TEST_ASSERT_TRUE( + storage.getSensorReadings("metric_extra", 0, UINT32_MAX).empty()); + TEST_ASSERT_EQUAL_size_t(IDataStorage::kMaxMetrics, + listDir(dir.path() + "/hist").size()); + + // Existing metrics keep accepting appends at the metric cap. + TEST_ASSERT_TRUE(storage.storeSensorReading("metric_0", 200, 2.0f)); +} + +// --- T020: torn tails (research D5, contract invariant 2) ---------------- + +/// Path of the single chunk file of `metric` (asserts exactly one exists). +std::string singleChunkPath(const TempDir& dir, const std::string& metric) +{ + const std::string metricDir = metricDirOf(dir, metric); + const auto chunks = listDir(metricDir); + TEST_ASSERT_EQUAL_size_t(1, chunks.size()); + return metricDir + "/" + chunks[0]; +} + +void test_torn_tail_truncated_on_read(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const std::string metric = "soil_moisture"; + + TEST_ASSERT_TRUE(storage.storeSensorReading(metric, 100, 1.0f)); + TEST_ASSERT_TRUE(storage.storeSensorReading(metric, 200, 2.0f)); + TEST_ASSERT_TRUE(storage.storeSensorReading(metric, 300, 3.0f)); + + // Simulate a power loss mid-append: a partial trailing record. + const std::string chunk = singleChunkPath(dir, metric); + appendGarbage(chunk, 5); + TEST_ASSERT_EQUAL_INT( + 5, static_cast( + sizeOf(chunk) % LittleFsDataStorage::kHistoryRecordBytes)); + + // The torn tail is logically truncated; earlier records are intact. + const auto all = storage.getSensorReadings(metric, 0, UINT32_MAX); + TEST_ASSERT_EQUAL_size_t(3, all.size()); + TEST_ASSERT_EQUAL_UINT32(100, all[0].epoch); + TEST_ASSERT_EQUAL_FLOAT(1.0f, all[0].value); + TEST_ASSERT_EQUAL_UINT32(300, all[2].epoch); + TEST_ASSERT_EQUAL_FLOAT(3.0f, all[2].value); +} + +void test_torn_tail_repaired_before_append(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const std::string metric = "soil_moisture"; + + TEST_ASSERT_TRUE(storage.storeSensorReading(metric, 100, 1.0f)); + TEST_ASSERT_TRUE(storage.storeSensorReading(metric, 200, 2.0f)); + const std::string chunk = singleChunkPath(dir, metric); + appendGarbage(chunk, 3); + + // The next append truncates the torn tail so committed records stay + // 8-byte aligned and parseable. + TEST_ASSERT_TRUE(storage.storeSensorReading(metric, 300, 3.0f)); + TEST_ASSERT_EQUAL_INT( + 0, static_cast( + sizeOf(chunk) % LittleFsDataStorage::kHistoryRecordBytes)); + + const auto all = storage.getSensorReadings(metric, 0, UINT32_MAX); + TEST_ASSERT_EQUAL_size_t(3, all.size()); + TEST_ASSERT_EQUAL_UINT32(200, all[1].epoch); + TEST_ASSERT_EQUAL_UINT32(300, all[2].epoch); + TEST_ASSERT_EQUAL_FLOAT(3.0f, all[2].value); +} + +// --- T023: event log (FR-011, storeEvent/getEvents contract) ------------- + +std::string eventFileOf(const TempDir& dir, int index) +{ + return dir.path() + "/events/" + std::to_string(index) + ".log"; +} + +std::vector readAll(const std::string& path) +{ + std::vector bytes; + FILE* file = std::fopen(path.c_str(), "rb"); + TEST_ASSERT_NOT_NULL_MESSAGE(file, path.c_str()); + uint8_t buf[256]; + std::size_t n = 0; + while ((n = std::fread(buf, 1, sizeof(buf), file)) > 0) { + bytes.insert(bytes.end(), buf, buf + n); + } + TEST_ASSERT_EQUAL_INT(0, std::fclose(file)); + return bytes; +} + +void appendBytes(const std::string& path, const uint8_t* bytes, + std::size_t count) +{ + FILE* file = std::fopen(path.c_str(), "ab"); + TEST_ASSERT_NOT_NULL(file); + TEST_ASSERT_EQUAL_size_t(count, std::fwrite(bytes, 1, count, file)); + TEST_ASSERT_EQUAL_INT(0, std::fclose(file)); +} + +/// 9-byte detail encoding `index`: every framed record is then exactly +/// 16 bytes, so one event file holds exactly 1024 records and the +/// rotation boundary lands on a precise event index. +constexpr std::size_t kFixedDetailBytes = 9; +constexpr std::size_t kFixedRecordBytes = + LittleFsDataStorage::kEventHeaderBytes + kFixedDetailBytes; // 16 +constexpr std::size_t kEventsPerFile = + LittleFsDataStorage::kEventFileMaxBytes / kFixedRecordBytes; // 1024 +static_assert(kEventsPerFile * kFixedRecordBytes == + LittleFsDataStorage::kEventFileMaxBytes, + "fixed-size events must fill an event file exactly"); + +std::string fixedDetail(std::size_t index) +{ + char buf[16]; + std::snprintf(buf, sizeof(buf), "%09lu", + static_cast(index)); + return std::string(buf, kFixedDetailBytes); +} + +/// Append fixed-size events with epoch = firstEpoch + index and detail = +/// fixedDetail(index), for index in [firstIndex, firstIndex + count). +/// Asserts once on the aggregate failure count (per-append asserts would +/// dominate the rotation/burst runtime). +void appendEvents(LittleFsDataStorage& storage, uint32_t firstEpoch, + std::size_t firstIndex, std::size_t count) +{ + std::size_t failures = 0; + for (std::size_t i = firstIndex; i < firstIndex + count; ++i) { + if (!storage.storeEvent(firstEpoch + static_cast(i), + IDataStorage::kCategoryPump, fixedDetail(i))) { + ++failures; + } + } + TEST_ASSERT_EQUAL_size_t(0, failures); +} + +void test_event_framing_round_trip(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + TEST_ASSERT_TRUE(storage.storeEvent( + 0x11223344u, IDataStorage::kCategoryConnectivity, "abc")); + + // On-disk frame (data-model.md): marker 0xE7, uint32 LE epoch, + // uint8 category, uint8 detail_len, detail bytes. Fresh log -> 0.log. + const auto raw = readAll(eventFileOf(dir, 0)); + const uint8_t expected[] = {0xE7, 0x44, 0x33, 0x22, 0x11, + 0x03, 0x03, 'a', 'b', 'c'}; + TEST_ASSERT_EQUAL_size_t(sizeof(expected), raw.size()); + TEST_ASSERT_EQUAL_UINT8_ARRAY(expected, raw.data(), sizeof(expected)); + + const auto events = storage.getEvents(10); + TEST_ASSERT_EQUAL_size_t(1, events.size()); + TEST_ASSERT_EQUAL_UINT32(0x11223344u, events[0].epoch); + TEST_ASSERT_EQUAL_UINT8(IDataStorage::kCategoryConnectivity, + events[0].category); + TEST_ASSERT_EQUAL_STRING("abc", events[0].detail.c_str()); +} + +void test_get_events_newest_first_with_max_count(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + // No data at all (not even the /events directory) -> empty, no error. + TEST_ASSERT_TRUE(storage.getEvents(10).empty()); + + TEST_ASSERT_TRUE(storage.storeEvent(100, IDataStorage::kCategoryPump, "a")); + TEST_ASSERT_TRUE( + storage.storeEvent(200, IDataStorage::kCategoryFailsafe, "b")); + TEST_ASSERT_TRUE( + storage.storeEvent(300, IDataStorage::kCategoryConnectivity, "c")); + TEST_ASSERT_TRUE(storage.storeEvent(400, IDataStorage::kCategoryOta, "d")); + TEST_ASSERT_TRUE( + storage.storeEvent(500, IDataStorage::kCategoryReset, "e")); + + const auto top = storage.getEvents(3); + TEST_ASSERT_EQUAL_size_t(3, top.size()); + TEST_ASSERT_EQUAL_UINT32(500, top[0].epoch); + TEST_ASSERT_EQUAL_UINT8(IDataStorage::kCategoryReset, top[0].category); + TEST_ASSERT_EQUAL_STRING("e", top[0].detail.c_str()); + TEST_ASSERT_EQUAL_UINT32(400, top[1].epoch); + TEST_ASSERT_EQUAL_UINT32(300, top[2].epoch); + + const auto all = storage.getEvents(100); + TEST_ASSERT_EQUAL_size_t(5, all.size()); + for (std::size_t i = 1; i < all.size(); ++i) { + TEST_ASSERT_TRUE(all[i - 1].epoch > all[i].epoch); + } + + TEST_ASSERT_TRUE(storage.getEvents(0).empty()); +} + +void test_event_detail_truncated_not_rejected(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + // Over-long detail: truncated to 120 bytes, event still recorded. + const std::string longDetail(IDataStorage::kEventDetailMaxLen + 80, 'x'); + TEST_ASSERT_TRUE( + storage.storeEvent(100, IDataStorage::kCategoryFailsafe, longDetail)); + + // The on-disk detail_len byte is the truncated length. + const auto raw = readAll(eventFileOf(dir, 0)); + TEST_ASSERT_EQUAL_size_t(LittleFsDataStorage::kEventHeaderBytes + + IDataStorage::kEventDetailMaxLen, + raw.size()); + TEST_ASSERT_EQUAL_UINT8(IDataStorage::kEventDetailMaxLen, raw[6]); + + // An exactly-120-byte detail is kept whole. + const std::string maxDetail(IDataStorage::kEventDetailMaxLen, 'y'); + TEST_ASSERT_TRUE( + storage.storeEvent(200, IDataStorage::kCategoryPump, maxDetail)); + + const auto events = storage.getEvents(10); + TEST_ASSERT_EQUAL_size_t(2, events.size()); + TEST_ASSERT_EQUAL_STRING(maxDetail.c_str(), events[0].detail.c_str()); + TEST_ASSERT_EQUAL_size_t(IDataStorage::kEventDetailMaxLen, + events[1].detail.size()); + TEST_ASSERT_EQUAL_STRING( + longDetail.substr(0, IDataStorage::kEventDetailMaxLen).c_str(), + events[1].detail.c_str()); +} + +void test_unknown_event_category_passthrough(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + // PR-08 may extend the enum: unknown values pass through verbatim. + TEST_ASSERT_TRUE(storage.storeEvent(100, 0xCC, "future")); + TEST_ASSERT_TRUE(storage.storeEvent(200, 0, "zero")); + + const auto events = storage.getEvents(10); + TEST_ASSERT_EQUAL_size_t(2, events.size()); + TEST_ASSERT_EQUAL_UINT8(0, events[0].category); + TEST_ASSERT_EQUAL_UINT8(0xCC, events[1].category); + TEST_ASSERT_EQUAL_STRING("future", events[1].detail.c_str()); +} + +void test_event_rotation_drops_oldest_never_newest(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const uint32_t base = 1000000; + + // Fill 0.log exactly (1024 records x 16 bytes); 1.log not yet created. + appendEvents(storage, base, 0, kEventsPerFile); + TEST_ASSERT_EQUAL_INT( + static_cast(LittleFsDataStorage::kEventFileMaxBytes), + static_cast(sizeOf(eventFileOf(dir, 0)))); + TEST_ASSERT_EQUAL_size_t(1, listDir(dir.path() + "/events").size()); + + // The next append would exceed the cap -> switches to 1.log; the full + // file is left intact (nothing dropped yet). + appendEvents(storage, base, kEventsPerFile, 1); + TEST_ASSERT_EQUAL_INT(static_cast(kFixedRecordBytes), + static_cast(sizeOf(eventFileOf(dir, 1)))); + TEST_ASSERT_EQUAL_INT( + static_cast(LittleFsDataStorage::kEventFileMaxBytes), + static_cast(sizeOf(eventFileOf(dir, 0)))); + + // Fill 1.log; the following append truncates 0.log (the oldest half) + // and starts over there. + appendEvents(storage, base, kEventsPerFile + 1, kEventsPerFile - 1); + TEST_ASSERT_EQUAL_INT( + static_cast(LittleFsDataStorage::kEventFileMaxBytes), + static_cast(sizeOf(eventFileOf(dir, 1)))); + appendEvents(storage, base, 2 * kEventsPerFile, 1); + TEST_ASSERT_EQUAL_INT(static_cast(kFixedRecordBytes), + static_cast(sizeOf(eventFileOf(dir, 0)))); + + // The oldest half (indices 0..1023) is gone; everything newer is + // intact and newest-first, the newest record always retained. + const auto events = storage.getEvents(4 * kEventsPerFile); + TEST_ASSERT_EQUAL_size_t(kEventsPerFile + 1, events.size()); + TEST_ASSERT_EQUAL_UINT32(base + 2 * static_cast(kEventsPerFile), + events.front().epoch); + TEST_ASSERT_EQUAL_UINT32(base + static_cast(kEventsPerFile), + events.back().epoch); + for (std::size_t i = 1; i < events.size(); ++i) { + TEST_ASSERT_TRUE(events[i - 1].epoch > events[i].epoch); + } +} + +void test_event_burst_stays_within_budget(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + const uint32_t base = 2000; + + // Burst of 3000 events with every detail length 0..120: several + // rotations; no append may fail (contract: never rejected at the + // bound) and the two files stay within the 32 KiB total budget. + constexpr std::size_t kBurst = 3000; + std::size_t failures = 0; + for (std::size_t i = 0; i < kBurst; ++i) { + const std::string detail(i % (IDataStorage::kEventDetailMaxLen + 1), + 'e'); + if (!storage.storeEvent(base + static_cast(i), + IDataStorage::kCategoryPump, detail)) { + ++failures; + } + } + TEST_ASSERT_EQUAL_size_t(0, failures); + + const long size0 = sizeOf(eventFileOf(dir, 0)); + const long size1 = sizeOf(eventFileOf(dir, 1)); + TEST_ASSERT_TRUE( + size0 <= static_cast(LittleFsDataStorage::kEventFileMaxBytes)); + TEST_ASSERT_TRUE( + size1 <= static_cast(LittleFsDataStorage::kEventFileMaxBytes)); + TEST_ASSERT_TRUE( + size0 + size1 <= + 2 * static_cast(LittleFsDataStorage::kEventFileMaxBytes)); + + // The newest record survived every rotation. + const auto newest = storage.getEvents(1); + TEST_ASSERT_EQUAL_size_t(1, newest.size()); + TEST_ASSERT_EQUAL_UINT32(base + static_cast(kBurst) - 1, + newest[0].epoch); +} + +void test_event_torn_tail_skipped_and_repaired(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + TEST_ASSERT_TRUE(storage.storeEvent(100, IDataStorage::kCategoryPump, + "one")); + TEST_ASSERT_TRUE(storage.storeEvent(200, IDataStorage::kCategoryReset, + "two")); + const std::string active = eventFileOf(dir, 0); + const long valid = sizeOf(active); + + // Power loss mid-append: marker written, rest of the header torn off. + const uint8_t torn[] = {LittleFsDataStorage::kEventMarker, 0xAA, 0xBB}; + appendBytes(active, torn, sizeof(torn)); + + auto events = storage.getEvents(10); + TEST_ASSERT_EQUAL_size_t(2, events.size()); + TEST_ASSERT_EQUAL_UINT32(200, events[0].epoch); + TEST_ASSERT_EQUAL_UINT32(100, events[1].epoch); + + // The next append repairs the tail: the file is the valid prefix plus + // the new frame, and every record parses. + TEST_ASSERT_TRUE(storage.storeEvent(300, IDataStorage::kCategoryOta, + "three")); + TEST_ASSERT_EQUAL_INT( + static_cast(valid + LittleFsDataStorage::kEventHeaderBytes + 5), + static_cast(sizeOf(active))); + events = storage.getEvents(10); + TEST_ASSERT_EQUAL_size_t(3, events.size()); + TEST_ASSERT_EQUAL_UINT32(300, events[0].epoch); + TEST_ASSERT_EQUAL_STRING("three", events[0].detail.c_str()); + TEST_ASSERT_EQUAL_UINT32(100, events[2].epoch); +} + +void test_event_torn_detail_length_mismatch_skipped(void) +{ + TempDir dir; + LittleFsDataStorage storage(dir.path()); + + TEST_ASSERT_TRUE(storage.storeEvent(100, IDataStorage::kCategoryPump, + "ok")); + + // Complete header claiming 9 detail bytes, but only 3 present + // (length mismatch at the end of the file). + const uint8_t torn[] = {LittleFsDataStorage::kEventMarker, + 0x01, 0x00, 0x00, 0x00, + IDataStorage::kCategoryPump, 9, + 'a', 'b', 'c'}; + appendBytes(eventFileOf(dir, 0), torn, sizeof(torn)); + + const auto events = storage.getEvents(10); + TEST_ASSERT_EQUAL_size_t(1, events.size()); + TEST_ASSERT_EQUAL_UINT32(100, events[0].epoch); + TEST_ASSERT_EQUAL_STRING("ok", events[0].detail.c_str()); +} + +void test_event_active_file_detected_after_restart(void) +{ + TempDir dir; + const uint32_t base = 5000000; + + { + // First life: rotate into 1.log and leave 5 records there. + LittleFsDataStorage first(dir.path()); + appendEvents(first, base, 0, kEventsPerFile + 5); + } + + // Restart: a new instance must derive the active file (1.log) from + // the files alone and append there — no spurious rotation that would + // truncate the full 0.log. + LittleFsDataStorage second(dir.path()); + appendEvents(second, base, kEventsPerFile + 5, 1); + TEST_ASSERT_EQUAL_INT( + static_cast(LittleFsDataStorage::kEventFileMaxBytes), + static_cast(sizeOf(eventFileOf(dir, 0)))); + TEST_ASSERT_EQUAL_INT(static_cast(6 * kFixedRecordBytes), + static_cast(sizeOf(eventFileOf(dir, 1)))); + + const auto events = second.getEvents(2); + TEST_ASSERT_EQUAL_size_t(2, events.size()); + TEST_ASSERT_EQUAL_UINT32( + base + static_cast(kEventsPerFile) + 5, events[0].epoch); + TEST_ASSERT_EQUAL_UINT32( + base + static_cast(kEventsPerFile) + 4, events[1].epoch); +} + +void test_mock_event_bound_and_category_passthrough(void) +{ + MockDataStorage mock; + + // Unknown category passes through the mock verbatim too. + TEST_ASSERT_TRUE(mock.storeEvent(1, 0xCC, "future")); + TEST_ASSERT_EQUAL_UINT8(0xCC, mock.getEvents(1)[0].category); + + // At the bound the mock evicts oldest and always keeps the newest. + for (std::size_t i = 0; i < MockDataStorage::kMaxEvents + 8; ++i) { + TEST_ASSERT_TRUE(mock.storeEvent(100 + static_cast(i), + IDataStorage::kCategoryPump, "e")); + } + TEST_ASSERT_EQUAL_size_t(MockDataStorage::kMaxEvents, mock.events.size()); + const auto newest = mock.getEvents(1); + TEST_ASSERT_EQUAL_size_t(1, newest.size()); + TEST_ASSERT_EQUAL_UINT32(100 + MockDataStorage::kMaxEvents + 7, + newest[0].epoch); +} + +// --- T028: LockedDataStorage decorator (FR-013) -------------------------- +// Delegates the full contract path unchanged (the wrapper adds task-level +// mutex serialization; see LockedDataStorage.h — same mechanism PR-02's +// LockedWaterPump test established). The instrumented mock proves every +// call reached the wrapped storage. + +void test_locked_data_storage_delegates_full_contract(void) +{ + MockDataStorage inner; + inner.stats = StorageStats{983040, 12288}; + LockedDataStorage storage(inner); + + // storeSensorReading: accepted appends and the rejection paths pass + // through unchanged. + TEST_ASSERT_TRUE(storage.storeSensorReading("soil_moisture", 100, 1.0f)); + TEST_ASSERT_TRUE(storage.storeSensorReading("soil_moisture", 200, 2.0f)); + TEST_ASSERT_TRUE(storage.storeSensorReading("soil_moisture", 300, 3.0f)); + TEST_ASSERT_FALSE(storage.storeSensorReading("bad/metric", 400, 4.0f)); + TEST_ASSERT_EQUAL(3, inner.acceptedWrites); + TEST_ASSERT_EQUAL(1, inner.rejectedWrites); + + // getSensorReadings: inclusive chronological range, empty on t0 > t1. + const auto mid = storage.getSensorReadings("soil_moisture", 200, 300); + TEST_ASSERT_EQUAL_size_t(2, mid.size()); + TEST_ASSERT_EQUAL_UINT32(200, mid[0].epoch); + TEST_ASSERT_EQUAL_UINT32(300, mid[1].epoch); + TEST_ASSERT_TRUE(storage.getSensorReadings("soil_moisture", 300, 200).empty()); + + // storeEvent: detail truncation happens behind the wrapper. + const std::string longDetail(IDataStorage::kEventDetailMaxLen + 30, 'd'); + TEST_ASSERT_TRUE(storage.storeEvent(500, IDataStorage::kCategoryPump, + "pump started")); + TEST_ASSERT_TRUE(storage.storeEvent(600, IDataStorage::kCategoryFailsafe, + longDetail)); + + // getEvents: newest-first with maxCount. + const auto events = storage.getEvents(2); + TEST_ASSERT_EQUAL_size_t(2, events.size()); + TEST_ASSERT_EQUAL_UINT32(600, events[0].epoch); + TEST_ASSERT_EQUAL_size_t(IDataStorage::kEventDetailMaxLen, + events[0].detail.size()); + TEST_ASSERT_EQUAL_UINT32(500, events[1].epoch); + TEST_ASSERT_EQUAL_STRING("pump started", events[1].detail.c_str()); + + // getStorageStats: injected stats come back verbatim. + const StorageStats stats = storage.getStorageStats(); + TEST_ASSERT_EQUAL_UINT32(983040, stats.totalBytes); + TEST_ASSERT_EQUAL_UINT32(12288, stats.usedBytes); + + // A persistence failure surfaces through the wrapper unchanged. + inner.failWrites = true; + TEST_ASSERT_FALSE(storage.storeSensorReading("soil_moisture", 700, 7.0f)); + TEST_ASSERT_FALSE(storage.storeEvent(700, IDataStorage::kCategoryPump, "x")); +} + +// The wrapped LittleFsDataStorage keeps its contract behind the wrapper +// (real-store spot check: on-disk round-trip through the mutex). +void test_locked_data_storage_over_real_storage(void) +{ + TempDir dir; + LittleFsDataStorage inner(dir.path()); + LockedDataStorage storage(inner); + + TEST_ASSERT_TRUE(storage.storeSensorReading("env_temperature", 100, 21.5f)); + TEST_ASSERT_TRUE(storage.storeEvent(200, IDataStorage::kCategoryReset, + "boot")); + + const auto readings = + storage.getSensorReadings("env_temperature", 0, UINT32_MAX); + TEST_ASSERT_EQUAL_size_t(1, readings.size()); + TEST_ASSERT_EQUAL_FLOAT(21.5f, readings[0].value); + + const auto events = storage.getEvents(10); + TEST_ASSERT_EQUAL_size_t(1, events.size()); + TEST_ASSERT_EQUAL_STRING("boot", events[0].detail.c_str()); +} + +} // namespace + +void run_data_storage_tests(void) +{ + // T018 — range-query semantics + mock conformance. + RUN_TEST(test_query_is_chronological_and_inclusive); + RUN_TEST(test_query_empty_never_an_error); + RUN_TEST(test_unsafe_metric_names_rejected); + RUN_TEST(test_mock_holds_range_query_contract); + RUN_TEST(test_mock_holds_bounds_and_event_contract); + RUN_TEST(test_storage_stats_provider_paths); + // T019 — bounding: sealing, ring eviction, retention, endurance, cap. + RUN_TEST(test_chunk_seals_at_1024_records); + RUN_TEST(test_non_monotonic_epoch_bumps_chunk_name); + RUN_TEST(test_ring_evicts_oldest_chunk); + RUN_TEST(test_retention_30_days_at_default_interval); + RUN_TEST(test_endurance_ten_times_the_bound); + RUN_TEST(test_eleventh_distinct_metric_rejected); + // T020 — torn-tail handling. + RUN_TEST(test_torn_tail_truncated_on_read); + RUN_TEST(test_torn_tail_repaired_before_append); + // T023 — event log: framing, retrieval, rotation, budget, torn tails. + RUN_TEST(test_event_framing_round_trip); + RUN_TEST(test_get_events_newest_first_with_max_count); + RUN_TEST(test_event_detail_truncated_not_rejected); + RUN_TEST(test_unknown_event_category_passthrough); + RUN_TEST(test_event_rotation_drops_oldest_never_newest); + RUN_TEST(test_event_burst_stays_within_budget); + RUN_TEST(test_event_torn_tail_skipped_and_repaired); + RUN_TEST(test_event_torn_detail_length_mismatch_skipped); + RUN_TEST(test_event_active_file_detected_after_restart); + RUN_TEST(test_mock_event_bound_and_category_passthrough); + // T028 — Locked* decorator (FR-013). + RUN_TEST(test_locked_data_storage_delegates_full_contract); + RUN_TEST(test_locked_data_storage_over_real_storage); +} diff --git a/firmware/test_apps/host/main/test_main.cpp b/firmware/test_apps/host/main/test_main.cpp new file mode 100644 index 0000000..cfef728 --- /dev/null +++ b/firmware/test_apps/host/main/test_main.cpp @@ -0,0 +1,31 @@ +// SPDX-FileCopyrightText: 2026 Cryptotomte +// SPDX-License-Identifier: AGPL-3.0-or-later +/** + * @file test_main.cpp + * @brief Unity runner for the host test app (linux preview target). + * + * Each suite file registers its tests inside a run_*_tests() function; + * this runner calls every suite between UNITY_BEGIN/UNITY_END. The process + * exit code equals the Unity failure count and is the CI gate. + */ + +#include + +#include "unity.h" + +void run_water_pump_tests(void); +void run_config_store_tests(void); +void run_data_storage_tests(void); + +// Unity requires setUp/tearDown definitions (shared by all suites). +extern "C" void setUp(void) {} +extern "C" void tearDown(void) {} + +extern "C" void app_main(void) +{ + UNITY_BEGIN(); + run_water_pump_tests(); + run_config_store_tests(); + run_data_storage_tests(); + std::exit(UNITY_END()); +} diff --git a/firmware/test_apps/host/main/test_water_pump.cpp b/firmware/test_apps/host/main/test_water_pump.cpp index ddf5bd9..68a5f33 100644 --- a/firmware/test_apps/host/main/test_water_pump.cpp +++ b/firmware/test_apps/host/main/test_water_pump.cpp @@ -6,14 +6,14 @@ * * Tests the REAL enforcement logic (WaterPump base class) via * MockWaterPump (records applyOutput transitions) and FakeTimeProvider - * (manual clock). Plain Unity runner: the process exit code equals the - * failure count and is the CI gate. + * (manual clock). Registered via run_water_pump_tests() from the shared + * Unity runner (test_main.cpp); the process exit code equals the failure + * count and is the CI gate. * * Coverage maps to the invariants in * specs/002-pump-gpio-board/contracts/iwaterpump.md. */ -#include #include #include "unity.h" @@ -22,10 +22,6 @@ #include "actuators/testing/FakeTimeProvider.h" #include "actuators/testing/MockWaterPump.h" -// Unity requires setUp/tearDown definitions. -extern "C" void setUp(void) {} -extern "C" void tearDown(void) {} - namespace { constexpr int64_t kMaxRunTimeMs = WaterPump::kDefaultMaxRunTimeMs; // 300 000 @@ -249,9 +245,8 @@ static void test_locked_wrapper_delegates_full_cycle(void) TEST_ASSERT_TRUE(inner.outputCalls == expected); } -extern "C" void app_main(void) +void run_water_pump_tests(void) { - UNITY_BEGIN(); RUN_TEST(test_duration_self_stop_at_exact_boundary); RUN_TEST(test_max_runtime_forced_stop); RUN_TEST(test_reject_zero_duration); @@ -262,5 +257,4 @@ extern "C" void app_main(void) RUN_TEST(test_accumulated_runtime_across_runs); RUN_TEST(test_enforcement_within_one_poll); RUN_TEST(test_locked_wrapper_delegates_full_cycle); - std::exit(UNITY_END()); } diff --git a/firmware/test_apps/host/partitions_host.csv b/firmware/test_apps/host/partitions_host.csv new file mode 100644 index 0000000..4a4bb91 --- /dev/null +++ b/firmware/test_apps/host/partitions_host.csv @@ -0,0 +1,10 @@ +# Partition table for the host test app (linux preview target). +# +# The linux esp_partition emulation builds its file-backed flash from this +# table, which lets the REAL nvs_flash implementation run in host tests +# (research.md D3; reference pattern: IDF components/nvs_flash/host_test). +# Only `nvs` is used; phy_init/factory keep the layout valid. +# Name, Type, SubType, Offset, Size, Flags +nvs, data, nvs, 0x9000, 0x6000, +phy_init, data, phy, 0xf000, 0x1000, +factory, app, factory, 0x10000, 1M, diff --git a/firmware/test_apps/host/sdkconfig.defaults b/firmware/test_apps/host/sdkconfig.defaults index 4e40885..8831a3a 100644 --- a/firmware/test_apps/host/sdkconfig.defaults +++ b/firmware/test_apps/host/sdkconfig.defaults @@ -1 +1,7 @@ -# Host test app (linux preview target) — IDF defaults are sufficient. +# Host test app (linux preview target). +# +# A custom partition table provides the `nvs` partition that the linux +# esp_partition emulation exposes to the real nvs_flash implementation +# (research.md D3 of 003-nvs-littlefs-storage). +CONFIG_PARTITION_TABLE_CUSTOM=y +CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_host.csv" diff --git a/specs/003-nvs-littlefs-storage/checklists/requirements.md b/specs/003-nvs-littlefs-storage/checklists/requirements.md new file mode 100644 index 0000000..5295c01 --- /dev/null +++ b/specs/003-nvs-littlefs-storage/checklists/requirements.md @@ -0,0 +1,40 @@ +# Specification Quality Checklist: NVS Configuration and LittleFS Data Storage + +**Purpose**: Validate specification completeness and quality before proceeding to planning +**Created**: 2026-06-11 +**Feature**: [spec.md](../spec.md) + +## Content Quality + +- [x] No implementation details (languages, frameworks, APIs) +- [x] Focused on user value and business needs +- [x] Written for non-technical stakeholders +- [x] All mandatory sections completed + +## Requirement Completeness + +- [x] No [NEEDS CLARIFICATION] markers remain +- [x] Requirements are testable and unambiguous +- [x] Success criteria are measurable +- [x] Success criteria are technology-agnostic (no implementation details) +- [x] All acceptance scenarios are defined +- [x] Edge cases are identified +- [x] Scope is clearly bounded +- [x] Dependencies and assumptions identified + +## Feature Readiness + +- [x] All functional requirements have clear acceptance criteria +- [x] User scenarios cover primary flows +- [x] Feature meets measurable outcomes defined in Success Criteria +- [x] No implementation details leak into specification + +## Notes + +- Content quality: partition names, the littlefs dependency pin, and component + layout appear only in Assumptions as inherited constraints from PR-01/PR-02 and + the partition plan — they are project givens, not design choices made here. +- The three original [NEEDS CLARIFICATION] markers (retention target, settable + interval items, reservoir flag persistence) were resolved by Paul 2026-06-11: + ≥30-day retention, intervals settable, reservoir flags deferred to PR-05. + Decisions are encoded in US2 scenario 4, FR-001, and FR-006. diff --git a/specs/003-nvs-littlefs-storage/contracts/IConfigStore.md b/specs/003-nvs-littlefs-storage/contracts/IConfigStore.md new file mode 100644 index 0000000..f6add49 --- /dev/null +++ b/specs/003-nvs-littlefs-storage/contracts/IConfigStore.md @@ -0,0 +1,37 @@ +# Contract: IConfigStore + +**Component**: `firmware/components/interfaces/include/interfaces/IConfigStore.h` +**Feature**: 003-nvs-littlefs-storage + +Typed, validated access to persisted configuration. Replaces the config half of +the legacy `IDataStorage` (string-keyed `storeConfig`/`getConfig` over one JSON +blob) — a deliberate contract redesign, same approach as PR-02's `IWaterPump`. + +## Operations + +| Operation | Contract | +|---|---| +| `getMoistureThresholdLow() -> float` (and analogous typed getters for every item in data-model.md) | Never fails. Returns the stored value if present and in range, otherwise the compiled-in factory default. | +| `setMoistureThresholdLow(float) -> bool` (and analogous typed setters) | Validates against the item's documented range. In-range: persists atomically, returns true, value survives power cycle. Out-of-range: returns false, stored value untouched. | +| `getWifiSsid() / getWifiPassword() -> std::string` | Empty string = unconfigured (factory state). | +| `setWifiCredentials(ssid, password) -> bool` | Length-validated (≤32 / ≤64 bytes). Implementations MUST NOT log the values. | +| `clearWifiCredentials() -> bool` | Returns both items to factory (empty) state. | +| `factoryReset() -> bool` | Every item reads its factory default afterwards; credentials removed. Equivalent to erasing the underlying config storage. | + +## Invariants + +1. A getter never returns an out-of-range value (defaults shadow invalid storage). +2. A failed/rejected write never alters the stored value (old-or-new, never torn). +3. No operation blocks on or is affected by network state (Constitution I). +4. Header is host-includable: no IDF/hardware includes (Constitution II). +5. Credentials never appear in any diagnostic/log output (spec FR-004). + +## Implementations + +- `NvsConfigStore` (target + linux-target NVS emulation): one NVS entry per item, + namespace `wscfg`, schema in data-model.md; factory reset = + `nvs_flash_erase_partition` + re-init. +- `MockConfigStore` (header-only, `testing/`): in-memory map + call/limit + instrumentation for consumer tests in later PRs (PR-07, PR-09, PR-11). +- Concurrency: implementations are unsynchronized; cross-task consumers wrap in + the `Locked*` decorator (research D9, PR-02 CP3 precedent). diff --git a/specs/003-nvs-littlefs-storage/contracts/IDataStorage.md b/specs/003-nvs-littlefs-storage/contracts/IDataStorage.md new file mode 100644 index 0000000..b733bae --- /dev/null +++ b/specs/003-nvs-littlefs-storage/contracts/IDataStorage.md @@ -0,0 +1,53 @@ +# Contract: IDataStorage + +**Component**: `firmware/components/interfaces/include/interfaces/IDataStorage.h` +**Feature**: 003-nvs-littlefs-storage + +Sensor history, event records, and storage statistics. Replaces the data half of +the legacy `IDataStorage`. The two legacy methods with no callers +(`getLastSensorReading`, `pruneOldReadings`) are dropped; retention is an +internal bounded-storage guarantee instead of a caller obligation (spec FR-010/ +FR-012). + +## Types + +```text +SensorReading { metric: string, epoch: uint32, value: float } +EventRecord { epoch: uint32, category: uint8 enum, detail: string ≤120 } +StorageStats { total_bytes: uint32, used_bytes: uint32 } +``` + +Event categories: pump=1, failsafe=2, connectivity=3, ota=4, reset=5 (PR-08 may +extend; unknown values are stored and returned verbatim). + +## Operations + +| Operation | Contract | +|---|---| +| `storeSensorReading(metric, epoch, value) -> bool` | Appends; durable once true is returned (survives power loss). Unknown metric accepted up to 10 metric directories; the 11th distinct metric is rejected (false). Bounding/eviction is internal (≥30-day retention at default log interval, oldest-first eviction). | +| `getSensorReadings(metric, t0, t1) -> vector` | Chronological, inclusive range. Empty vector on no data, unknown metric, t0 > t1, or read error — never throws/fails (legacy parity). | +| `storeEvent(epoch, category, detail) -> bool` | Appends; `detail` longer than 120 bytes is silently truncated (the event is always recorded, never rejected for length). Rotation keeps total event storage ≤ budget and always retains the newest records. | +| `getEvents(maxCount) -> vector` | Newest-first, at most maxCount. Empty vector on no data/error. | +| `getStorageStats() -> StorageStats` | Total/used bytes of the data filesystem. | + +## Invariants + +1. History and event storage never exceed their documented budgets + (data-model.md); writes at the bound evict oldest data, never fail the append. +2. A power loss during append loses at most the in-flight record; previously + stored records remain readable (torn tails detected and skipped on read). +3. Reads are side-effect free except torn-tail truncation/skip. +4. Timestamps are caller-supplied epoch seconds, stored verbatim (parity + checklist 184). +5. Header is host-includable: no IDF/hardware includes (Constitution II). + +## Implementations + +- `LittleFsDataStorage`: POSIX stdio against an injectable base path — `/storage` + (VFS) on target, temp dir in host tests. On-disk formats in data-model.md. + Stats provider injected (esp_littlefs_info on target, fake on host). +- `StorageMount` (target-only helper, not part of the interface): mount-or-format + of the `storage` partition at startup, per research D2. +- `MockDataStorage` (header-only, `testing/`): in-memory vectors for consumer + tests in later PRs (PR-08, PR-09). +- Concurrency: unsynchronized; cross-task consumers use the `Locked*` decorator. diff --git a/specs/003-nvs-littlefs-storage/data-model.md b/specs/003-nvs-littlefs-storage/data-model.md new file mode 100644 index 0000000..832c94d --- /dev/null +++ b/specs/003-nvs-littlefs-storage/data-model.md @@ -0,0 +1,118 @@ +# Data Model: NVS Configuration and LittleFS Data Storage + +**Feature**: 003-nvs-littlefs-storage | **Date**: 2026-06-11 + +## Configuration items (NVS, namespace `wscfg`) + +| Item | NVS key | NVS type | Logical type | Default | Valid range | Unit | +|---|---|---|---|---|---|---| +| moistureThresholdLow | `moist_low` | u32 (float bits) | float | 30.0 | 0–100 | % | +| moistureThresholdHigh | `moist_high` | u32 (float bits) | float | 55.0 | 0–100 | % | +| wateringDuration | `water_dur` | u32 | uint32 | 20 | 1–300 | s | +| minWateringInterval (soak pause) | `soak_pause` | u32 | uint32 | 300 | ≥ 1 | s | +| wateringEnabled | `water_en` | u8 | bool | true | 0/1 | — | +| sensorReadInterval | `read_iv` | u32 | uint32 | 5000 | ≥ 1000 | ms | +| dataLogInterval | `log_iv` | u32 | uint32 | 300000 | ≥ 60000 | ms | +| wifiSsid | `wifi_ssid` | str | string | "" (unconfigured) | ≤ 32 bytes | — | +| wifiPassword | `wifi_pass` | str | string | "" | ≤ 64 bytes | — | + +Rules: + +- Read of a missing key → compiled-in default. Read of a stored value outside its + valid range → compiled-in default (FR-002; the invalid entry is left in place + until the next valid write). +- Write of an out-of-range value → rejected, stored value untouched (FR-003). +- Each accepted write goes directly to its own NVS entry (atomic old-or-new under + power loss; research D5/D8). +- Factory reset: `nvs_flash_erase_partition` + re-init → every read returns + defaults; credentials gone (FR-005). WiFi credentials never appear in logs or + diagnostics output (FR-004). +- Reservoir flags deliberately absent (FR-006, deferred to PR-05). New items = + new keys in the same namespace; no schema version needed (per-key layout is + self-describing). +- Defaults/ranges mirror the legacy firmware (`src/WateringController.cpp:16-21`, + `:445-475`) except: soak-pause semantics (2026-06-10 decision, enforced in + PR-11) and the two interval items being settable (2026-06-11 clarification). + Range floors for the intervals are new (legacy had none); 1 s / 1 min floors + prevent log-storm misconfiguration. + +## Sensor reading (littlefs) + +Logical entity: `{metric: string, timestamp: uint32 epoch seconds, value: float}`. + +Metric identifiers follow legacy naming: `env_temperature`, `env_humidity`, +`env_pressure`, `soil_moisture`, `soil_temperature`, `soil_ph`, `soil_ec`, +`soil_nitrogen`, `soil_phosphorus`, `soil_potassium`. The set is open — unknown +metrics are accepted (spec edge case), subject to a max of **10** metric +directories (budget guard sized to the legacy metric set: 10 × 80 KiB = 800 KiB +is the worst case the partition can absorb; oldest-inactive is NOT auto-evicted, +an 11th distinct metric is rejected with an error — prevents a buggy caller from +silently destroying history or blowing the budget). + +On-disk layout (research D6): + +```text +/storage/hist//.dat # append-only chunk +``` + +- Record: 8 bytes LE: `uint32 epoch`, `float value`. No per-record framing needed + beyond fixed size; a torn tail (file size % 8 ≠ 0) is truncated-on-read. +- Chunk cap: 8 KiB = 1024 records. New chunk when cap reached; filename = first + record's epoch. +- Ring bound: max 10 chunks per metric (80 KiB); creating chunk #11 deletes the + oldest (atomic remove). Guarantees ≥31.9 days at the 5-min default interval. +- Query (metric, t0, t1): pick chunks whose [first_epoch, next chunk's + first_epoch) overlaps the range, scan, filter. Chronological order is by + construction (appends use caller-supplied epochs; a non-monotonic timestamp is + stored as-is — time correctness is the caller's concern, parity checklist 184). +- Empty/no-data/unknown-metric query → empty result, not an error (FR-009). + +## Event record (littlefs) + +Logical entity: `{timestamp: uint32 epoch, category: enum, detail: string ≤120}`. +A longer `detail` is silently truncated to 120 bytes on store (documented +contract semantics — never rejected, the event itself is always recorded). + +Categories (u8): `pump=1`, `failsafe=2`, `connectivity=3`, `ota=4`, `reset=5` +(FR-011; PR-08 may extend the enum — unknown categories are stored and returned +verbatim). + +On-disk layout (research D7): + +```text +/storage/events/0.log +/storage/events/1.log +``` + +- Record framing: `0xEV-marker byte (0xE7)`, `uint32 epoch`, `uint8 category`, + `uint8 detail_len`, `detail bytes`. Torn tail detected by marker/length + mismatch and skipped. +- Active file appends until 16 KiB cap; rotation truncates the other file and + switches (oldest half dropped, newest always retained — FR Acceptance US3.2). +- Retrieval: newest-first across both files, optional max-count. + +## Storage statistics + +`{total_bytes: uint32, used_bytes: uint32}` from `esp_littlefs_info("storage")` +on target; mocked on host. Exposed through the data-storage interface for the +serial status line and PR-09 status API (FR-008, parity checklist 106). + +## Budget (worst case) + +| Consumer | Budget | +|---|---| +| History: 10 metrics × 80 KiB | 800 KiB | +| Events: 2 × 16 KiB | 32 KiB | +| Headroom (littlefs metadata, future) | ~38–68 KiB | +| **Total available after web assets** | **~870–900 KiB** | + +## State transitions + +- **Filesystem**: unmounted → mounted (boot, valid FS) | unmounted → formatted → + mounted (first boot/corruption; FR-007) — data loss accepted, bricking not. +- **Config item**: missing → default-on-read → set (valid write) → updated | + unchanged (invalid write rejected) → missing (factory reset). +- **History chunk**: active (appending) → sealed (cap reached, successor created) + → deleted (ring eviction). +- **Event file**: active (appending) → full (cap) → standby (other truncated, + becomes active) → truncated. diff --git a/specs/003-nvs-littlefs-storage/plan.md b/specs/003-nvs-littlefs-storage/plan.md new file mode 100644 index 0000000..ba54cf8 --- /dev/null +++ b/specs/003-nvs-littlefs-storage/plan.md @@ -0,0 +1,162 @@ +# Implementation Plan: NVS Configuration and LittleFS Data Storage + +**Branch**: `003-nvs-littlefs-storage` | **Date**: 2026-06-11 | **Spec**: [spec.md](spec.md) + +**Input**: Feature specification from `/specs/003-nvs-littlefs-storage/spec.md` + +## Summary + +Port the storage layer to ESP-IDF as two redesigned contracts: `IConfigStore` +(typed, validated configuration in NVS with compiled-in factory defaults and +factory reset — FR13) and `IDataStorage` (bounded sensor history, rotating event +log, and usage statistics on a littlefs partition). Real NVS runs in host tests +via the IDF linux target; littlefs-specific code is confined to a target-only +mount helper, with all file logic exercised over POSIX on the host. Formats and +budgets are fixed in [data-model.md](data-model.md); verified groundwork in +[research.md](research.md) (D1–D10). + +## Technical Context + +**Language/Version**: C++ (modern, RAII), native ESP-IDF v6.0.1 APIs only — no +Arduino layers; `std::string` not `String` + +**Primary Dependencies**: `nvs_flash` (IDF built-in), `joltwallet/littlefs` +==1.22.1 (already pinned in `firmware/main/idf_component.yml` + +`dependencies.lock`), Unity (host tests) + +**Storage**: `nvs` partition 16 KiB @ 0x9000 (namespace `wscfg`, one entry per +item); `storage` partition (littlefs subtype) 960 KiB @ 0x310000, VFS at +`/storage` — both per `firmware/partitions.csv` (PR-01, unchanged) + +**Testing**: Unity host test app on the IDF linux preview target +(`firmware/test_apps/host`, exit code = failure count, CI job from the PR-02 +branch); real nvs_flash on linux (research D3); POSIX temp-dir for file logic +(D4); HIL checklist on the rev1 rig at Checkpoint 3 + +**Target Platform**: ESP32-WROOM-32E (rev1 devkit + rev2 PCB Kconfig targets); +host tests on linux target. This component is board-agnostic (no pins) — both +board targets build identically + +**Project Type**: ESP-IDF component set within the existing `firmware/` project + +**Performance Goals**: append path (1 record / dataLogInterval, default 5 min) +trivially exceeds need; worst-case range query (10 chunks ≈ 80 KiB scan) well +under one second — no further targets warranted + +**Constraints**: power-loss safety (config old-or-new; history/events lose at +most the in-flight record — research D5); history+events ≤ ~832 KiB worst case +within the ~870–900 KiB available after web assets; ≥30-day retention at default +log interval; no migration of Arduino-era data (FR-014) + +**Scale/Scope**: 9 config items (7 watering + 2 WiFi credential), ≤10 metrics × +80 KiB history (= 800 KiB worst case), 32 KiB event log; 2 new interface +headers, 1 new `storage` component, 2 header-only mocks + 2 `Locked*` +decorators, host test suites + CI hook + +## Constitution Check + +*GATE: evaluated pre-Phase 0 and re-checked post-design — PASS (no violations).* + +- **I. Safety First**: No pump/actuator interaction. Storage never blocks on + network (contract invariant); invalid stored config shadows to safe defaults + (FR-002) which feeds the fail-safe story of later PRs. ✅ +- **II. Host-Testability**: Both contracts are host-includable headers in + `components/interfaces`; all logic (validation, defaulting, bounding, eviction, + rotation, range queries) runs in the host suite — real NVS on linux target, + POSIX file I/O for data storage; only `StorageMount` touches esp_littlefs and + contains no logic. Mocks provided for downstream consumers. ✅ +- **III. Reproducible Builds**: No new dependencies; littlefs already pinned + exactly; builds stay inside the pinned Docker image; CI extended, not altered + (D1 caveat: image step pip-installs pinned littlefs-python at build time — + accepted, noted). ✅ +- **IV. Frozen Legacy**: Legacy code read-only reference; parity divergences + (bounded history, settable intervals, dropped dead methods, redesigned + contracts) are documented in the parity checklist as part of this feature. ✅ +- **V. Checkpoint-Gated AI Workflow**: This plan is CP2 material; implementation + via `implementer` subagent only; review never skipped. ✅ +- **VI. English Outward**: All artifacts English. ✅ + +## Project Structure + +### Documentation (this feature) + +```text +specs/003-nvs-littlefs-storage/ +├── spec.md # Feature specification (clarified 2026-06-11) +├── plan.md # This file +├── research.md # Verified decisions D1–D10 +├── data-model.md # NVS schema, file formats, budgets, state transitions +├── quickstart.md # Build/host-test/HIL validation guide +├── contracts/ +│ ├── IConfigStore.md +│ └── IDataStorage.md +├── checklists/ +│ └── requirements.md +└── tasks.md # /speckit-tasks output (next phase) +``` + +### Source Code (repository root) + +```text +firmware/ +├── components/ +│ ├── interfaces/ # exists after PR-02 merge (D10) +│ │ └── include/interfaces/ +│ │ ├── IConfigStore.h # NEW — contract per contracts/IConfigStore.md +│ │ └── IDataStorage.h # NEW — contract per contracts/IDataStorage.md +│ └── storage/ # NEW component +│ ├── CMakeLists.txt # littlefs REQUIRES + StorageMount.cpp +│ │ # excluded when IDF_TARGET=linux +│ │ # (esp_littlefs has no linux port) +│ ├── include/storage/ +│ │ ├── NvsConfigStore.h +│ │ ├── LittleFsDataStorage.h +│ │ ├── StorageMount.h # target-only mount/format/info wrapper +│ │ ├── LockedConfigStore.h # header-only Locked* decorators +│ │ ├── LockedDataStorage.h # (FR-013, PR-02 CP3 precedent) +│ │ └── testing/ +│ │ ├── MockConfigStore.h # header-only, never in target builds +│ │ └── MockDataStorage.h +│ └── src/ +│ ├── NvsConfigStore.cpp +│ ├── LittleFsDataStorage.cpp +│ └── StorageMount.cpp # target-only (see CMakeLists) +├── main/ +│ └── CMakeLists.txt # + littlefs_create_partition_image(storage ...) +├── storage_image/ # NEW — committed seed dir for the image +└── test_apps/host/ # exists after PR-02 merge; extended: + ├── main/ # + nvs_flash REQUIRES, custom partition CSV + │ ├── test_config_store.cpp # NEW suites + │ └── test_data_storage.cpp + └── partitions_host.csv # NEW — nvs partition for linux emulation + +.github/workflows/firmware-build.yml # + test -f firmware/build/storage.bin +docs/parity-checklist.md # + §6 divergence notes (bounded history, + # settable intervals, dropped methods) +``` + +**Structure Decision**: One new `storage` component (one concern per component, +`firmware/CLAUDE.md`); contracts join the shared `interfaces` component from +PR-02. Implementation begins by merging `main` after PR-02 (#7) lands — fallback +per research D10 if it hasn't. + +### Phases + +- **Phase A — contracts & mocks**: interface headers + header-only mocks + + contract docs cross-references. Host-includable, no IDF includes. +- **Phase B — NvsConfigStore**: schema per data-model.md; host suite against real + linux-target NVS (defaults, round-trips, rejection, shadowing, factory reset). +- **Phase C — LittleFsDataStorage**: chunked history + rotating events over POSIX + with injectable base path; host suite (bounding, eviction, rotation, torn-tail, + range queries, metric-cap). +- **Phase D — target integration**: StorageMount, partition image in build, CI + image check, parity-checklist updates, HIL checklist delivery. + +Each phase maps to independently testable user stories (US1 ↔ B, US2/US3 ↔ C, +US4 ↔ D); detailed ordering is `/speckit-tasks`' job. + +## Complexity Tracking + +| Violation | Why Needed | Simpler Alternative Rejected Because | +|-----------|------------|-------------------------------------| +| Principle III (borderline): littlefs partition-image step pip-installs `littlefs-python==0.15.0` from PyPI at build time, inside the otherwise fully pinned container build (research D1) | The pinned joltwallet/littlefs component's own `project_include.cmake` does this; the version is exactly pinned by the component | Pre-baking the wheel into a custom Docker image forks the canonical `espressif/idf:v6.0.1` image — a larger Principle III deviation than a pinned, component-managed pip install. Accepted with the precondition that CI has PyPI access. | diff --git a/specs/003-nvs-littlefs-storage/quickstart.md b/specs/003-nvs-littlefs-storage/quickstart.md new file mode 100644 index 0000000..f432498 --- /dev/null +++ b/specs/003-nvs-littlefs-storage/quickstart.md @@ -0,0 +1,71 @@ +# Quickstart: NVS Configuration and LittleFS Data Storage + +**Feature**: 003-nvs-littlefs-storage + +## Prerequisites + +- Docker with the pinned `espressif/idf:v6.0.1` image (Constitution III). +- PR-02 merged into the branch (provides `components/interfaces` and + `firmware/test_apps/host`; research D10). +- For HIL: the rev1 devkit bench rig. + +## Build (both targets) + +```bash +docker run --rm -v "$PWD/firmware":/project -w /project espressif/idf:v6.0.1 \ + idf.py -DSDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.board.rev1_devkit" build +docker run --rm -v "$PWD/firmware":/project -w /project espressif/idf:v6.0.1 \ + idf.py -DSDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.board.rev2" build +``` + +When switching boards, delete the generated `firmware/sdkconfig` (or run +`idf.py fullclean`) first — stale sdkconfig silently keeps the previous board +(see `firmware/CLAUDE.md`). + +**Expected**: green build AND `firmware/build/storage.bin` exists (littlefs image +created by `littlefs_create_partition_image` — CI verifies this file). + +## Host tests (linux preview target) + +```bash +docker run --rm -v "$PWD/firmware":/project -w /project/test_apps/host \ + espressif/idf:v6.0.1 bash -c \ + "idf.py --preview set-target linux && idf.py build && ./build/*_host_tests.elf" +``` + +**Expected**: exit code 0 (= failure count). Suites cover, per spec SC-005: + +- every factory default read on erased NVS (FR-001/FR-002) +- set/get round-trip per item; rejection per documented out-of-range case (FR-003) +- out-of-range *stored* value shadowed by default (FR-002, US1.3) +- factory reset semantics incl. credential removal (FR-005, SC-003) +- history range filtering incl. empty/no-data/t0>t1 cases (FR-009) +- bounded eviction: simulate 10× retention bound, verify budget + retrievability + (FR-010, SC-004) +- event rotation: newest retained at budget, torn-tail skip (FR-011) +- 17th-metric rejection, unknown-category passthrough (data-model guards) + +## HIL checklist (Paul, bench rig, Checkpoint 3) + +1. **Fresh flash** (`idf.py erase-flash flash monitor`): boot log shows littlefs + formatted + mounted, usage reported (total ≈ 960 KiB, used small) → SC-001, + US4.1. +2. **Config persistence**: change a threshold (serial/test console), power-cycle, + verify value retained → SC-002, US1.2. +3. **Factory reset**: invoke reset operation, verify all defaults restored and + WiFi credentials gone → SC-003, US1.4. +4. **Reboot persistence**: write a few readings/events, reboot (no erase), verify + data survives and usage figures reflect it → US4.3. +5. **Corruption recovery**: deliberately corrupt the storage partition + (`esptool.py erase_region 0x310000 0x2000` — both littlefs superblock blocks, + a single-block erase would still mount via the redundant superblock), boot, + verify reformat-and-continue (no boot loop) → US4.2, FR-007. + +## Artifacts + +- Spec: [spec.md](spec.md) — requirements and scenarios +- Plan: [plan.md](plan.md) — component layout and phases +- Research: [research.md](research.md) — verified facts D1–D10 +- Data model: [data-model.md](data-model.md) — NVS schema, file formats, budgets +- Contracts: [contracts/IConfigStore.md](contracts/IConfigStore.md), + [contracts/IDataStorage.md](contracts/IDataStorage.md) diff --git a/specs/003-nvs-littlefs-storage/research.md b/specs/003-nvs-littlefs-storage/research.md new file mode 100644 index 0000000..e2b0e86 --- /dev/null +++ b/specs/003-nvs-littlefs-storage/research.md @@ -0,0 +1,190 @@ +# Research: NVS Configuration and LittleFS Data Storage + +**Feature**: 003-nvs-littlefs-storage | **Date**: 2026-06-11 + +Facts below were verified against the pinned component archive +(joltwallet__littlefs 1.22.1, hash matching `firmware/dependencies.lock`; bundled +littlefs core v2.11), the ESP-IDF v6.0.1 tree inside the pinned +`espressif/idf:v6.0.1` Docker image, the frozen Arduino firmware, and the +PR-02 branch (`002-pump-gpio-board`). + +## D1. Partition image creation in the build (CI acceptance) + +**Decision**: Call `littlefs_create_partition_image(storage FLASH_IN_PROJECT)` +with a committed seed directory; extend the CI "verify binaries" step with +`test -f firmware/build/storage.bin`. + +**Rationale**: The component's `project_include.cmake` provides +`littlefs_create_partition_image( [FLASH_IN_PROJECT] +[DEPENDS ...])` as an ALL target — it runs on every `idf.py build`, looks up the +`storage` partition (0xF0000 @ 0x310000 in `firmware/partitions.csv`), and emits +`build/storage.bin`. `FLASH_IN_PROJECT` attaches it to the flash target, which +gives HIL "fresh-flash boots with formatted FS" a deterministic starting image. + +**Alternatives considered**: Letting first boot format an empty partition (no +image) — rejected: the acceptance criterion explicitly requires image creation in +the build, and PR-10 will need the same mechanism for web assets. + +**Caveat**: The image step pip-installs `littlefs-python==0.15.0` into a build-dir +venv at build time (pinned by the component); CI needs PyPI access (GitHub runners +have it). + +## D2. Mount and usage API + +**Decision**: `esp_vfs_littlefs_register()` with `.partition_label = "storage"`, +`.base_path = "/storage"`, `.format_if_mount_failed = true`; usage via +`esp_littlefs_info("storage", &total, &used)`. + +**Rationale**: Exactly matches the legacy mount-or-format parity behavior +(`LittleFS.begin(true)`) and the component's canonical example. The partition +**name** is `storage` (littlefs is the subtype) — the PR-06 PRD wording "label +littlefs" is a known imprecision, already corrected in the spec assumptions. + +## D3. Host-testing strategy for NVS + +**Decision**: Run the **real** `nvs_flash` implementation on the linux preview +target in the host test app. `NvsConfigStore` is tested directly against it; a +header-only `MockConfigStore` is still provided for later PRs' consumers. + +**Rationale**: Verified in the IDF v6.0.1 tree: `nvs_flash` has an explicit linux +branch (drops esp_libc/esptool_py deps, `-DLINUX_TARGET`), and `esp_partition` +ships `partition_linux.c` (file-backed flash emulation); IDF's own +`nvs_flash/host_test` app proves the combination. Testing the real NVS engine +covers defaulting, round-trips, and factory-reset semantics with no mock skew — +strictly stronger than the "NVS layer mocked" fallback in the PR-06 acceptance +criteria. + +**Requirements**: host test app needs `CONFIG_PARTITION_TABLE_CUSTOM` with an +`nvs` partition CSV and `nvs_flash_erase()` between tests for isolation. + +**Alternatives considered**: KV-primitive abstraction with an in-memory fake — +rejected as the primary strategy (mock skew, more interface surface), kept as +non-goal. + +## D4. Host-testing strategy for littlefs data storage + +**Decision**: `LittleFsDataStorage` is written against POSIX stdio with an +injectable base path. On target it operates under the `/storage` VFS mount; in +host tests it operates on a temp directory on the build host. The mount/format/ +usage wrapper (`StorageMount`) is the only target-only code; storage statistics +reach consumers through the data-storage interface and are mocked on host. + +**Rationale**: Verified: the esp_littlefs component has **no** linux-target branch +in its CMakeLists — it cannot run on the host. All record-format, bounding, +eviction, and range-query logic is plain file I/O and runs identically over POSIX +on both targets; only mount and `esp_littlefs_info` are littlefs-specific. + +**Mechanism**: the storage component's CMakeLists conditionally excludes the +littlefs REQUIRES and `StorageMount.cpp` when `IDF_TARGET=linux` +(`if(NOT ${IDF_TARGET} STREQUAL "linux")`), so the host test app can link +`NvsConfigStore` + `LittleFsDataStorage` without pulling in esp_littlefs. + +## D5. Power-loss safety model + +**Decision**: Append-only chunk files with `fflush`+`fsync` per record append; +chunk eviction by file delete; no in-place overwrites of committed data. Per-record +sentinel/length framing so a partially committed tail record is detected and +skipped on read. NVS handles its own atomicity. + +**Rationale**: littlefs core v2.11 guarantees (README/DESIGN): rename and remove +are atomic under power loss; file updates are copy-on-write and not visible until +sync/close — interruption reverts to the last synced state (old data is never +corrupted, in-flight data is dropped). NVS entries are CRC-protected and +log-structured: a torn `nvs_set_*` yields old-or-new, never garbage; +`nvs_flash_erase_partition()` + `nvs_flash_init()` is the standard factory-reset +sequence (interrupted erase just fails init and is erased again — no key +resurrection). + +**Consequence for the spec's torn-write edge case**: config = old-or-new by NVS +design; history/events = at most the in-flight record is lost, earlier records +intact by COW design. Both host-testable as behavior (write → simulated-crash → +re-open semantics) at the file level on POSIX. + +## D6. History format and retention budget + +**Decision**: Per-metric directory of append-only fixed-record chunk files, +ring-evicted per metric: + +- Record: `{uint32 epoch_seconds, float value}` = 8 bytes, little-endian. +- Chunk file: `/storage/hist//.dat`, capped at 8 KiB + (1024 records ≈ 3.55 days at the 5-min default log interval). +- Per metric: at most 10 chunks (80 KiB). Evicting the oldest full chunk when an + 11th would be created leaves ≥9 full chunks + the active one ⇒ **≥31.9 days** + retained at default interval — meets the ≥30-day clarification with the worst + case of all 10 metrics active staying within budget: + 10 metrics × 80 KiB = 800 KiB ≤ ~870 KiB available (partition plan), with the + realistic case (7 always-on metrics) at 560 KiB. +- Query: select chunks by filename epoch + linear scan; records are + chronological within and across chunks by construction. + +**Rationale**: Append-only avoids header rewrite amplification; eviction is an +atomic file delete; chunk granularity keeps per-file littlefs block overhead +amortized (8 KiB files vs 4 KiB blocks — unlike daily files at 2.3 KiB, which +would waste a block each across ~310 files). Numbers documented as deliberate +divergence from the legacy unbounded JSON files (parity checklist update is part +of this feature). + +**Alternatives considered**: single pre-sized ring file per metric (rejected: +in-place header updates on every append, harder torn-write reasoning); daily +files (rejected: block-overhead explosion); legacy JSON arrays (rejected: +unbounded, parse-buffer failure mode is the very defect this PR removes). + +## D7. Event log format and budget + +**Decision**: Two rotating append-only files (`/storage/events/0.log`, +`1.log`), 16 KiB each (~32 KiB total). Record: `{uint32 epoch, uint8 category, +uint8 detail_len, char detail[≤120]}` framed with a leading record marker for +torn-tail detection. When the active file reaches its cap, the other file is +truncated and becomes active (oldest-half rotation, newest events always +retained). Retrieval returns newest-first across both files. + +**Rationale**: Satisfies FR-011 rotation semantics with two atomic primitives +(append, truncate-on-rotate); 32 KiB ≈ several hundred events — months of normal +operation; keeps total worst-case littlefs usage (800 + 32 KiB) inside the +~870 KiB budget. Categories per FR-011/PR-08 PRD: pump, fail-safe, connectivity, +OTA, reset/watchdog (stored as an enum byte; PR-08 may extend). + +## D8. NVS schema + +**Decision**: Single namespace `wscfg`; one NVS entry per configuration item with +short fixed keys (`moist_low`, `moist_high`, `water_dur`, `soak_pause`, +`water_en`, `read_iv`, `log_iv`, `wifi_ssid`, `wifi_pass`). Types: `float` stored +as u32 bit-pattern (NVS has no float type), durations/intervals as u32, bool as +u8, credentials as strings. Factory defaults compiled in as constants; reads +fall back to the default on missing key **or** out-of-range stored value +(spec FR-002); factory reset = `nvs_flash_erase_partition("nvs")` + re-init. + +**Rationale**: Individual entries (vs one blob) give per-item atomic updates, no +read-modify-write of unrelated items, and trivial PR-05/PR-07 extensibility; +16 KiB ≈ 378 entries dwarfs the ~10 needed (partition plan). Key names ≤15 chars +(NVS limit). + +**Alternatives considered**: one ~200 B struct blob (partition plan mentioned +both) — rejected: any single-item change rewrites the whole blob and a layout +change invalidates everything; per-entry layout is self-versioning by key. + +## D9. Concurrency + +**Decision**: Base implementations stay unsynchronized (host-test friendly); +cross-task use goes through header-only `LockedConfigStore`/`LockedDataStorage` +decorators over the interfaces, following the PR-02 CP3 precedent +(`LockedWaterPump` pattern). The decorators ship **in this PR** (FR-013 coverage) +even though the first concurrent consumer arrives in PR-08/PR-09. + +**Rationale**: Spec FR-013 requires no corruption/torn values under concurrent +use; the established project pattern solves it without polluting pure logic with +RTOS primitives, keeping Constitution II (host-testability) intact. + +## D10. Sequencing relative to PR-02 + +**Decision**: Implementation starts by merging current `main` into this branch +**after PR-02 (#7) merges** — the `interfaces` component and +`firmware/test_apps/host` app this feature extends live there. If PR-02 is still +unmerged when implementation is approved, the fallback is to create the same +scaffolding here and reconcile at merge (both are additive), but waiting is the +recommended path since #7 only awaits its HIL pass. + +**CI note**: the host-test job pattern (esp-idf-ci-action with explicit +`target: linux` — the action's default `IDF_TARGET=esp32` otherwise aborts +`set-target linux`) also comes from the PR-02 branch and gets extended, not +duplicated. diff --git a/specs/003-nvs-littlefs-storage/spec.md b/specs/003-nvs-littlefs-storage/spec.md new file mode 100644 index 0000000..fd766ae --- /dev/null +++ b/specs/003-nvs-littlefs-storage/spec.md @@ -0,0 +1,326 @@ +# Feature Specification: NVS Configuration and LittleFS Data Storage + +**Feature Branch**: `003-nvs-littlefs-storage` + +**Created**: 2026-06-11 + +**Status**: Draft + +**Input**: User description: "PR-06 nvs-littlefs-storage (Phase 2 — infrastructure). Port the storage layer to ESP-IDF per docs/prd/PR-06-nvs-littlefs-storage.md: typed configuration in NVS with compiled-in factory defaults (FR13), mounted littlefs partition with usage reporting, sensor history and event records with explicit bounding, host-testable interfaces with mocks. No migration of Arduino-era data." + +## Clarifications + +### Session 2026-06-11 + +- Q: Sensor history retention target — legacy capacity (~1 week), fixed ≥30-day + window, or maximize (~months, fill available ~870–900 KiB)? → A: Fixed ≥30-day + retention for all metrics at default log interval; keep headroom for the event + log and future needs. +- Q: Do sensorReadInterval and dataLogInterval become first-class settable + configuration items (legacy persists them but has no setters or API)? → A: Yes, + settable typed items with validation; deliberate improvement, divergence + recorded in the parity checklist. +- Q: Reservoir feature flags (reservoirPumpEnabled, reservoirAutoLevelControl) — + persist in NVS (partition plan) or keep legacy non-persistence? → A: Deferred + to PR-05 (which introduces the reservoir board flag); not part of this PR's + configuration schema. The store must be extensible for PR-05 additions. + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - Configuration survives restarts and resets to safe defaults (Priority: P1) + +The greenhouse operator adjusts watering settings (moisture thresholds, watering +duration, soak pause, automatic watering on/off). Those settings persist across +power cycles, firmware restarts, and OTA updates. A fresh device — or one whose +configuration storage was erased or corrupted — boots with the documented factory +defaults rather than failing or behaving unpredictably. An explicit factory-reset +operation returns all configuration to those defaults. + +**Why this priority**: FR13 is the core requirement of this PR. Every later phase +(WiFi provisioning, HTTP API, watering controller) reads its configuration through +this layer; safe defaults are part of the safety story — a device must never run +with undefined watering parameters. + +**Independent Test**: On a fresh-flashed device, verify documented defaults are in +effect; change a setting, power-cycle, verify it persisted; invoke factory reset, +verify defaults are restored. Fully verifiable on host against the configuration +interface with storage mocked. + +**Acceptance Scenarios**: + +1. **Given** a device with empty/erased configuration storage, **When** the firmware + boots and the configuration layer initializes, **Then** every configuration item + reads back its documented factory default. +2. **Given** a running device, **When** the operator changes a configuration item to + a valid value, **Then** the change is persisted immediately and survives a power + cycle. +3. **Given** a stored value that is out of its documented valid range (e.g. written + by a future buggy build), **When** the configuration is read, **Then** the + documented default is returned for that item instead of the invalid value. +4. **Given** a configured device, **When** factory reset is invoked, **Then** all + configuration items return to factory defaults and stored WiFi credentials are + removed. +5. **Given** any failed write (storage full, I/O error), **When** the write is + rejected, **Then** the previously stored value remains intact and readable. + +--- + +### User Story 2 - Sensor history is recorded, bounded, and retrievable (Priority: P2) + +The system records periodic sensor readings (environment temperature/humidity/ +pressure, soil moisture/temperature/pH/EC, and optional nutrient values) so the +operator can later view history over a chosen time window. Storage is explicitly +bounded: the device runs for months without history growth degrading or breaking +the system — old data is discarded in favor of new data, never the reverse. + +**Why this priority**: Enables the history part of FR7 (consumed by the HTTP API in +PR-09). The legacy firmware's unbounded history files eventually break retrieval; +the explicit bound is the headline improvement required by the parity checklist +(line 172: "equivalent or better retention behavior with explicit bounding"). + +**Independent Test**: Through the data-storage interface (mocked filesystem on +host): store readings, query by metric and time range, verify correct filtering; +fill storage past its bound, verify oldest data is evicted and writes keep +succeeding. + +**Acceptance Scenarios**: + +1. **Given** stored readings for a metric, **When** history is queried for a time + range, **Then** exactly the readings inside the range are returned in + chronological order. +2. **Given** a query for a metric or range with no data, **When** history is + queried, **Then** an empty result is returned (not an error) — parity with + legacy behavior. +3. **Given** history storage at its configured bound, **When** new readings arrive, + **Then** the oldest readings are discarded, the new readings are stored, and + retrieval keeps working. +4. **Given** sustained recording of all metrics at the default log interval, + **Then** at least the most recent 30 days of history remain retrievable and + total history storage stays within its documented budget (≥30-day retention + per the 2026-06-11 clarification). + +--- + +### User Story 3 - Safety-relevant events are persisted (Priority: P3) + +The system can persist operational event records — pump start/stop with cause, +fail-safe activations, connectivity state changes, OTA events, and reset/watchdog +reasons — so the operator can diagnose what the device did and why, even after +restarts. (This PR provides the storage capability; wiring actual producers is +PR-08.) + +**Why this priority**: New surface with no legacy equivalent; the constitution +requires safety-relevant events to be persisted. Needed by PR-08/PR-09 but has no +consumer inside this PR, so it ranks below configuration and history. + +**Independent Test**: Through the data-storage interface on host: append event +records, retrieve them newest-first, verify rotation keeps total event storage +within its bound while always retaining the most recent events. + +**Acceptance Scenarios**: + +1. **Given** appended event records, **When** events are retrieved, **Then** they + come back with timestamp, category, and detail, most recent first. +2. **Given** event storage at its bound, **When** new events are appended, **Then** + the oldest events are rotated out and the newest events are always retained. +3. **Given** a burst of events (e.g. crash loop), **When** events are appended + rapidly, **Then** event storage never exceeds its budget and never starves + history or configuration storage. + +--- + +### User Story 4 - Storage health is visible (Priority: P3) + +The operator (and later the status API) can see how much data storage is in use: +total and used bytes for the data filesystem. A freshly flashed device prepares its +data filesystem automatically on first boot; a corrupted filesystem is recovered by +reformatting rather than leaving the device unusable. + +**Why this priority**: Parity item (legacy reports storage in serial status and +`/api/status`); also the bring-up proof that the data partition works at all. + +**Independent Test**: [HIL] Fresh-flash the rig: first boot formats and mounts the +data filesystem and reports plausible total/used numbers; subsequent boots mount +without reformatting and previously stored data is still present. + +**Acceptance Scenarios**: + +1. **Given** a fresh-flashed device (blank data partition), **When** it boots, + **Then** the data filesystem is formatted and mounted automatically and usage + reporting shows total/used bytes. +2. **Given** a device with an unmountable/corrupted data filesystem, **When** it + boots, **Then** the filesystem is reformatted and the device continues operating + (data loss is accepted; bricking is not) — parity with legacy mount behavior. +3. **Given** a mounted filesystem with stored data, **When** the device reboots, + **Then** the data survives and usage figures reflect it. + +--- + +### Edge Cases + +- Power loss mid-write: a torn configuration write must never leave the item + unreadable — the previous value or the default must win; a torn history/event + append may lose that record but must not corrupt earlier records. +- Configuration storage full: writes fail explicitly; existing values remain + readable; the system keeps running. +- System clock not yet set (before time sync): readings/events are still accepted; + timestamps are epoch-based as provided by the caller (time correctness is the + caller's concern, per parity checklist line 184). +- Concurrent access from multiple tasks (controller logging while status/history is + being read): no corruption, no torn reads. +- Query with start time after end time: empty result, not an error. +- Storing a reading for a metric never seen before: accepted and retrievable up + to the documented metric cap of 10 distinct metrics (exactly the legacy metric + set's size; nutrient metrics appear only when the sensor provides them). An + 11th distinct metric is rejected with an error — this guards the storage + budget and prevents a buggy caller from silently destroying history. + +## Requirements *(mandatory)* + +### Functional Requirements + +- **FR-001**: The system MUST provide a typed configuration store with these items, + defaults, and valid ranges (the documented factory defaults, verified against the + legacy firmware): + + | Item | Default | Valid range | Unit | + |---|---|---|---| + | moistureThresholdLow | 30.0 | 0–100 | % | + | moistureThresholdHigh | 55.0 | 0–100 | % | + | wateringDuration | 20 | 1–300 | s | + | minWateringInterval (soak pause) | 300 | ≥ 1 | s | + | wateringEnabled | true | — | bool | + | sensorReadInterval | 5000 | ≥ 1000 | ms | + | dataLogInterval | 300000 | ≥ 60000 | ms | + + All seven items are first-class settable items, including sensorReadInterval + and dataLogInterval (decision 2026-06-11: a deliberate improvement — legacy + persists these two but offers no way to change them; the divergence is recorded + in the parity checklist, and the future API may expose them). + +- **FR-002**: Reading any configuration item that is missing, erased, or out of its + valid range MUST return the documented factory default (FR13). The device MUST + never operate with undefined configuration. + +- **FR-003**: Accepted configuration changes MUST be persisted immediately and + survive power cycles, restarts, and OTA updates. Writes of out-of-range values + MUST be rejected without altering the stored value. + +- **FR-004**: The configuration store MUST reserve storage for WiFi credentials + (network name and secret) with "unconfigured" as factory state. Provisioning UX + is out of scope (PR-07); this PR defines where credentials live, that they are + excluded from any diagnostic output, and that factory reset removes them. + +- **FR-005**: An explicit factory-reset operation MUST restore every configuration + item to its factory default and remove stored WiFi credentials, equivalent to + erasing the configuration storage (per docs/partition-plan.md factory-reset + doctrine). + +- **FR-006**: Reservoir feature flags (reservoirPumpEnabled, + reservoirAutoLevelControl) are NOT part of this PR's configuration schema + (decision 2026-06-11): legacy deliberately resets them to false at boot (parity + checklist line 171) while docs/partition-plan.md line 91 anticipated NVS entries + — the persistence decision is deferred to PR-05, which introduces the + reservoir board flag (rev1-only since the single-pump decision). The + configuration store's design allows PR-05 to add items without contract + changes (per-item schema, see plan/research D8). + +- **FR-007**: The system MUST prepare its data filesystem on the dedicated data + partition at startup: mount if valid, format-then-mount on first boot or + corruption (no manual intervention, no bricking — parity checklist line 166). + +- **FR-008**: The system MUST report data-filesystem usage as total and used bytes, + suitable for the serial status output and status API (parity checklist line 106). + +- **FR-009**: The system MUST store sensor readings (metric identifier, epoch + timestamp, numeric value) and return them filtered by metric and time range in + chronological order. Errors and no-data conditions yield an empty result, not a + failure (parity with legacy retrieval). At most 10 distinct metrics are + accepted (budget guard, sized to the legacy metric set); storing an 11th + distinct metric is rejected with an error. + +- **FR-010**: Sensor history storage MUST be explicitly bounded with + oldest-data-first eviction; retrieval MUST keep working at and beyond the bound. + The bound and resulting retention MUST be documented in the parity checklist as a + deliberate divergence from the legacy unbounded format. + +- **FR-011**: The system MUST store event records (epoch timestamp, category, + detail) covering at least: pump start/stop with cause, fail-safe activation, + connectivity state change, OTA event, reset/watchdog reason. Event storage MUST + rotate within an explicit budget, always retaining the most recent events. + Producers are out of scope (PR-08); this PR delivers the storage and retrieval + capability. + +- **FR-012**: Configuration and data-storage capabilities MUST be exposed through + hardware-agnostic interfaces with mock implementations, fully exercisable in the + host test suite (Constitution II). The legacy single mixed-concern interface is + deliberately split: configuration store and data storage are separate contracts; + the two legacy methods with no callers (latest-reading lookup, manual pruning) + are dropped in favor of the internal bounding guarantee (FR-010). + +- **FR-013**: Concurrent use from multiple tasks (logging writes while history or + status is read) MUST NOT corrupt stored data or return torn values. + +- **FR-014**: No data is migrated from the legacy firmware (Arduino LittleFS + history, wifi_config.json): first boot of the new firmware starts clean. Storage + formats may diverge from legacy; every divergence is recorded in the parity + checklist. + +### Key Entities + +- **Configuration item**: a named, typed setting with factory default and valid + range; persisted individually in non-volatile configuration storage (16 KiB + partition, ~378 usable entries — far above the ~10 items needed). +- **Sensor reading**: metric identifier + epoch timestamp + numeric value; metrics + follow the legacy naming (env_temperature, env_humidity, env_pressure, + soil_moisture, soil_temperature, soil_ph, soil_ec, optional soil_nitrogen/ + soil_phosphorus/soil_potassium). +- **Event record**: epoch timestamp + category + human-readable detail; new in this + firmware generation. +- **Storage statistics**: total and used bytes of the data filesystem. + +## Success Criteria *(mandatory)* + +### Measurable Outcomes + +- **SC-001**: A fresh-flashed device reaches a fully configured state (all defaults + active, data filesystem mounted and reporting usage) on first boot with zero + manual steps. +- **SC-002**: 100% of accepted configuration changes survive an immediate power + cycle ([HIL] verified on the bench rig). +- **SC-003**: Factory reset restores 100% of configuration items to their + documented defaults and removes stored credentials ([HIL] verified). +- **SC-004**: After continuous simulated recording at default intervals to 10× the + retention bound, history retrieval still succeeds and total data storage stays + within its documented budget (host-tested). +- **SC-005**: The host test suite covers: every default value, set/get round-trips + for every item, rejection of every documented out-of-range case, factory-reset + semantics, history range filtering, bounded eviction, and event rotation — and + runs green in CI alongside both board targets' builds (the host suite itself + is board-independent and runs once). +- **SC-006**: Both board targets build green in CI including data-filesystem image + creation. + +## Assumptions + +- The partition layout from PR-01 is authoritative: 16 KiB configuration (NVS) + partition and 960 KiB data partition named `storage` (littlefs subtype) per + `firmware/partitions.csv`; the PR-06 PRD wording "label littlefs" is understood + as that partition (name `storage`). +- The pinned filesystem dependency (joltwallet/littlefs 1.22.1, already in + `firmware/main/idf_component.yml`) is used as-is; no version change. +- ~870–900 KiB of the data partition remain for history + events after gzipped web + assets and filesystem metadata (docs/partition-plan.md) — web assets themselves + are PR-10's concern. +- The component/mock/host-test layout follows the pattern established by PR-02 + (interfaces component, header-only mocks under `testing/`, host test app on the + linux preview target with failure-count exit code). If PR-06 merges before PR-02, + this PR creates the shared interfaces component; otherwise it extends it. +- "Mode" in the PR-06 PRD maps to the persisted `wateringEnabled` flag; pump manual + mode is runtime-only state (legacy parity) and is not persisted. +- AP-mode password for provisioning is a build-time option (PR-07 decision), not a + stored configuration item. +- Timestamps are epoch seconds supplied by callers; time synchronization is PR-08's + concern (parity checklist line 184: epoch-based and monotonic across migration). +- Event record categories listed in FR-011 follow the PR-08 PRD; PR-08 may extend + the set without changing this PR's storage contract. diff --git a/specs/003-nvs-littlefs-storage/tasks.md b/specs/003-nvs-littlefs-storage/tasks.md new file mode 100644 index 0000000..1d3a040 --- /dev/null +++ b/specs/003-nvs-littlefs-storage/tasks.md @@ -0,0 +1,168 @@ +# Tasks: NVS Configuration and LittleFS Data Storage + +**Input**: Design documents from `/specs/003-nvs-littlefs-storage/` + +**Prerequisites**: plan.md, spec.md, research.md, data-model.md, contracts/, quickstart.md + +**Tests**: Host tests are explicitly required (spec SC-005, acceptance criteria, Constitution II) — within each story: interface header + skeleton first (tests must compile), then failing tests, then implementation. + +**Organization**: Grouped by user story; each story phase is an independently testable increment. + +## Format: `[ID] [P?] [Story] Description` + +## Phase 1: Setup + +**Purpose**: Branch baseline and component scaffolding + +- [x] T001 Merge current `origin/main` into `003-nvs-littlefs-storage` — PR-02 (#7) must be merged to main first (research D10). If #7 is still open: STOP and surface to Paul; the D10 fallback (create scaffolding here, reconcile at merge) requires his explicit approval. Then verify both target builds green per quickstart.md +- [x] T002 Create `storage` component skeleton: `firmware/components/storage/CMakeLists.txt` with REQUIRES nvs_flash + interfaces, and the littlefs REQUIRES + `src/StorageMount.cpp` wrapped in `if(NOT ${IDF_TARGET} STREQUAL "linux")` (esp_littlefs has no linux port — research D4 mechanism); empty include/src tree per plan.md structure +- [x] T003 [P] Create committed littlefs seed directory `firmware/storage_image/` (a single `README` placeholder file explaining the image's role) + +--- + +## Phase 2: Foundational (Blocking Prerequisites) + +**Purpose**: Host test app must be able to run NVS + new suites before any story's tests can be written + +**⚠️ CRITICAL**: Blocks all user stories + +- [x] T004 Extend host test app for NVS on linux: add `nvs_flash` to REQUIRES in `firmware/test_apps/host/main/CMakeLists.txt`, add `firmware/test_apps/host/partitions_host.csv` with an `nvs` partition, set `CONFIG_PARTITION_TABLE_CUSTOM` in `firmware/test_apps/host/sdkconfig.defaults` (research D3; IDF's nvs_flash/host_test is the reference pattern) +- [x] T005 Register two empty Unity suites `firmware/test_apps/host/main/test_config_store.cpp` and `firmware/test_apps/host/main/test_data_storage.cpp` in the host app build; verify the host app still builds and runs (exit 0) on the linux target + +**Checkpoint**: Host harness ready — story implementation can begin + +--- + +## Phase 3: User Story 1 — Configuration survives restarts and resets to safe defaults (Priority: P1) 🎯 MVP + +**Goal**: Typed NVS-backed configuration with compiled-in defaults, validation, credential handling, and factory reset (FR-001..FR-006, FR13) + +**Independent Test**: Host suite against real linux-target NVS: erased NVS → all defaults; set/get round-trips; out-of-range rejection and shadowing; factory reset (quickstart.md "Host tests") + +### Contract & skeleton for User Story 1 + +- [x] T006 [P] [US1] `IConfigStore` interface header per contracts/IConfigStore.md in `firmware/components/interfaces/include/interfaces/IConfigStore.h` (host-includable, no IDF includes; documents contract + divergences from legacy) +- [x] T007 [P] [US1] Header-only `MockConfigStore` in `firmware/components/storage/include/storage/testing/MockConfigStore.h` (in-memory, instrumented; never compiled into target builds) +- [x] T008 [US1] `NvsConfigStore` declaration + stub (compiles, all methods fail/return defaults) in `firmware/components/storage/include/storage/NvsConfigStore.h` + `firmware/components/storage/src/NvsConfigStore.cpp` + +### Tests for User Story 1 (write against the stub, must fail) + +- [x] T009 [P] [US1] Default-on-erased-NVS tests (every item from data-model.md table) in `firmware/test_apps/host/main/test_config_store.cpp` +- [x] T010 [P] [US1] Round-trip + persistence-across-reinit tests per item in `firmware/test_apps/host/main/test_config_store.cpp` +- [x] T011 [P] [US1] Out-of-range write rejection (every documented bound) and out-of-range *stored* value shadowing (US1 scenario 3) in `firmware/test_apps/host/main/test_config_store.cpp` +- [x] T012 [P] [US1] Factory reset semantics + credential set/clear/never-logged tests, plus `MockConfigStore` contract-conformance cases (same invariants as the real store — FR-012) in `firmware/test_apps/host/main/test_config_store.cpp` + +### Implementation for User Story 1 + +- [x] T013 [US1] Implement `NvsConfigStore` per data-model.md NVS schema (namespace `wscfg`, per-item entries, float-as-u32-bits, defaults/range constants, factory reset via `nvs_flash_erase_partition` + re-init) in `firmware/components/storage/src/NvsConfigStore.cpp` +- [x] T014 [US1] Run US1 suites green on linux target; fix until exit 0 — NOTE: execution deferred to the main session (implementer agent does not run docker builds) + +**Checkpoint**: Config layer fully verified on host — MVP of this feature + +--- + +## Phase 4: User Story 2 — Sensor history is recorded, bounded, and retrievable (Priority: P2) + +**Goal**: Chunked, ring-evicted per-metric history with ≥30-day retention and parity query semantics (FR-009, FR-010) + +**Independent Test**: Host suite over POSIX temp dir: range filtering, bounded eviction at 10× retention, torn-tail handling, metric cap (quickstart.md) + +### Contract & skeleton for User Story 2 + +- [x] T015 [P] [US2] `IDataStorage` interface header (SensorReading/EventRecord/StorageStats types, full contract per contracts/IDataStorage.md) in `firmware/components/interfaces/include/interfaces/IDataStorage.h` +- [x] T016 [P] [US2] Header-only `MockDataStorage` in `firmware/components/storage/include/storage/testing/MockDataStorage.h` +- [x] T017 [US2] `LittleFsDataStorage` declaration + stub (injectable base path + stats provider, compiles) in `firmware/components/storage/include/storage/LittleFsDataStorage.h` + `firmware/components/storage/src/LittleFsDataStorage.cpp` + +### Tests for User Story 2 (write against the stub, must fail) + +- [x] T018 [P] [US2] Range-query tests: chronological order, inclusive bounds, empty result on no-data/unknown-metric/t0>t1 (FR-009, edge cases), plus `MockDataStorage` contract-conformance cases (FR-012) in `firmware/test_apps/host/main/test_data_storage.cpp` +- [x] T019 [P] [US2] Bounding tests: chunk sealing at 8 KiB, eviction at 11th chunk, ≥30-day retention guarantee at default interval, SC-004 10×-bound endurance, 11th-distinct-metric rejection in `firmware/test_apps/host/main/test_data_storage.cpp` +- [x] T020 [P] [US2] Torn-tail tests: file size % 8 ≠ 0 → truncate-on-read, earlier records intact (research D5) in `firmware/test_apps/host/main/test_data_storage.cpp` + +### Implementation for User Story 2 + +- [x] T021 [US2] Implement history part of `LittleFsDataStorage` per data-model.md (8-byte LE records, `/hist//.dat` chunks, fsync-per-append, ring eviction, 10-metric cap) in `firmware/components/storage/src/LittleFsDataStorage.cpp` +- [x] T022 [US2] Run US2 suites green on linux target — NOTE: execution deferred to the main session (implementer agent does not run docker builds) + +**Checkpoint**: History storage verified independently of US1 + +--- + +## Phase 5: User Story 3 — Safety-relevant events are persisted (Priority: P3) + +**Goal**: Rotating two-file event log, newest always retained (FR-011) + +**Independent Test**: Host suite: append/retrieve newest-first, rotation at cap, burst behavior, torn-tail skip + +### Tests for User Story 3 (write first, must fail) + +- [x] T023 [P] [US3] Event tests: framed record round-trip, newest-first retrieval with maxCount, rotation truncates oldest half and never the newest, burst stays within 32 KiB budget, torn-tail marker/length detection, unknown-category passthrough, >120-byte detail truncated-not-rejected in `firmware/test_apps/host/main/test_data_storage.cpp` + +### Implementation for User Story 3 + +- [x] T024 [US3] Implement event-log part of `LittleFsDataStorage` per data-model.md (`/events/0.log`+`1.log`, 0xE7-framed records, 16 KiB cap, truncate-and-switch rotation, 120-byte detail truncation) in `firmware/components/storage/src/LittleFsDataStorage.cpp` +- [x] T025 [US3] Run US3 suite green on linux target — NOTE: execution deferred to the main session (implementer agent does not run docker builds) + +**Checkpoint**: All host-testable behavior (US1–US3) green + +--- + +## Phase 6: Concurrency decorators (FR-013, cross-story) + +**Goal**: Safe cross-task use without polluting the unsynchronized base implementations (research D9, PR-02 CP3 precedent) + +- [x] T026 [P] Header-only `LockedConfigStore` decorator over `IConfigStore` in `firmware/components/storage/include/storage/LockedConfigStore.h` +- [x] T027 [P] Header-only `LockedDataStorage` decorator over `IDataStorage` in `firmware/components/storage/include/storage/LockedDataStorage.h` +- [x] T028 Concurrency host tests (delegation correctness for every method; mutex-held invariants per the mechanism PR-02's `LockedWaterPump` tests established) in `firmware/test_apps/host/main/test_data_storage.cpp` and `test_config_store.cpp`; suites green — NOTE: suite execution deferred to the main session (implementer agent does not run docker builds) + +--- + +## Phase 7: User Story 4 — Storage health is visible (Priority: P3, target integration) + +**Goal**: Mount-or-format at boot, usage reporting, build-time partition image, HIL readiness (FR-007, FR-008) + +**Independent Test**: [HIL] quickstart.md checklist on the rev1 rig (fresh flash → format+mount+usage; reboot persistence; corruption recovery via 0x2000 superblock erase) + +### Implementation for User Story 4 + +- [x] T029 [P] [US4] Target-only `StorageMount` (esp_vfs_littlefs_register with `partition_label="storage"`, `base_path="/storage"`, `format_if_mount_failed=true`; `esp_littlefs_info` stats provider wired into `LittleFsDataStorage`) in `firmware/components/storage/include/storage/StorageMount.h` + `firmware/components/storage/src/StorageMount.cpp` (research D2/D4; excluded from linux build per T002) +- [x] T030 [P] [US4] Partition image in build: `littlefs_create_partition_image(storage ../storage_image FLASH_IN_PROJECT)` in `firmware/main/CMakeLists.txt` (research D1) +- [x] T031 [US4] Boot wiring in `firmware/main/`: NVS init (with the standard erase-on-`NO_FREE_PAGES/NEW_VERSION` recovery), StorageMount at startup, one-line usage log (parity: serial status block) — keep ESP_LOG only, no business logic in main +- [x] T032 [US4] HIL verification path for config persistence and factory reset on the rig (follow the verification mechanism PR-02 established for its HIL pass — extend it, don't invent a parallel one) in `firmware/main/` or the PR-02 test console location +- [x] T033 [US4] CI: add `test -f firmware/build/storage.bin` to the verify-binaries step in `.github/workflows/firmware-build.yml`; confirm host-test job picks up the new suites; both target builds + host job green in CI + +**Checkpoint**: Feature complete pending HIL sign-off at Checkpoint 3 + +--- + +## Phase 8: Polish & Cross-Cutting + +- [x] T034 [P] Update `docs/parity-checklist.md` §6: bounded history format (D6), settable interval items, dropped `getLastSensorReading`/`pruneOldReadings`, redesigned split contracts, event log as new surface, WiFi-unconfigured representation change (legacy `CONFIGURE_ME` sentinel in `/wifi_config.json` → empty-string NVS factory state) — each marked as deliberate divergence with rationale (spec FR-010/FR-012/FR-014) +- [x] T035 [P] Update `firmware/CLAUDE.md` component list with the `storage` component and host-test pointers +- [x] T036 Run full quickstart.md validation (both target builds + storage.bin check + host suite) in the pinned container; deliver test checklist incl. HIL items for Checkpoint 3 + +--- + +## Dependencies & Execution Order + +- **Phase 1 → Phase 2 → stories**: T001 gates everything (PR-02 merge). T004–T005 gate all test tasks. +- **US1 (Phase 3)**: T006 → (T007, T008) → T009–T012 [P] → T013 → T014. Independent of US2–US4. **MVP.** +- **US2 (Phase 4)**: T015 → (T016, T017) → T018–T020 [P] → T021 → T022. Independent of US1 (different files; both stories append to suite files created in T005 — coordinate or sequence suite-file edits). +- **US3 (Phase 5)**: T023 → T024 → T025; depends on US2's T015/T017/T021 (same class/files) — sequential after US2. +- **Phase 6 (FR-013)**: T026/T027 depend on T006/T015 (interfaces); T028 after both. Can run any time after the interfaces exist; placed here to decorate finished implementations. +- **US4 (Phase 7)**: T029 depends on T021/T024 (stats provider into LittleFsDataStorage); T031 depends on T013/T029; T033 last. +- **Polish (Phase 8)**: after all stories; T034/T035 parallel. + +### Parallel opportunities + +- T006+T007 then T009–T012 (US1 tests) in parallel. +- T015+T016 then T018–T020 (US2 tests) in parallel. +- US1 and US2 phases can run concurrently after Phase 2 (coordinate the shared suite files). +- T026+T027, T029+T030, T034+T035 in parallel. + +## Implementation Strategy + +MVP = Phase 1–3 (config store host-verified). Incremental: each story phase ends +green and independently testable; stop at any checkpoint. Suggested single-agent +order: T001→T005, US1, US2, US3, decorators, US4, polish — with a commit after +each phase checkpoint (one branch, commit immediately per CLAUDE.md git rules).