Add database summary and comparison utilities for symmetry testing#115
Merged
Add database summary and comparison utilities for symmetry testing#115
Conversation
Introduce symmetry test infrastructure following the NDI-matlab pattern, enabling cross-language validation between MATLAB and Python DID implementations. - makeArtifacts: generates a small DID database with random demoA/demoB/demoC documents across 3 branches (branch_main, branch_dev, branch_feature), exports the SQLite database file and per-branch JSON audit files as persistent artifacts - readArtifacts: parameterized test that reads artifacts from either matlabArtifacts or pythonArtifacts, validates document counts, class names, field values, and dependencies against the JSON audit files - INSTRUCTIONS.md files document the directory conventions and artifact format https://claude.ai/code/session_01TT3ycsjcvsKLnAp1WffRPe
Introduce reusable utilities for summarizing and comparing DID databases, then refactor symmetry tests to use them: - did.util.databaseSummary(db): produces a struct capturing the full database state (branches, hierarchy, documents sorted by ID with class names and properties), suitable for JSON serialization - did.util.compareDatabaseSummary(a, b): compares two summaries (structs, JSON files, or database objects) and returns a report with isEqual flag and detailed per-branch/per-document mismatch messages - makeArtifacts now exports summary.json via databaseSummary and includes a self-check round-trip comparison - readArtifacts now loads summary.json, re-summarizes the live database, and uses compareDatabaseSummary for the primary validation https://claude.ai/code/session_01TT3ycsjcvsKLnAp1WffRPe
Update compareDatabaseSummary to return a cell array of message strings (empty = equal) instead of a struct with .isEqual, matching the pattern used by ndi.util.compareSessionSummary on the NDI-matlab side. Also add arguments block with excludeFields option and use sprintf for messages. Update symmetry tests to use verifyEmpty(report) accordingly. https://claude.ai/code/session_01TT3ycsjcvsKLnAp1WffRPe
Runs makeArtifacts tests first (generating the database and JSON audit artifacts in tempdir), then readArtifacts tests (validating the artifacts against the live database). Triggered on pushes to main, PRs to main, or manually via workflow_dispatch. https://claude.ai/code/session_01TT3ycsjcvsKLnAp1WffRPe
Contributor
Test Results104 tests 102 ✅ 1m 14s ⏱️ Results for commit 294982b. ♻️ This comment has been updated with latest results. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #115 +/- ##
==========================================
+ Coverage 44.15% 45.63% +1.47%
==========================================
Files 53 55 +2
Lines 2976 3158 +182
==========================================
+ Hits 1314 1441 +127
- Misses 1662 1717 +55 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
The sqlitedb implementation requires the mksqlite package. Add steps to install MatBox first, then use matbox.installRequirements to install mksqlite and other dependencies from requirements.txt. https://claude.ai/code/session_01TT3ycsjcvsKLnAp1WffRPe
stevewds
approved these changes
Mar 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces utilities for generating and comparing DID database summaries, along with a comprehensive cross-language symmetry testing framework. These tools enable validation of database state consistency across MATLAB and other DID implementations (e.g., Python).
Key Changes
New Utility Functions
did.util.databaseSummary(db): Generates a deterministic struct summarizing a DID database's complete state, including:did.util.compareDatabaseSummary(summaryA, summaryB, options): Compares two database summaries and returns a detailed report of differences:did.databaseobjectsexcludeFieldsoptionSymmetry Testing Framework
tests/+did/+symmetry/+makeArtifacts/+database/buildDatabase.m: Test class that generates standard DID database artifacts:tempdir()compareDatabaseSummary()tests/+did/+symmetry/+readArtifacts/+database/buildDatabase.m: Test class that validates artifacts from both MATLAB and external implementations:matlabArtifactsandpythonArtifactsGitHub Actions Workflow (
.github/workflows/test-symmetry.yml): Automated CI pipeline that:makeArtifactstests to generate reference artifactsreadArtifactstests to validate consistencyDocumentation
INSTRUCTIONS.mdfiles in bothmakeArtifactsandreadArtifactspackages explaining:Implementation Details
matlab.unittestframework with proper fixtures and parameterization for maintainabilityhttps://claude.ai/code/session_01TT3ycsjcvsKLnAp1WffRPe