[Technical] Revisit metadata / snapshot storage design

[eblanshey]

Snapshots are currently created by generating a FreeCAD model tree from open documents, and when the FreeCAD file is marked as Reviewed by the user, the snapshot is persisted to disk as a yaml file in the .snapshots directory. This file is added and committed to git alongside FreeCAD documents. The next time the FreeCAD document is updated, the same snapshot file is overwritten.

This approach allows the FreeCAD tree to be reconstructed for any historical FreeCAD file without having to open it, enabling faster diff generation, as well as not requiring currently open documents to be closed. There are, however, several drawbacks to this approach:

1. When the snapshot structure evolves, old committed snapshots will be out of date.
   • Not every property is currently stored in the snapshot, for example. If the snapshot is updated to start storing a new property type, diffs between old commits and newer commits will show an addition of a new property even if it actually existed in the old document.
   • Existing properties could have their values or value types changed in the snapshot to support smarter diffing
2. New users of the workbench with existing git repositories can't generate diffs for existing commits.
3. Other metadata can't be added to historical commits, such as screenshots

To better support the above use-cases, a new solution should support the ability to recreate historical snapshots via a dedicated command, without it affecting historical project commits. The following approaches can be considered:

• Continue storing YAML snapshot files in .snapshots, but don't overwrite previous versions. Each commit-file combination will have its own snapshot file. Benefits:
  • This directory can optionally be gitignored, as the history can be regenerated for each commit.
  • Note: would need a way to associate FreeCAD files with specific snapshots, as snapshots are made before a commit hash is generated.
• Store YAML snapshots in another git branch, out of view of the user. Git can essentially be used as a file database. Benefits:
  • No .snapshots directory visible to the user.
  • Snapshot branch can be updated after FCStd files are committed, making commit hashes available as a snapshot index
• Store snapshots in an sqlite database or other method

Feedback/thoughts welcome.

[eblanshey]

Update: based on how the snapshot logic will be changed in #9, this task will be simplified. Extended snapshots will simply be stored in a new gitignored directory, such as .history-metadata. Alternatively (or additionally) we can have the option of storing this file metadata in the central freecad cache directory.