[WIP] Implement snapshots

[TanmayArya-1p]

Addresses #38

Currently based on top of the kd_tree_refactor branch.

TODO:

• Implement serialization and deserialization for FlatIndex
  • Write and pass unit tests
• Implement serialization and deserialization for KDTree
  • Write and pass unit tests
• Implement serialization and deserialization for HNSWIndex - waiting on other restructuring changes
  • Write and pass unit tests
• Implement a checkpoint system for StorageEngine (only rocksdb as of now)
  • Write and pass unit tests
• Implement an API in the snapshot crate that creates and restores a snapshot tarball including: StorageEngine checkpoint, snapshot of Index topology from the serialized bytes, snapshot of the metadata of the Index, manifest.json containing checksums and other information needed for restore.
  • Write and pass unit tests
• Create a SnapshotRegistry that is responsible for storing and managing snapshot files. - Currently LocalRegistry is implemented which just stores the snapshots in a directory but in the future a RemoteRegistry can be created that implements the SnapshotRegistry trait.
• Implement a SnapshotEngine that manages a registry of snapshots and spawns a worker thread that periodically takes snapshots of the database and adds it to the registry.
  • Write and pass unit tests
• Add methods in the api crate to take and restore snapshots
  • Write and pass unit tests
• Implement RPCs in the grpc crate and configuration options that actually exposes these methods to users.

[Arshdeep54]

Open a separate PR for hnsw index(checked out from hnsw branch)

[TanmayArya-1p]

Opened another PR (#67) from a cleaner branch that is up to date with refactor