Barman Cloud compatibility: cache the (static) Barman Cloud catalog locally on enablement

[gbartolini]

Problem

Both the WAL fallback and the base-backup fallback need repeated, cheap access to what exists in a Barman Cloud archive: its backup list and metadata, and the extent of its archived WAL. Barman Cloud stops
receiving writes the moment Klio takes over forward archiving, so from that point the archive is effectively a static, one-time snapshot. Re-listing and re-parsing the object store on every lookup is wasted work against data that isn't changing, and needlessly increases latency and object-store request volume.

Requirements

• The feature is opt-in, not auto-detected
• On enablement, Klio builds a local cache of the Barman Cloud catalog (backup list with metadata, WAL archive extent) once, rather than querying object storage per WAL or backup lookup. Compression can be inferred by the name of the files at that point, using one of the known formats of Barman Cloud.
• The cache design must assume the underlying Barman Cloud archive is static after cutover, and that assumption must be explicit and checkable, not silent.
• This work introduces no write against Barman Cloud objects. The only write path against Barman Cloud objects, now or in future, is retention policy enforcement (tracked separately): the cache must be designed so a future retention pass can invalidate or update it correctly, rather than leaving it to serve entries for backups that have since been deleted.

Verification

• Enabling the feature against a real Barman Cloud archive builds a correct local cache of its backup list and WAL extent in a single pass, with no repeated re-listing.
• Subsequent WAL and backup lookups are served from the cache, with no additional object-store LIST/GET calls beyond fetching the actual WAL/backup payload.
• With the feature disabled, no cache is built and no extra object-store calls occur.
• The static-catalog assumption is explicit: behaviour if a new object appears in the Barman Cloud archive after the cache is built is deliberately decided and tested, not left undefined.
• No write, delete, or pin operation is ever issued against Barman Cloud objects by this feature, confirmed by test.

Open questions

• Where does the cache live, and does it survive a Klio server restart, or get rebuilt on enablement each time? Shall it be renewed at regular intervals (configurable)?
• Correct behaviour if the static-catalog assumption is violated (new Barman Cloud writes appear post-cutover): reject, warn, or refresh?
• How does the future retention ticket invalidate/update this cache once it starts deleting Barman Cloud objects?

Out of scope

• Retention policy enforcement against Barman Cloud objects (separate, future ticket). This ticket only needs to leave room for it.

Alternative approach

An alternative approch to be evaluated in analysis is the possibility to convert the existing base backups and WAL files in the Klio format, using a queue that transforms them, encrypts them, and migrates them.