Automatic cleanup / decrease of no longer used Pools / Test-DBs

[majodev]

Automatically cleanup x previous template databases and it's test databases after a new template database was requested, without manually invoking the DELETE /templates endpoint.

Enable this handling via the env var INTEGRESQL_KEEP_PREVIOUS_POOLS_COUNT (int), default 0 (unlimited).

(Initial proposal via @MorpheusXAUT , cc aaa_internal)

[majodev]

I consider this to be the most critical feature for the next release (v1.2.x). As mentioned in the CHANGELOG.md:

• We still have no mechanism to limit the global (cross-pool) number of test-databases.
  • This is especially problematic if you have many pools running at the same time.
  • This could lead to situations where the pool manager is unable to create a new test-databases because the limit (e.g. disk size) is reached even tough some pools/test-databases will probably never be used again.
  • This is a known issue and will be addressed in a future release.

Specifically, here are some first thoughts about it:

• I want that downscaling to happen in a dynamic manner, especially for test databases. Meaning, after a certain time of no active (/ or even very limited) pool usage, a kind of descaler worker should start to do its work:
  • Reduce the always ready test database count in the unused pool for the incoming next minutes
  • Reduce the number of dirty databases in the unused pool for the incoming next minutes
  • Both all the way down to 0
• Full pool removal / template database removal is more destructive, but after a certain time (e.g. full 24h) - why not, it's just testing anyways.
• It might be hard to come up with good defaults that work for small and big (template / test) databases. Therefore a quite important metric to include in the descaling behavior might be the target maximal size (MB vs. GB) to use for all databases.
• Anyways, all values of this de-scaler have to be configurable.