To reduce database load, certain things could be stored in memory only. These are:
- quota usage - is fetched from the servers on management start anyway
- nic lists - is completely replaced with each heartbeat. in this case, management should request heartbeats from all known nodes on startup to fill this without delay.
Fortunately, it is not only possible to attach a memory database to an existing connection, but also share it between connections (within the process). This is important when we implement a db connection pool in ThinkParQ/beegfs-rs#85 . The old, disregarded way, is using ?cache=shared, but as far as I could figure out, this doesn't work with WAL mode. There is a relatively new feature (3 years) that I think should work with it? Need to try that out.
However, if the load ever becomes a problem, it might also be sufficient to instead carefully opt out of fsyncing after certain operations. The nic list update when receiving a heartbeat already does this.
To reduce database load, certain things could be stored in memory only. These are:
Fortunately, it is not only possible to attach a memory database to an existing connection, but also share it between connections (within the process). This is important when we implement a db connection pool in ThinkParQ/beegfs-rs#85 . The old, disregarded way, is using
?cache=shared, but as far as I could figure out, this doesn't work with WAL mode. There is a relatively new feature (3 years) that I think should work with it? Need to try that out.However, if the load ever becomes a problem, it might also be sufficient to instead carefully opt out of fsyncing after certain operations. The nic list update when receiving a heartbeat already does this.