The current design is to scrape the data from the agent using fluentbit. However, this might be a problem when a network partition happens and the data is not properly scrapped.
How do we use MQTT?
The naive way is to push every event to MQTT and then process it on the server, this way is very inefficient.
Another way to do it is to push the aggregated metric similar to Prometheus /metric, if fluentbit can consume from MQTT in this format, that will be really great. Or may be we can explore fluentd if that's not possible.
The current design is to scrape the data from the agent using fluentbit. However, this might be a problem when a network partition happens and the data is not properly scrapped.
How do we use MQTT?
The naive way is to push every event to MQTT and then process it on the server, this way is very inefficient.
Another way to do it is to push the aggregated metric similar to Prometheus /metric, if fluentbit can consume from MQTT in this format, that will be really great. Or may be we can explore fluentd if that's not possible.