Hi folks,
I have a question regarding the use of a Checkpointer when using the Event Hubs Stream Provider.
For context, I'm designing a system where a multi-silo Orleans instance hosted in AKS will be the primary owner of some device data. Separately, I'll also have around a hundred sites with on-prem single-silo instances of Orleans running for edge processing of ingested data. Periodically, I need to synchronise mapping data for specific devices from the cloud instance down to the corresponding on-prem instances, and ideally they should be synchonised in near real-time (i.e. within a second or so). I'm hoping to use the Event Hubs Stream Provider for this, and then have each on-prem instance subscribe to events from it's own subset of Device IDs (around 20 - 30 devices).
I noticed in the code sample on the Event Hubs Stream Provider Docs that it also configures an Azure Table Checkpointer for the stream provider. I'll have a local instance of SQL Server on-prem which I plan to use for grain storage, reminders etc, however I won't have Azure Storage available locally. So I have two questions:
- Are there any other implementations of the Checkpointer (e.g. an ADO.Net one) which would avoid me calling Azure Storage from on-prem?
- What are the implications if I don't configure a Checkpointer for Event Hubs Streaming?
Any guidance would be much appreciated, thanks!
Paul
Hi folks,
I have a question regarding the use of a Checkpointer when using the Event Hubs Stream Provider.
For context, I'm designing a system where a multi-silo Orleans instance hosted in AKS will be the primary owner of some device data. Separately, I'll also have around a hundred sites with on-prem single-silo instances of Orleans running for edge processing of ingested data. Periodically, I need to synchronise mapping data for specific devices from the cloud instance down to the corresponding on-prem instances, and ideally they should be synchonised in near real-time (i.e. within a second or so). I'm hoping to use the Event Hubs Stream Provider for this, and then have each on-prem instance subscribe to events from it's own subset of Device IDs (around 20 - 30 devices).
I noticed in the code sample on the Event Hubs Stream Provider Docs that it also configures an Azure Table Checkpointer for the stream provider. I'll have a local instance of SQL Server on-prem which I plan to use for grain storage, reminders etc, however I won't have Azure Storage available locally. So I have two questions:
Any guidance would be much appreciated, thanks!
Paul