@@ -21,57 +21,76 @@ This plugin extends the workflow engine with:
2121
2222The plugin runs as a subprocess. The host workflow engine launches it on startup and communicates over a local gRPC socket managed by the go-plugin framework.
2323
24- ```
25- ┌─────────────────────────────────────────────────────────────────┐
26- │ Host Process (workflow engine) │
27- │ │
28- │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
29- │ │ HTTP/Event │ │ EventBus │ │ Module Registry │ │
30- │ │ Triggers │───>│ (broker) │<───│ bento.input │ │
31- │ └──────────────┘ └──────┬───────┘ │ bento.output │ │
32- │ │ │ bento.stream │ │
33- │ │ │ bento.broker │ │
34- │ │ └──────────────────┘ │
35- │ │ │
36- └──────────────────────────────┼──────────────────────────────────┘
37- │ go-plugin gRPC (IPC)
38- ┌──────────────────────────────┼──────────────────────────────────┐
39- │ Plugin Subprocess (bento) │ │
40- │ │ │
41- │ ┌──────────────────────────▼───────────────────────────────┐ │
42- │ │ BentoPlugin │ │
43- │ │ ┌─────────────┐ ┌──────────────┐ ┌───────────────┐ │ │
44- │ │ │ Bento Input │ │ Bloblang │ │ Bento Output │ │ │
45- │ │ │ (consumer) │ │ Processors │ │ (producer) │ │ │
46- │ │ └──────┬──────┘ └──────────────┘ └───────┬───────┘ │ │
47- │ └─────────┼─────────────────────────────────────┼──────────┘ │
48- │ │ │ │
49- └─────────────┼─────────────────────────────────────┼─────────────┘
50- │ │
51- External Source External Sink
52- (Kafka / SQS / HTTP / ...) (S3 / Pub/Sub / NATS / ...)
24+ ``` mermaid
25+ graph TB
26+ subgraph HostProcess["Host Process (workflow engine)"]
27+ Triggers["HTTP/Event Triggers"]
28+ EventBus["EventBus (broker)"]
29+ subgraph ModuleRegistry["Module Registry"]
30+ BentoInputMod["bento.input"]
31+ BentoOutputMod["bento.output"]
32+ BentoStreamMod["bento.stream"]
33+ BentoBrokerMod["bento.broker"]
34+ end
35+ Triggers --> EventBus
36+ BentoInputMod --> EventBus
37+ BentoOutputMod --> EventBus
38+ BentoStreamMod --> EventBus
39+ BentoBrokerMod --> EventBus
40+ end
41+
42+ EventBus <-->|"go-plugin gRPC (IPC)"| BentoPlugin
43+
44+ subgraph PluginSubprocess["Plugin Subprocess (bento)"]
45+ subgraph BentoPlugin["BentoPlugin"]
46+ BInput["Bento Input (consumer)"]
47+ Bloblang["Bloblang Processors"]
48+ BOutput["Bento Output (producer)"]
49+ BInput --> Bloblang --> BOutput
50+ end
51+ end
52+
53+ ExtSource["External Source\n(Kafka / SQS / HTTP / ...)"] --> BInput
54+ BOutput --> ExtSink["External Sink\n(S3 / Pub/Sub / NATS / ...)"]
5355```
5456
5557### Message Flow
5658
5759** Input path** (` bento.input ` ):
58- ```
59- External Source -> Bento Input -> go-plugin gRPC -> Host EventBus -> Workflow Handler
60+
61+ ``` mermaid
62+ flowchart LR
63+ ExtSource["External Source"] --> BentoInput["Bento Input"]
64+ BentoInput --> gRPC["go-plugin gRPC"]
65+ gRPC --> HostEB["Host EventBus"]
66+ HostEB --> WorkflowHandler["Workflow Handler"]
6067```
6168
6269** Output path** (` bento.output ` ):
63- ```
64- Workflow Handler -> Host EventBus -> go-plugin gRPC -> Bento Output -> External Sink
70+
71+ ``` mermaid
72+ flowchart LR
73+ WorkflowHandler["Workflow Handler"] --> HostEB["Host EventBus"]
74+ HostEB --> gRPC["go-plugin gRPC"]
75+ gRPC --> BentoOutput["Bento Output"]
76+ BentoOutput --> ExtSink["External Sink"]
6577```
6678
6779** Full stream** (` bento.stream ` ):
68- ```
69- External Source -> Bento Pipeline (Bloblang) -> External Sink
80+
81+ ``` mermaid
82+ flowchart LR
83+ ExtSource["External Source"] --> BentoPipeline["Bento Pipeline (Bloblang)"]
84+ BentoPipeline --> ExtSink["External Sink"]
7085```
7186
7287** Inline step** (` step.bento ` ):
73- ```
74- Workflow Step Input -> go-plugin gRPC -> Bento Processors (Bloblang) -> Workflow Step Output
88+
89+ ``` mermaid
90+ flowchart LR
91+ StepInput["Workflow Step Input"] --> gRPC["go-plugin gRPC"]
92+ gRPC --> Processors["Bento Processors (Bloblang)"]
93+ Processors --> StepOutput["Workflow Step Output"]
7594```
7695
7796## Installation
0 commit comments