Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 5 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,21 +38,11 @@ Production‑grade event‑driven autocomplete system built with Kafka Streams,

## Architecture Flow

```mermaid
flowchart LR
UI[Frontend] -->|GET /api/search?q=java| SC[SearchController]
SC --> SEP[SearchEventProducer]
SEP -->|search-events| K[(Kafka)]
K --> SST[SearchStatsTopology]
SST --> PG[(PostgreSQL search_stats)]
PG -->|CDC| DBZ[Debezium Connect]
DBZ -->|db-changes.public.search_stats| DC[DebeziumConsumer]
DC --> RSU[RedisSearchUpdater]
RSU --> R[(Redis keys autocomplete:*)]
UI -->|GET /api/complete?q=ja| AC[AutocompleteController]
AC --> AQS[AutocompleteQueryService]
AQS --> R
```
![Autocomplete system context diagram](docs/diagrams/context-autocomplete-system.png)

Source: `docs/diagrams/context-autocomplete-system.puml`

Detailed sequence diagrams: [`docs/architecture.md`](docs/architecture.md)

Core behavior:

Expand Down
21 changes: 21 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Architecture

Diagrams are stored in `docs/diagrams/` as PlantUML source and exported PNG files.

## Context: Event-driven autocomplete pipeline

![Autocomplete system context diagram](./diagrams/context-autocomplete-system.png)

Source: `docs/diagrams/context-autocomplete-system.puml`

## Sequence: search ingestion and indexing

![Search ingestion and indexing sequence](./diagrams/sequence-search-ingestion.png)

Source: `docs/diagrams/sequence-search-ingestion.puml`

## Sequence: autocomplete query flow

![Autocomplete query flow sequence](./diagrams/sequence-autocomplete-query.png)

Source: `docs/diagrams/sequence-autocomplete-query.puml`
Binary file added docs/diagrams/context-autocomplete-system.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
37 changes: 37 additions & 0 deletions docs/diagrams/context-autocomplete-system.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
@startuml
title Autocomplete System - Context Diagram

skinparam backgroundColor #FFFFFF
skinparam packageStyle rectangle
skinparam shadowing false
skinparam defaultTextAlignment center

actor "User" as user

rectangle "Autocomplete System" as system {
rectangle "Frontend\n(Angular + Nginx, :4200)" as frontend
rectangle "Search Service\n(Spring Boot, :8082)" as search
rectangle "CDC Service\n(Spring Boot, :8084)" as cdc
rectangle "Autocomplete Service\n(Spring Boot, :8081)" as autocomplete
}

queue "Kafka\n(search-events, search-stats,\ndb-changes.public.search_stats)" as kafka
database "PostgreSQL\n(search_stats)" as postgres
database "Redis\n(autocomplete:<prefix>)" as redis
rectangle "Debezium Connect\n(:8083)" as debezium

user --> frontend : Submit query / request suggestions
frontend --> search : GET /api/search?q=...
search --> kafka : Publish search-events
kafka --> search : Kafka Streams aggregate
search --> postgres : Upsert query frequency
Comment thread
igorsatsyuk marked this conversation as resolved.
search --> kafka : Publish search-stats
postgres --> debezium : WAL CDC
debezium --> kafka : Emit db-changes.public.search_stats
kafka --> cdc : Consume Debezium envelope
cdc --> redis : Update prefix sorted sets
frontend --> autocomplete : GET /api/complete?q=...&limit=...
autocomplete --> redis : Read top suggestions
autocomplete --> frontend : Ranked suggestions

@enduml
Binary file added docs/diagrams/sequence-autocomplete-query.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 20 additions & 0 deletions docs/diagrams/sequence-autocomplete-query.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
@startuml
title Autocomplete Query Flow

actor User
participant "Frontend\n:4200" as Frontend
participant "AutocompleteController\nautocomplete-service :8081" as Controller
participant "AutocompleteQueryService" as QueryService
database "Redis\nautocomplete:<prefix>" as Redis

User -> Frontend: Type prefix "ja"
Frontend -> Controller: GET /api/complete?q=ja&limit=10
Controller -> QueryService: getSuggestions("ja", 10)
QueryService -> QueryService: trim + lowercase validation
QueryService -> Redis: ZREVRANGE autocomplete:ja 0 9
Redis --> QueryService: ["java", "javascript", ...]
QueryService --> Controller: Suggestion list
Controller --> Frontend: 200 OK + JSON array
Frontend --> User: Render ranked suggestions

@enduml
Binary file added docs/diagrams/sequence-search-ingestion.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
31 changes: 31 additions & 0 deletions docs/diagrams/sequence-search-ingestion.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
@startuml
title Search Ingestion and Redis Indexing Flow

actor User
participant "Frontend\n:4200" as Frontend
participant "SearchController\nsearch-service :8082" as SearchController
participant "SearchEventProducer" as Producer
queue "Kafka topic\nsearch-events" as SearchEvents
participant "SearchStatsTopology" as Topology
database "PostgreSQL\nsearch_stats" as Postgres
participant "Debezium Connect\n:8083" as Debezium
queue "Kafka topic\ndb-changes.public.search_stats" as DbChanges
participant "DebeziumConsumer\ncdc-service :8084" as Consumer
participant "RedisSearchUpdater" as Updater
database "Redis\nautocomplete:<prefix>" as Redis

User -> Frontend: Type search query
Frontend -> SearchController: GET /api/search?q=java
SearchController -> Producer: sendSearchEvent("java")
Producer -> SearchEvents: Produce search event
SearchEvents -> Topology: Consume event
Topology -> Topology: trim + lowercase + count
Topology -> Postgres: Upsert query frequency
Postgres -> Debezium: Capture row change (CDC)
Debezium -> DbChanges: Publish payload.after event
DbChanges -> Consumer: Consume CDC record
Consumer -> Updater: updateFromAfter(query, frequency)
Updater -> Updater: Generate prefixes (j, ja, jav, java)
Updater -> Redis: ZADD autocomplete:<prefix> query score

@enduml
Loading