From e82b977aa06a8d1520e85dad5ef42f1c717277f0 Mon Sep 17 00:00:00 2001
From: Tillman Mosley III <tillman@withobsrvr.com>
Date: Fri, 28 Nov 2025 08:23:22 -0500
Subject: [PATCH] Update processors and consumers

---
 docs/flow/consumers/index.md  | 805 ++++++++++++++++++++++++++++++----
 docs/flow/processors/index.md | 750 ++++++++++++++++++++++++++++---
 2 files changed, 1412 insertions(+), 143 deletions(-)

diff --git a/docs/flow/consumers/index.md b/docs/flow/consumers/index.md
index 7c6d410..6b3149f 100644
--- a/docs/flow/consumers/index.md
+++ b/docs/flow/consumers/index.md
@@ -9,92 +9,747 @@ Consumers are the destination components in Flow pipelines that receive processe
 
 ## Available Consumers
 
-### Database Storage
-
-#### [PostgreSQL](./postgresql.md)
-The most versatile consumer for structured data storage. Features:
-- Multiple schema options (generic, specialized)
-- Batch processing for efficiency
-- Automatic table creation
-- Connection pooling
-
-**Use Cases:**
-- Analytics databases
-- Application backends
-- Historical data storage
-- Reporting systems
-
-#### Redis
-High-performance in-memory storage for real-time access:
-- Latest ledger information
-- Real-time balance tracking
-- Cache layer for applications
-- Pub/sub capabilities
-
-#### DuckDB
-Embedded analytical database perfect for:
-- Local analytics
-- Data science workflows
-- No infrastructure required
-- Columnar storage efficiency
-
-#### SQLite
-Lightweight embedded database ideal for:
-- Development and testing
-- Edge deployments
-- Single-user applications
-- Portable data storage
+### Database Storage - PostgreSQL
+
+#### [PostgreSQL (Generic)](./postgresql.md)
+General-purpose PostgreSQL consumer for ledger data with flexible JSON format.
+
+**Configuration:**
+```yaml
+type: postgres
+config:
+  connection_string: "postgresql://user:pass@host:5432/database"
+  batch_size: 50
+```
+
+**Use Cases:** Analytics databases, application backends, historical data storage
+
+---
+
+#### PostgreSQL Bronze
+Bronze layer data ingestion to PostgreSQL for data lakehouse architecture.
+
+**Configuration:**
+```yaml
+type: postgres_bronze
+config:
+  host: "localhost"
+  database: "bronze"
+  username: "user"
+  password: "password"
+```
+
+**Use Cases:** Raw data layer storage, medallion architecture
+
+---
+
+#### Buffered PostgreSQL
+Buffered batch inserts with retry logic for high-throughput scenarios.
+
+**Configuration:**
+```yaml
+type: buffered_postgres
+config:
+  buffer_size: 1000
+  flush_interval_ms: 5000
+  max_retries: 3
+```
+
+**Use Cases:** High-volume data ingestion with batching
+
+---
+
+#### Account Data PostgreSQL
+Stores Stellar account data with full metadata.
+
+**Configuration:**
+```yaml
+type: account_data_postgres
+config:
+  host: "localhost"
+  port: 5432
+  database: "accounts"
+  max_open_conns: 20
+```
+
+**Use Cases:** Account state tracking and historical analysis
+
+---
+
+#### Asset PostgreSQL
+Stores asset/ticker information with batching.
+
+**Configuration:**
+```yaml
+type: asset_postgres
+config:
+  batch_size: 100
+  connection_string: "postgresql://user:pass@host:5432/assets"
+```
+
+**Use Cases:** Asset catalog and enrichment data storage
+
+---
+
+#### Asset Enrichment PostgreSQL
+Updates asset enrichment data (TOML, auth flags).
+
+**Configuration:**
+```yaml
+type: asset_enrichment
+config:
+  connection_string: "postgresql://user:pass@host:5432/assets"
+```
+
+**Use Cases:** Asset metadata enrichment from stellar.toml
+
+---
+
+#### Payments PostgreSQL
+Batched payment transaction storage.
+
+**Configuration:**
+```yaml
+type: payments_postgres
+config:
+  batch_size: 1000
+  connection_string: "postgresql://user:pass@host:5432/payments"
+```
+
+**Use Cases:** Payment history and analytics
+
+---
+
+#### Event Payment PostgreSQL
+Event-based payment storage with account relationships.
+
+**Configuration:**
+```yaml
+type: event_payment_postgres
+config:
+  host: "localhost"
+  database: "payments"
+  max_open_conns: 20
+```
+
+**Use Cases:** Event-driven payment tracking with foreign keys
+
+---
+
+#### Contract Events PostgreSQL
+Soroban contract events storage.
+
+**Configuration:**
+```yaml
+type: contract_events_postgres
+config:
+  host: "localhost"
+  database: "soroban_events"
+```
+
+**Use Cases:** Smart contract event logging and analysis
+
+---
+
+#### Contract Invocations PostgreSQL
+Contract invocations with dual XDR/decoded format, diagnostic events, state changes.
+
+**Configuration:**
+```yaml
+type: contract_invocations_postgres
+config:
+  host: "localhost"
+  database: "contract_invocations"
+```
+
+**Use Cases:** Complete contract execution tracking
+
+---
+
+#### Extracted Contract Invocations PostgreSQL
+Business-logic extracted contract data (funder, recipient, amount, project_id).
+
+**Configuration:**
+```yaml
+type: extracted_contract_invocations_postgres
+config:
+  host: "localhost"
+  database: "business_data"
+```
+
+**Use Cases:** Business intelligence from smart contracts
+
+---
+
+#### Claimable Balance PostgreSQL
+Claimable balance tracking with claimants.
+
+**Configuration:**
+```yaml
+type: claimable_balance_postgres
+config:
+  database_url: "postgresql://user:pass@host:5432/claimable_balances"
+```
+
+**Use Cases:** Claimable balance lifecycle management
+
+---
+
+#### Soroswap PostgreSQL
+Soroswap DEX events (pairs, swaps, liquidity).
+
+**Configuration:**
+```yaml
+type: soroswap_postgres
+config:
+  host: "localhost"
+  database: "soroswap"
+```
+
+**Use Cases:** Soroswap DEX analytics
+
+---
+
+#### Wallet Backend PostgreSQL
+Wallet backend with state changes and participants.
+
+**Configuration:**
+```yaml
+type: wallet_backend_postgres
+config:
+  host: "localhost"
+  database: "wallet"
+```
+
+**Use Cases:** Wallet application backend database
+
+---
+
+### Database Storage - DuckDB
+
+#### DuckDB (Generic)
+General DuckDB storage for analytics-friendly columnar format.
+
+**Configuration:**
+```yaml
+type: duckdb
+config:
+  db_path: "/path/to/data.duckdb"
+```
+
+**Use Cases:** Local analytics, data science workflows
+
+---
+
+#### DuckLake
+DuckDB lakehouse pattern with schema registry.
+
+**Configuration:**
+```yaml
+type: ducklake
+config:
+  db_path: "ducklake.duckdb"
+  catalog_name: "main"
+  schema_name: "bronze"
+  table_prefix: "stellar_"
+```
+
+**Use Cases:** Data lakehouse architecture with managed schemas
+
+---
+
+#### DuckLake Enhanced
+Enhanced DuckLake with better performance and features.
+
+**Configuration:**
+```yaml
+type: ducklake_enhanced
+config:
+  db_path: "ducklake_enhanced.duckdb"
+  batch_size: 100
+  flush_interval: 5
+```
+
+**Use Cases:** High-performance data lakehouse
+
+---
+
+#### Bronze DuckDB
+Bronze layer DuckDB ingestion with appenders for medallion architecture.
+
+**Configuration:**
+```yaml
+type: bronze_duckdb
+config:
+  db_path: "bronze.duckdb"
+  batch_size: 100
+  flush_interval_seconds: 10
+```
+
+**Use Cases:** Raw data ingestion for data lakehouse
+
+---
+
+#### Account Data DuckDB
+Account data with schema validation.
+
+**Configuration:**
+```yaml
+type: account_data_duckdb
+config:
+  db_path: "accounts.duckdb"
+```
+
+**Use Cases:** Account analytics in DuckDB
+
+---
+
+#### Assets DuckDB
+Asset catalog with upsert logic.
+
+**Configuration:**
+```yaml
+type: assets_duckdb
+config:
+  db_path: "assets.duckdb"
+```
+
+**Use Cases:** Asset tracking and analytics
+
+---
+
+#### Contract Events DuckDB
+Soroban contract events with analytics views.
+
+**Configuration:**
+```yaml
+type: contract_events_duckdb
+config:
+  db_path: "soroban_events.duckdb"
+```
+
+**Use Cases:** Contract event analytics with built-in views
+
+---
+
+#### Soroswap Pairs DuckDB
+Soroswap pair and sync events.
+
+**Configuration:**
+```yaml
+type: soroswap_pairs_duckdb
+config:
+  db_path: "soroswap_pairs.duckdb"
+```
+
+**Use Cases:** DEX pair tracking and reserves
+
+---
+
+#### Soroswap Router DuckDB
+Soroswap router transactions.
+
+**Configuration:**
+```yaml
+type: soroswap_router_duckdb
+config:
+  db_path: "soroswap_router.duckdb"
+```
+
+**Use Cases:** Router analytics and swap tracking
+
+---
+
+### Database Storage - Other
+
+#### SQLite (Soroswap)
+Lightweight SQLite versions of Soroswap consumers.
+
+**Configuration:**
+```yaml
+type: soroswap_pairs_sqlite
+config:
+  db_path: "soroswap.db"
+```
+
+**Use Cases:** Portable DEX data storage, development
+
+---
+
+#### ClickHouse
+OLAP database with materialized views for real-time analytics.
+
+**Configuration:**
+```yaml
+type: clickhouse
+config:
+  address: "localhost:9000"
+  database: "stellar"
+  username: "default"
+  password: ""
+  max_open_conns: 25
+```
+
+**Use Cases:** Payment stats, price analytics, trustlines, high-volume analytics
+
+---
+
+#### MongoDB
+Document-based storage for flexible schemas.
+
+**Configuration:**
+```yaml
+type: mongodb
+config:
+  uri: "mongodb://localhost:27017"
+  database: "stellar"
+  collection: "transactions"
+```
+
+**Use Cases:** Flexible document storage, semi-structured data
+
+---
+
+#### TimescaleDB
+Time-series database for temporal analytics.
+
+**Configuration:**
+```yaml
+type: timescaledb
+config:
+  host: "localhost"
+  database: "timeseries"
+```
+
+**Use Cases:** Time-series data and analytics
+
+---
+
+### Caching - Redis
+
+#### Redis (Generic)
+Multi-operation Redis storage (payments, accounts, assets, prices, offers, trustlines).
+
+**Configuration:**
+```yaml
+type: redis
+config:
+  redis_url: "redis://localhost:6379"
+  key_prefix: "flow:"
+  ttl_hours: 24
+  use_tls: true
+```
+
+**Use Cases:** Fast lookups, caching, real-time data
+
+---
+
+#### Payments Redis
+Payment-specific Redis storage with indices.
+
+**Configuration:**
+```yaml
+type: payments_redis
+config:
+  redis_url: "redis://localhost:6379"
+  key_prefix: "payment:"
+  ttl_hours: 48
+```
+
+**Use Cases:** Payment caching and fast retrieval
+
+---
+
+#### Latest Ledger Redis
+Latest ledger tracking with history.
+
+**Configuration:**
+```yaml
+type: latest_ledger_redis
+config:
+  redis_url: "redis://localhost:6379"
+  key_prefix: "ledger:"
+  use_tls: true
+```
+
+**Use Cases:** Ledger progress tracking
+
+---
+
+#### Orderbook Redis
+Order book data storage.
+
+**Configuration:**
+```yaml
+type: orderbook_redis
+config:
+  redis_url: "redis://localhost:6379"
+```
+
+**Use Cases:** Trading orderbook caching
+
+---
+
+#### Market Analytics Redis
+Market analytics and metrics.
+
+**Configuration:**
+```yaml
+type: market_analytics_redis
+config:
+  redis_url: "redis://localhost:6379"
+```
+
+**Use Cases:** Trading analytics and market data
+
+---
+
+### Cloud Storage
+
+#### Google Cloud Storage
+Upload to Google Cloud Storage buckets.
+
+**Configuration:**
+```yaml
+type: gcs
+config:
+  bucket_name: "my-stellar-data"
+  object_prefix: "ledgers/"
+  credentials_file: "/path/to/credentials.json"
+```
+
+**Use Cases:** Cloud backup and archival
+
+---
+
+#### Parquet Files
+Parquet file generation with cloud storage support (local/S3/GCS).
+
+**Configuration:**
+```yaml
+type: parquet
+config:
+  storage_type: "s3"
+  s3_bucket: "my-stellar-data"
+  output_path: "parquet/ledgers/"
+```
+
+**Use Cases:** Columnar data export for analytics
+
+---
+
+#### Ledger Parquet
+Ledger-specific Parquet files.
+
+**Configuration:**
+```yaml
+type: ledger_parquet
+config:
+  storage_type: "gcs"
+  gcs_bucket: "my-stellar-ledgers"
+  output_path: "parquet/"
+```
+
+**Use Cases:** Ledger data archival in Parquet format
+
+---
 
 ### Streaming & Messaging
 
-#### Webhook
-HTTP endpoint delivery for real-time notifications:
-- Configurable batch sizes
-- Secret-based authentication
-- Retry logic
-- Custom headers support
-
-**Use Cases:**
-- Real-time notifications
-- Microservice integration
-- Third-party webhooks
-- Event-driven architectures
-
-#### Kafka
-Enterprise message streaming platform:
-- High-throughput data pipelines
-- Multiple consumer groups
-- Guaranteed delivery
-- Horizontal scaling
+#### Google Pub/Sub
+Publisher for Google Pub/Sub with EventPayment support.
+
+**Configuration:**
+```yaml
+type: pubsub
+config:
+  project_id: "my-project"
+  topic_id: "stellar-events"
+  credentials_file: "/path/to/credentials.json"
+```
+
+**Use Cases:** Event streaming to Google Pub/Sub
+
+---
+
+#### Google Pub/Sub V2
+V2 format with chain identifier (StellarMainnet/Testnet).
+
+**Configuration:**
+```yaml
+type: pubsub_v2
+config:
+  project_id: "my-project"
+  topic_id: "stellar-events-v2"
+  chain_identifier: "StellarMainnet"
+```
+
+**Use Cases:** Multi-chain Pub/Sub publishing
+
+---
 
 #### ZeroMQ
-Lightweight message queue for:
-- High-performance IPC
-- Minimal latency
-- Flexible patterns (pub/sub, push/pull)
-- No broker required
+Low-latency message publishing.
 
-### Cloud Storage
+**Configuration:**
+```yaml
+type: zeromq
+config:
+  endpoint: "tcp://localhost:5555"
+```
+
+**Use Cases:** High-performance IPC, minimal latency
+
+---
+
+### Real-time Communications
+
+#### WebSocket
+WebSocket server with client filtering and queuing.
+
+**Configuration:**
+```yaml
+type: websocket
+config:
+  port: 8080
+  path: "/ws"
+  max_queue_size: 1000
+```
+
+**Use Cases:** Real-time browser/app updates with custom filters
+
+---
+
+### File Export
+
+#### Excel
+Excel spreadsheet export.
+
+**Configuration:**
+```yaml
+type: excel
+config:
+  file_path: "/path/to/output.xlsx"
+  sheet_name: "Data"
+```
+
+**Use Cases:** Manual analysis and reporting
+
+---
+
+#### Latest Ledger Excel
+Ledger-specific Excel export.
+
+**Configuration:**
+```yaml
+type: latest_ledger_excel
+config:
+  file_path: "/path/to/ledgers.xlsx"
+```
+
+**Use Cases:** Ledger progress tracking in Excel
 
-#### Amazon S3
-Scalable object storage for:
-- Data archival
-- Batch processing inputs
-- Data lake architectures
-- Backup and compliance
+---
+
+### AI/ML Integration
+
+#### Anthropic Claude
+Batches messages and sends to Claude API for analysis.
+
+**Configuration:**
+```yaml
+type: claude
+config:
+  anthropic_api_key: "sk-ant-..."
+  batch_size: 10
+  flush_interval_seconds: 30
+```
+
+**Use Cases:** AI-powered transaction analysis and insights
+
+---
+
+### Notifications
+
+#### Notification Dispatcher
+Multi-channel notifications (Slack, email, webhook) with rule-based filtering.
+
+**Configuration:**
+```yaml
+type: notification_dispatcher
+config:
+  slack_token: "xoxb-..."
+  webhook_urls:
+    - "https://hooks.slack.com/services/..."
+  rules:
+    - condition: "amount > 1000000"
+      channel: "high-value-payments"
+```
+
+**Use Cases:** Alert system for threshold-based monitoring
+
+---
+
+### Data Transformation
 
-## Specialized Consumers
+#### Silver Ingester
+Bronze to Silver layer transformation for medallion architecture.
 
-### PostgreSQL Schema Variants
+**Configuration:**
+```yaml
+type: silver_ingester
+config:
+  source_db: "bronze.duckdb"
+  target_db: "silver.duckdb"
+```
+
+**Use Cases:** Curated data layer processing
+
+---
 
-Flow offers specialized PostgreSQL consumers with optimized schemas:
+### Debugging & Development
 
-- **Account Data PostgreSQL**: Stellar account information
-- **Contract Invocations PostgreSQL**: Soroban contract calls
-- **Contract Events PostgreSQL**: Smart contract events
-- **SwapService PostgreSQL**: DEX-specific data
-- **Contract Data PostgreSQL**: Contract storage with custom schemas
-- **Extracted Contract Invocations PostgreSQL**: Business data extraction
+#### Stdout
+Prints JSON to stdout for debugging.
+
+**Configuration:**
+```yaml
+type: stdout
+```
+
+**Use Cases:** Development, debugging, testing
+
+---
+
+#### Debug Logger
+Detailed message inspection with field limits.
+
+**Configuration:**
+```yaml
+type: debug_logger
+config:
+  name: "pipeline-debug"
+  log_prefix: "DEBUG"
+  max_fields: 10
+```
+
+**Use Cases:** Deep debugging of message structure
+
+---
+
+#### Log Debug
+Logs message types and metadata.
+
+**Configuration:**
+```yaml
+type: log_debug
+config:
+  log_level: "info"
+```
+
+**Use Cases:** Pipeline flow debugging
+
+---
 
 ## Choosing the Right Consumer
 
diff --git a/docs/flow/processors/index.md b/docs/flow/processors/index.md
index 46251b0..66da7c8 100644
--- a/docs/flow/processors/index.md
+++ b/docs/flow/processors/index.md
@@ -9,100 +9,714 @@ Processors are the core components that transform raw blockchain data into struc
 
 ## Available Processors
 
-### Payment Processing
+### Core Ledger & Transaction
 
-#### [Payments with Memo](./payments-with-memo.md)
-Process payment operations with memo field filtering. Perfect for:
-- Invoice reconciliation systems
-- Payment tracking with reference IDs
-- Multi-tenant payment processing
+#### Ledger Reader
+Reads ledger close metadata and emits individual transactions.
 
-**Configuration Options:**
-- Filter by minimum amount
-- Match specific memo text
-- Filter by asset code
-- Monitor specific addresses
+**Configuration:**
+```yaml
+type: ledger_reader
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Entry point for transaction-level processing pipelines
+
+---
+
+#### Passthrough Processor
+Passes ledger data through with minimal transformation, maintains XDR structure.
+
+**Configuration:**
+```yaml
+type: passthrough
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+  add_metadata: false
+  include_full_xdr: true
+  include_xdr_json: false
+```
+
+**Use Case:** Preserve raw ledger data while adding lightweight metadata
 
-### Transaction Processing
+---
 
-#### Raw Transactions
-Access and process all network transactions without filtering. Ideal for:
-- Complete transaction history
-- Network-wide analytics
-- Custom transaction processing
+#### Ledger to JSON
+Converts LedgerCloseMeta to JSON format.
+
+**Configuration:**
+```yaml
+type: ledger_to_json
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Export ledger data as JSON for downstream systems
+
+---
+
+#### Ledger Changes
+Extracts and processes ledger entry changes (creates, updates, deletes).
+
+**Configuration:**
+```yaml
+type: ledger_changes
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Track state changes in the ledger
+
+---
 
-**Configuration Options:**
-- Filter by transaction types
-- Full transaction envelope access
+#### Latest Ledger
+Tracks and emits latest ledger information with transaction metrics.
+
+**Configuration:**
+```yaml
+type: latest_ledger
+```
+
+**Use Case:** Monitor ledger progression and transaction activity
+
+---
+
+#### Latest Ledger RPC
+Transforms raw getLatestLedger RPC data into structured messages.
+
+**Configuration:**
+```yaml
+type: latest_ledger_rpc
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Process RPC responses for latest ledger queries
+
+---
+
+#### Operation Processor
+Extracts all operation types from transactions.
+
+**Configuration:**
+```yaml
+type: operation
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Track all operations regardless of type
+
+---
 
 ### Account Monitoring
 
-#### Account Balance
-Track balance changes for specified accounts. Essential for:
-- Treasury management
-- Wallet balance monitoring
-- Portfolio tracking
+#### Account Data
+Process Stellar account data changes including creation/updates/deletion.
+
+**Configuration:**
+```yaml
+type: account_data
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Track account state changes, balance updates
+
+---
+
+#### Account Data Filter
+Filters account records by various criteria.
+
+**Configuration:**
+```yaml
+type: account_data_filter
+config:
+  account_ids:
+    - "GABC..."
+    - "GDEF..."
+  min_balance: "100000000"
+  change_types:
+    - "created"
+    - "updated"
+  start_date: "2024-01-01T00:00:00Z"
+  end_date: "2024-12-31T23:59:59Z"
+```
+
+**Use Case:** Filter account changes for specific accounts or thresholds
+
+---
+
+#### Account Effects
+Extracts effects (side-effects of operations) for accounts.
+
+**Configuration:**
+```yaml
+type: account_effects
+config:
+  account: "GABC123..."
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Track all effects on specific accounts (credits, debits, trustlines)
+
+---
+
+#### Account Transactions
+Extracts transactions involving specific accounts (Stellar & Soroban).
+
+**Configuration:**
+```yaml
+type: account_transactions
+config:
+  account: "GABC123..."
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
 
-**Configuration Options:**
-- Monitor multiple accounts
-- Filter by specific assets
-- Real-time balance updates
+**Use Case:** Transaction history for accounts
+
+---
+
+#### Account Year Analytics
+Generates yearly analytics for a specific account.
+
+**Configuration:**
+```yaml
+type: account_year_analytics
+config:
+  account_id: "GABC123..."
+  year: 2024
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Annual account activity reports
+
+---
+
+#### Create Account
+Extracts create account operations.
+
+**Configuration:**
+```yaml
+type: create_account
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Track new account creation events
+
+---
 
 ### Soroban Contract Processing
 
 #### [Contract Events](./contract-events.md)
-Subscribe to Soroban smart contract events for:
-- DeFi protocol monitoring
-- dApp event tracking
-- Custom contract analytics
+Captures and processes events emitted by Soroban smart contracts.
+
+**Configuration:**
+```yaml
+type: contract_events
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** DeFi protocol monitoring, dApp event tracking
+
+---
 
 #### Contract Invocation
-Track all contract invocations with detailed information about:
-- Function calls
-- Parameters
-- Results
+Extracts contract invocations with full execution details (arguments, results, state changes).
+
+**Configuration:**
+```yaml
+type: contract_invocation
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Comprehensive contract execution tracking
+
+---
 
 #### Contract Filter
-Filter events by specific contract IDs for targeted monitoring of:
-- Your deployed contracts
-- Third-party protocol contracts
-- Specific contract interactions
+Filters events by specific contract IDs.
+
+**Configuration:**
+```yaml
+type: contract_filter
+config:
+  contract_ids:
+    - "CCABC123..."
+    - "CCDEF456..."
+```
+
+**Use Case:** Targeted monitoring of specific contracts
+
+---
 
 #### Contract Data
-Process contract data changes on the Soroban network for:
-- State change monitoring
-- Contract storage analytics
-- Data integrity tracking
+Processes contract data changes on the Soroban network.
 
-#### Extracted Contract Invocation
-Extract structured business data from contract invocations using configurable schemas. Perfect for:
-- Custom business logic extraction
-- Structured data pipelines
-- Domain-specific processing
+**Configuration:**
+```yaml
+type: contract_data
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
 
-### Network Metrics
+**Use Case:** State change monitoring, contract storage analytics
 
-#### Latest Ledger Metrics
-Real-time network statistics and metrics including:
-- Latest ledger information
-- Transaction counts
-- Network health indicators
+---
 
-### DeFi Specific
+#### Contract Creation
+Tracks contract creation events.
 
-#### SwapService
-Specialized processor for Soroswap DEX events:
-- Swap transactions
-- Liquidity events
-- Price tracking
+**Configuration:**
+```yaml
+type: contract_creation
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
 
-### Account Data
+**Use Case:** Monitor new contract deployments
 
-#### Account Data
-Process Stellar account data changes including:
-- Account creation/updates
-- Trustline modifications
-- Account flags and thresholds
+---
+
+#### Filtered Contract Invocation
+Filters contract invocations by criteria.
+
+**Configuration:**
+```yaml
+type: filtered_contract_invocation
+config:
+  contract_id: "CCABC123..."
+  function_name: "transfer"
+```
+
+**Use Case:** Filter invocations by contract ID or function name
+
+---
+
+#### Stellar Contract Events
+Processes Stellar Asset Contract (SAC) events.
+
+**Configuration:**
+```yaml
+type: stellar_contract_events
+```
+
+**Use Case:** Track Stellar Asset Contract interactions
+
+---
+
+#### Get Events RPC
+Fetches events via RPC getEvents endpoint.
+
+**Configuration:**
+```yaml
+type: get_events_rpc
+config:
+  rpc_url: "https://rpc.nodeswithobsrvr.co/"
+```
+
+**Use Case:** Query historical contract events via RPC
+
+---
+
+#### Filter Events
+Filters contract events by various criteria.
+
+**Configuration:**
+```yaml
+type: filter_events
+config:
+  event_type: "contract"
+  contract_ids:
+    - "CCABC123..."
+```
+
+**Use Case:** Filter events by type, topic, or contract
+
+---
+
+### Bronze Layer (Hubble-Compatible)
+
+#### Bronze Extractors
+Extracts ALL 19 Bronze table data types from LedgerCloseMeta (Hubble-compatible schema).
+
+**Configuration:**
+```yaml
+type: bronze_extractors
+config:
+  network_passphrase: "Test SDF Network ; September 2015"
+```
+
+**Output:** ledgers_row_v2, transactions_row_v2, operations_row_v2, effects_row_v1, trades_row_v1, accounts_snapshot_v1, trustlines_snapshot_v1, contract_events_stream_v1, and 11 more
+
+**Use Case:** Export all data for medallion architecture
+
+---
+
+#### Bronze to Contract Invocation
+Converts Bronze SQL query results to ContractInvocation format.
+
+**Configuration:**
+```yaml
+type: bronze_to_contract_invocation
+```
+
+**Use Case:** Bridge Bronze data queries with contract invocation extractors
+
+---
+
+### Payment & Operation Processing
+
+#### Filter Payments
+Filters payment operations by amount and asset.
+
+**Configuration:**
+```yaml
+type: filter_payments
+config:
+  min_amount: "100"
+  asset_code: "USDC"
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Track large payments or specific asset transfers
+
+---
+
+#### Participant Extractor
+Identifies all accounts involved in a transaction.
+
+**Configuration:**
+```yaml
+type: participant_extractor
+```
+
+**Use Case:** Extract transaction participants for network analysis
+
+---
+
+#### Event Payment Extractor
+Extracts structured EventPayment data from contract events.
+
+**Configuration:**
+```yaml
+type: event_payment_extractor
+```
+
+**Use Case:** Parse payment events from payment processor contracts
+
+---
+
+#### Transaction XDR Extractor
+Extracts transaction XDR data.
+
+**Configuration:**
+```yaml
+type: transaction_xdr_extractor
+```
+
+**Use Case:** Export transaction XDR for debugging or archival
+
+---
+
+### Asset & Market Data
+
+#### Asset Processor
+Processes asset events from operations.
+
+**Configuration:**
+```yaml
+type: asset
+```
+
+**Use Case:** Track asset usage in payments and trades
+
+---
+
+#### Asset Enrichment
+Enriches asset data with issuer account information.
+
+**Configuration:**
+```yaml
+type: asset_enrichment
+config:
+  connection_string: "postgresql://user:pass@host:5432/database"
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Add issuer metadata (home domain, auth flags) to asset data
+
+---
+
+#### Transform to Asset Stats
+Generates asset statistics.
+
+**Configuration:**
+```yaml
+type: transform_to_asset_stats
+```
+
+**Use Case:** Asset analytics and metrics (holder counts, volume)
+
+---
+
+#### Transform to Token Price
+Calculates token prices.
+
+**Configuration:**
+```yaml
+type: transform_to_token_price
+```
+
+**Use Case:** Price tracking for assets
+
+---
+
+#### Transform to Market Cap
+Calculates market cap and supply data.
+
+**Configuration:**
+```yaml
+type: transform_to_market_cap
+```
+
+**Use Case:** Market cap tracking for assets
+
+---
+
+#### Transform to Market Analytics
+Generates market analytics.
+
+**Configuration:**
+```yaml
+type: transform_to_market_analytics
+```
+
+**Use Case:** Comprehensive market analysis
+
+---
+
+#### Market Metrics
+Generates sliding window market metrics.
+
+**Configuration:**
+```yaml
+type: market_metrics
+```
+
+**Use Case:** Trading pair analytics with time-series data
+
+---
+
+#### Transform to Ticker Asset
+Transforms operations into ticker asset data.
+
+**Configuration:**
+```yaml
+type: transform_to_ticker_asset
+```
+
+**Use Case:** Asset ticker data for exchanges
+
+---
+
+#### Transform to Ticker Orderbook
+Generates orderbook ticker data.
+
+**Configuration:**
+```yaml
+type: transform_to_ticker_orderbook
+```
+
+**Use Case:** Orderbook snapshots for trading pairs
+
+---
+
+### DeFi Protocol Processors
+
+#### Soroswap
+Processes Soroswap DEX events (new pairs, syncs).
+
+**Configuration:**
+```yaml
+type: soroswap
+```
+
+**Use Case:** Track Soroswap liquidity pool creation and state changes
+
+---
+
+#### Soroswap Router
+Processes Soroswap router events.
+
+**Configuration:**
+```yaml
+type: soroswap_router
+```
+
+**Use Case:** Track Soroswap swap routing
+
+---
+
+#### Phoenix AMM
+Processes Phoenix AMM contract events.
+
+**Configuration:**
+```yaml
+type: phoenix_amm
+```
+
+**Use Case:** Track Phoenix AMM swaps and liquidity events
+
+---
+
+#### Kale
+Processes Kale contract events and invocations.
+
+**Configuration:**
+```yaml
+type: kale
+```
+
+**Use Case:** Track Kale protocol metrics
+
+---
+
+### Application Transforms
+
+#### Transform to App Account
+Transforms account data for application use.
+
+**Configuration:**
+```yaml
+type: transform_to_app_account
+```
+
+**Use Case:** Application-specific account data formatting
+
+---
+
+#### Transform to App Metrics
+Generates application network metrics.
+
+**Configuration:**
+```yaml
+type: transform_to_app_metrics
+```
+
+**Use Case:** Network-wide metrics for applications
+
+---
+
+#### Transform to App Payment
+Transforms payments for application use.
+
+**Configuration:**
+```yaml
+type: transform_to_app_payment
+```
+
+**Use Case:** Application-friendly payment formatting
+
+---
+
+#### Transform to App Trade
+Transforms trades for application use.
+
+**Configuration:**
+```yaml
+type: transform_to_app_trade
+```
+
+**Use Case:** Application-friendly trade formatting
+
+---
+
+#### Transform to App Trustline
+Transforms trustlines for application use.
+
+**Configuration:**
+```yaml
+type: transform_to_app_trustline
+```
+
+**Use Case:** Application-friendly trustline formatting
+
+---
+
+### State & Effects
+
+#### Stellar Effects
+Processes Stellar effects.
+
+**Configuration:**
+```yaml
+type: stellar_effects
+```
+
+**Use Case:** Extract Horizon-style effects from ledgers
+
+---
+
+#### Claimable Balance
+Processes claimable balance entries.
+
+**Configuration:**
+```yaml
+type: claimable_balance
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Track claimable balance creation/updates/claims
+
+---
+
+### Wallet Backend
+
+#### Wallet Backend
+Processes state changes for wallet backend.
+
+**Configuration:**
+```yaml
+type: wallet_backend
+```
+
+**Use Case:** Track all state changes relevant to wallet applications
+
+---
+
+### Utilities
+
+#### Blank Processor
+Example/template processor for payments.
+
+**Configuration:**
+```yaml
+type: blank
+config:
+  network_passphrase: "Public Global Stellar Network ; September 2015"
+```
+
+**Use Case:** Template for building custom processors
+
+---
+
+#### Stdout Sink
+Writes payload to stdout (terminal sink).
+
+**Configuration:**
+```yaml
+type: stdout_sink
+```
+
+**Use Case:** Debug output, pipe to other tools
+
+---
 
 ## Choosing the Right Processor