Skip to content

README.md

Latest commit

 

History

History
346 lines (278 loc) · 12.6 KB

README.md

File metadata and controls

346 lines (278 loc) · 12.6 KB

EventLogger Module

Go Reference

The EventLogger Module provides structured logging capabilities for Observer pattern events in Modular applications. It acts as an Observer that can be registered with any Subject to log events to various output targets including console, files, and syslog.

Features

  • Multiple Output Targets: Support for console, file, and syslog outputs
  • Configurable Log Levels: DEBUG, INFO, WARN, ERROR with per-target configuration
  • Multiple Output Formats: Text, JSON, and structured formats
  • Event Type Filtering: Log only specific event types
  • Async Processing: Non-blocking event processing with buffering
  • Log Rotation: Automatic file rotation for file outputs
  • Error Handling: Graceful handling of output target failures
  • Observer Pattern Integration: Seamless integration with ObservableApplication

Installation

import (
    "github.com/GoCodeAlone/modular"
    "github.com/GoCodeAlone/modular/modules/eventlogger"
)

// Register the eventlogger module with your Modular application
app.RegisterModule(eventlogger.NewModule())

Configuration

The eventlogger module can be configured using the following options:

eventlogger:
  enabled: true                    # Enable/disable event logging
  logLevel: INFO                   # Minimum log level (DEBUG, INFO, WARN, ERROR)
  format: structured               # Default output format (text, json, structured)
  bufferSize: 100                  # Event buffer size for async processing
  flushInterval: 5s                # How often to flush buffered events
  includeMetadata: true            # Include event metadata in logs
  includeStackTrace: false         # Include stack traces for error events
  startupSync: false               # Emit startup operational events synchronously (no async delay)
  shutdownEmitStopped: true        # Emit logger.stopped operational event on Stop()
  shutdownDrainTimeout: 2s         # Max time to drain buffered events on Stop (0 = wait forever)
  eventTypeFilters:                # Optional: Whitelist - Only log specific event types
    - module.registered
    - service.registered
    - application.started
  eventTypeBlacklist:              # Optional: Blacklist - Exclude specific event types (overrides whitelist)
    - com.modular.eventlogger.event.received
    - com.modular.eventlogger.output.success
  excludeOwnEvents: false          # Automatically exclude EventLogger's own operational events
  outputTargets:
    - type: console                # Console output
      level: INFO
      format: structured
      console:
        useColor: true
        timestamps: true
    - type: file                   # File output with rotation
      level: DEBUG
      format: json
      file:
        path: /var/log/modular-events.log
        maxSize: 100               # MB
        maxBackups: 5
        maxAge: 30                 # days
        compress: true
    - type: syslog                 # Syslog output
      level: WARN
      format: text
      syslog:
        network: unix
        address: ""
        tag: modular
        facility: user

Usage

Basic Usage with ObservableApplication

import (
    "github.com/GoCodeAlone/modular"
    "github.com/GoCodeAlone/modular/modules/eventlogger"
)

func main() {
    // Create application with observer support
    app := modular.NewObservableApplication(configProvider, logger)

    // Register event logger module
    app.RegisterModule(eventlogger.NewModule())

    // Initialize application - event logger will auto-register as observer
    if err := app.Init(); err != nil {
        log.Fatal(err)
    }

    // Now all application events will be logged
    app.RegisterModule(&MyModule{})  // Logged as module.registered event
    app.Start()                      // Logged as application.started event
}

Manual Observer Registration

// Get the event logger service
var eventLogger *eventlogger.EventLoggerModule
err := app.GetService("eventlogger.observer", &eventLogger)
if err != nil {
    log.Fatal(err)
}

// Register with any subject for specific event types
err = subject.RegisterObserver(eventLogger, "user.created", "order.placed")
if err != nil {
    log.Fatal(err)
}

Event Type Filtering

Whitelist Filtering - Only log specific event types:

config := &eventlogger.EventLoggerConfig{
    EventTypeFilters: []string{
        "module.registered",
        "service.registered", 
        "application.started",
        "application.failed",
    },
}

Blacklist Filtering - Exclude specific event types:

config := &eventlogger.EventLoggerConfig{
    // Log everything except these specific events
    EventTypeBlacklist: []string{
        "com.modular.eventlogger.event.received",
        "com.modular.eventlogger.event.processed",
        "com.modular.eventlogger.output.success",
    },
}

Automatic Self-Event Exclusion - Prevent event amplification by excluding EventLogger's own operational events:

config := &eventlogger.EventLoggerConfig{
    // Automatically exclude EventLogger operational events
    ExcludeOwnEvents: true,
}

Combined Filtering - Use whitelist and blacklist together:

config := &eventlogger.EventLoggerConfig{
    // First apply whitelist
    EventTypeFilters: []string{
        "user.*",           // Allow all user events
        "order.*",          // Allow all order events
        "application.*",    // Allow all application events
    },
    // Then apply blacklist (takes precedence)
    EventTypeBlacklist: []string{
        "user.debug",       // Exclude user.debug even though user.* is whitelisted
    },
}

Preventing Event Amplification

The EventLogger emits operational events about its own processing (event.received, event.processed, output.success, etc.). In high-volume scenarios, these operational events can cause event amplification - where logging 1,000 business events generates 3,000+ operational events, which themselves get logged, creating a feedback loop.

Option 1: Use excludeOwnEvents (Recommended for most cases)

eventlogger:
  enabled: true
  excludeOwnEvents: true  # Simple one-line solution

Option 2: Use Blacklist (For fine-grained control)

eventlogger:
  enabled: true
  eventTypeBlacklist:
    - "com.modular.eventlogger.event.received"
    - "com.modular.eventlogger.event.processed"
    - "com.modular.eventlogger.output.success"
    - "com.modular.eventlogger.buffer.full"
    - "com.modular.eventlogger.event.dropped"

Output Formats

Text Format

Human-readable single-line format:

2024-01-15 10:30:15 INFO [module.registered] application Module 'auth' registered (type=AuthModule)

JSON Format

Machine-readable JSON format:

{"timestamp":"2024-01-15T10:30:15Z","level":"INFO","type":"module.registered","source":"application","data":{"moduleName":"auth","moduleType":"AuthModule"},"metadata":{}}

Structured Format

Detailed multi-line structured format:

[2024-01-15 10:30:15] INFO module.registered
  Source: application
  Data: map[moduleName:auth moduleType:AuthModule]
  Metadata: map[]

Output Targets

Console Output

Outputs to stdout with optional color coding and timestamps:

outputTargets:
  - type: console
    level: INFO
    format: structured
    console:
      useColor: true      # ANSI color codes for log levels
      timestamps: true    # Include timestamps in output

File Output

Outputs to files with automatic rotation:

outputTargets:
  - type: file
    level: DEBUG
    format: json
    file:
      path: /var/log/events.log
      maxSize: 100        # MB before rotation
      maxBackups: 5       # Number of backup files to keep
      maxAge: 30          # Days to keep files
      compress: true      # Compress rotated files

Syslog Output

Outputs to system syslog:

outputTargets:
  - type: syslog
    level: WARN
    format: text
    syslog:
      network: unix       # unix, tcp, udp
      address: ""         # For tcp/udp: "localhost:514"
      tag: modular        # Syslog tag
      facility: user      # Syslog facility

Event Level Mapping

The module automatically maps event types to appropriate log levels:

  • ERROR: application.failed, module.failed
  • WARN: Custom warning events
  • INFO: module.registered, service.registered, application.started, etc.
  • DEBUG: config.loaded, config.validated

Filter Evaluation Order

When multiple filtering options are configured, they are evaluated in the following order:

  1. Whitelist Filter (eventTypeFilters) - If configured, only events matching the whitelist are considered
  2. ExcludeOwnEvents - If enabled, EventLogger's own operational events are filtered out
  3. Blacklist Filter (eventTypeBlacklist) - If configured, events matching the blacklist are excluded (takes precedence over whitelist)
  4. Log Level Filter - Events must meet the minimum log level requirement

Example: If an event type is in both the whitelist AND the blacklist, it will be excluded (blacklist wins).

Performance Considerations

  • Async Processing: Events are processed asynchronously to avoid blocking the application
  • Buffering: Events are buffered in memory before writing to reduce I/O overhead
  • Error Isolation: Failures in one output target don't affect others
  • Graceful Degradation: Buffer overflow results in dropped events with warnings
  • Filter Performance: Event type filters use simple string matching for optimal performance

Startup & Shutdown Behavior

The module supports fine‑grained control over lifecycle behavior:

Setting Purpose Typical Usage
startupSync When true, emits operational startup events (config.loaded, output.registered, logger.started) synchronously inside Start() so tests or dependent logic can immediately observe them without arbitrary sleeps. Enable in deterministic test suites to remove time.Sleep calls. Leave false in production to minimize startup blocking.
shutdownEmitStopped When true (default), emits a eventlogger.logger.stopped operational event after draining. Set false to suppress if you prefer a silent shutdown or want to avoid any late emissions during teardown. Disable in environments where observers are already torn down or to reduce noise in integration tests.
shutdownDrainTimeout Bounded duration to wait for the event queue to drain during Stop(). If the timeout elapses, a warning is logged and shutdown proceeds. Zero (or negative) means wait indefinitely. Tune to balance fast shutdown vs. ensuring critical events are flushed (e.g. increase for audit trails, reduce for fast ephemeral jobs).

Early Lifecycle Event Suppression

Benign framework lifecycle events (e.g. config.loaded, config.validated, module.registered, service.registered) that arrive before the logger has fully started are silently dropped instead of producing event logger not started errors. This prevents noisy, misleading logs during application bootstrapping while keeping genuine misordering issues visible for other event types.

Recommended Testing Pattern

Enable startupSync: true in test configuration to make assertions against startup events immediately after app.Start() without introducing sleeps. Pair with a small shutdownDrainTimeout (e.g. 500ms) to keep CI fast while still flushing most buffered events.

Error Handling

The module handles various error conditions gracefully:

  • Output Target Failures: Logged but don't stop other targets
  • Buffer Overflow: Oldest events are dropped with warnings
  • Configuration Errors: Reported during module initialization
  • Observer Errors: Logged but don't interrupt event flow

Integration with Existing EventBus

The EventLogger module complements the existing EventBus module:

  • EventBus: Provides pub/sub messaging between modules
  • EventLogger: Provides structured logging of Observer pattern events
  • Use Together: EventBus for inter-module communication, EventLogger for audit trails

Testing

The module includes comprehensive tests:

cd modules/eventlogger
go test ./... -v

Implementation Notes

  • Uses Go's log/syslog package for syslog support
  • File rotation could be enhanced with external libraries like lumberjack
  • Async processing uses buffered channels and worker goroutines
  • Thread-safe implementation supports concurrent event logging
  • Implements the Observer interface for seamless integration