refactor: improve transaction processing flow with observability, structured logging, and test coverage

CLAUDE.md

@@ -580,6 +580,8 @@ Full rules in `docs/PROJECT_RULES.md` (1130 lines). Key points:
 14. **Query builder**: Use `squirrel` for all SQL query construction (SELECT, INSERT, UPDATE). Do not use raw SQL string concatenation with `strconv.Itoa` for parameter placeholders.
 15. **Entity name constants**: Use `constant.Entity*` instead of `reflect.TypeOf(mmodel.Foo{}).Name()`. See `pkg/constant/entity.go`.
 16. **Timestamps on create**: Capture `time.Now()` once and reuse for both `CreatedAt` and `UpdatedAt` to guarantee identical values.
+17. **Declaration order**: Within a file, declare in this order: exported interface → exported types → constructor → exported methods → unexported helpers. The interface is the contract readers look for first.
+18. **Repository tests**: Repository/adapter code (thin wrappers around Redis, Postgres, etc.) should be covered by integration tests with testcontainers, not unit tests. Unit-testing mock interactions only verifies you called the mock correctly — it does not catch real issues like key format mismatches, TTL semantics, or Lua script behavior. Unit tests in adapter packages should be reserved for pure functions and business logic branches that don't require external dependencies.
 
 ## Observability Conventions
 
@@ -597,6 +599,33 @@ logger.Log(ctx, libLog.LevelInfo, "Organization created", libLog.String("id", or
 logger.Log(ctx, libLog.LevelError, fmt.Sprintf("Failed to create organization: %v", err))
 ```
 
+### Log Level Guidelines
+
+**Never log sensitive data (balances, financial values, PII) at any log level.**
+
+Use the correct log level based on **who caused the problem**, not just what went wrong:
+
+| Level | When to use | Example |
+|-------|------------|---------|
+| **Debug** | Operational details useful only during development or troubleshooting. | Cache key written, Lua script completed, batch stats |
+| **Info** | Significant state changes or milestones in a business flow. Should be sparse enough to read in production without filtering. | Transaction created, balance sync flushed, idempotency key claimed |
+| **Warn** | **Business validation failures** (the caller sent invalid data, not a system fault). The system is healthy; the request was rejected. Also used for degraded-but-recoverable situations (e.g., cache connection error with DB fallback). Note: a normal cache miss (TTL expiry) is not a warning — it is the expected flow and needs no log at all. | Insufficient funds, asset mismatch, account sending disabled, accounting rule violation, Redis connection error with DB fallback |
+| **Error** | **Infrastructure or system failures** that indicate something is broken and may need operator attention. The system could not fulfill a valid request. | Redis connection refused, DB query failed, Lua script execution error, message broker unavailable |
+
+Key principle: if the fix requires the **caller** to change their request → **Warn**. If the fix requires an **operator** to investigate the system → **Error**.
+
+```go
+// Business validation failure — Warn (caller's problem)
+logger.Log(ctx, libLog.LevelWarn, "Balance rule validation failed", libLog.Err(err))
+
+// Infrastructure failure — Error (system's problem)
+logger.Log(ctx, libLog.LevelError, "Failed to execute atomic balance operation", libLog.Err(err))
+
+// Operational detail — Debug (development only)
+logger.Log(ctx, libLog.LevelDebug, "Lua script executed successfully",
+    libLog.String("backup_queue", prefixedKeys[0]))
+```
+
 ### Span Lifecycle
 - Always use `defer span.End()` immediately after `tracer.Start`.
 - Child spans for I/O operations (DB exec, external calls) should use `defer` and must not

Makefile

@@ -573,11 +573,29 @@ generate-docs:
 dev-setup:
 	$(call print_title,"Setting up development environment for all components")
 	@echo "Installing development tools..."
-	@command -v gitleaks >/dev/null 2>&1 || (echo "Installing gitleaks..." && go install github.com/zricethezav/gitleaks/v8@latest) || echo "⚠️  Failed to install gitleaks"
-	@command -v gofumpt >/dev/null 2>&1 || (echo "Installing gofumpt..." && go install mvdan.cc/gofumpt@latest) || echo "⚠️  Failed to install gofumpt"
-	@command -v goimports >/dev/null 2>&1 || (echo "Installing goimports..." && go install golang.org/x/tools/cmd/goimports@latest) || echo "⚠️  Failed to install goimports"
-	@command -v gosec >/dev/null 2>&1 || (echo "Installing gosec..." && go install github.com/securego/gosec/v2/cmd/gosec@latest) || echo "⚠️  Failed to install gosec"
-	@command -v golangci-lint >/dev/null 2>&1 || (echo "Installing golangci-lint..." && go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION)) || echo "⚠️  Failed to install golangci-lint"
+	@GOBIN="$$(go env GOPATH)/bin"; \
+	check_or_install() { \
+		local name="$$1" pkg="$$2"; \
+		if command -v "$$name" >/dev/null 2>&1 || [ -x "$$GOBIN/$$name" ]; then \
+			echo "  ✓ $$name already installed"; \
+		else \
+			echo "  ⏳ Installing $$name..."; \
+			go install "$$pkg" || echo "  ⚠️  Failed to install $$name"; \
+		fi; \
+	}; \
+	check_or_install gitleaks github.com/zricethezav/gitleaks/v8@latest; \
+	check_or_install gofumpt mvdan.cc/gofumpt@latest; \
+	check_or_install goimports golang.org/x/tools/cmd/goimports@latest; \
+	check_or_install gosec github.com/securego/gosec/v2/cmd/gosec@latest; \
+	check_or_install golangci-lint github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION)
+	@GOBIN="$$(go env GOPATH)/bin"; \
+	if ! echo "$$PATH" | tr ':' '\n' | grep -qx "$$GOBIN"; then \
+		echo ""; \
+		echo "  ⚠️  $$GOBIN is not in your PATH."; \
+		echo "  Add to your shell profile: export PATH=\"\$$PATH:$$GOBIN\""; \
+		echo "  Without this, git hooks (gofumpt, goimports) will be silently skipped."; \
+		echo ""; \
+	fi
 	@echo "Setting up git hooks..."
 	@$(MAKE) setup-git-hooks
 	@for dir in $(COMPONENTS); do \

components/crm/Dockerfile

@@ -1,7 +1,7 @@
 # syntax=docker/dockerfile:1.4
 # ci: force rebuild crm image - 2026-04-01
 
-FROM --platform=$BUILDPLATFORM golang:1.25.8-alpine AS builder
+FROM --platform=$BUILDPLATFORM golang:1.26.2-alpine AS builder
 
 WORKDIR /crm-app
 

components/ledger/.env.example

@@ -35,9 +35,6 @@ MULTI_TENANT_ENABLED=false
 # MULTI_TENANT_REDIS_PASSWORD=
 # MULTI_TENANT_REDIS_TLS=false
 
-# ACCOUNTING CONFIG
-ACCOUNT_TYPE_VALIDATION=
-
 # DB POSTGRESQL PRIMARY - Onboarding
 DB_ONBOARDING_HOST=midaz-postgres-primary
 DB_ONBOARDING_USER=midaz

components/ledger/Dockerfile

@@ -1,4 +1,4 @@
-FROM --platform=$BUILDPLATFORM golang:1.25.8-alpine AS builder
+FROM --platform=$BUILDPLATFORM golang:1.26.2-alpine AS builder
 # Force rebuild for develop branch - 2026-04-03 (5)
 
 WORKDIR /ledger-app