Lake root URI: configuration and S3 lake I/O

[cursor]

Parent

#105

Spec

spec/features/014-cicd-infrastructure/spec.md

What to build

Add application support for configurable lake root URI so the same pipeline code works against local data/ and s3://smartwealthai-data-lake-dev/:

• Configuration module: LAKE_URI env var (file path or s3:// prefix)
• Lake I/O abstraction: read/write Parquet under raw/, curated/, etc. for file and S3 backends
• Add boto3 to Poetry pipeline optional group
• Unit tests with local temp dirs (no live S3 in PR CI); optional moto if needed

This slice completes the application side of M2; Terraform provisions buckets, this issue wires the pipeline to use them.

Acceptance criteria

• LAKE_URI selects storage backend without code forks per zone
• Existing local demo path (data/) still works when LAKE_URI unset or points to local dir
• S3 write/read works against dev bucket when credentials and URI are set (manual or ingest-smoke verification)
• PR CI remains hermetic (no live AWS calls)

Blocked by

#107

Type

AFK

Related

• Phase 2a: S3 data lake, lake root URI, and pipeline Dockerfile #94 — Phase 2a tracer (Dockerfile + ingest-smoke); shares LAKE_URI scope