Databricks framework to validate Data Quality of pySpark DataFrames and Tables

Metadata-driven framework for Databricks Spark Declarative Pipelines. Config-driven, pattern based approach to batch & streaming across the medallion architecture. Deploys via Declarative Automation Bundles. Built for simplicity, extensibility, and alignment with the Databricks product roadmap.

Databricks-native data trust pipeline — intake certification, drift gating, and control benchmarking in a single deployable product.

Open-source community study guide for all six Databricks certifications (Data Engineer Associate / Professional, Data Analyst Associate, ML Associate / Professional, GenAI Engineer Associate). Aligned to the 2025-2026 official exam guides. Obsidian-flavoured Markdown; PRs welcome.

Medallion Architecture for Data Engineering projects

Databricks SQL in Action — End-to-end medallion architecture lab using Unity Catalog, Volumes, Streaming Tables, Materialized Views, AI SQL functions, dashboards, lineage, and workflow orchestration.

Demo of Databricks Lakeflow Jobs Automation with StackQL and Databricks Asset Bundles

End-to-end Azure Databricks Data Engineering Pipeline with Medallion Architecture, Delta Lake, Unity Catalog, and Lakeflow Jobs.

Sample Databricks Asset Bundle: hotel daily performance KPIs with Lakeflow SDP, UC Metric Views, AI/BI dashboard (Brickstar styled), and Genie NL→SQL

Metadata-driven pipeline orchestration pattern on Databricks — dynamic DAG from a Delta control table, Jobs API v2.2, Unity Catalog, Lakeflow. Deployable via Databricks Asset Bundle.

A Databricks control pattern that certifies every record before downstream consumption. 7 contract checks, replay detection, schema drift handling, and quarantine with explicit reasons. 56 passing tests. Databricks Free Edition validated. Enterprise Data Trust, Chapter 1.

Build a Databricks Lakeflow PSP payment system with Bronze, Silver, and Gold pipelines for end-to-end payment analytics