Update README and CLAUDE.md for polars→arrow migration

Update all code examples, API docs, and architecture description to
reflect that the data bridge now uses pyarrow instead of polars.
register() accepts any type that pyarrow.table() can convert (pyarrow,
polars, pandas, etc.).

Author: cpsievert

CLAUDE.md

@@ -0,0 +1,43 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## What This Is
+
+Python bindings for the [ggsql](https://github.com/posit-dev/ggsql) Rust crate — a SQL extension for declarative data visualization. The Rust crate handles parsing, validation, and Vega-Lite generation; this repo wraps it via PyO3/maturin and adds a Python-native API layer (`render_altair()`, `VegaLiteWriter.render_chart()`).
+
+## Build & Development
+
+Requires Rust toolchain and Python 3.10+. Uses `uv` for Python dependency management.
+
+```bash
+# Install dev dependencies
+uv sync
+
+# Build the Rust extension in-place (required after any Rust changes)
+uv run maturin develop
+
+# Run all tests
+uv run pytest tests/ -v
+
+# Run a single test
+uv run pytest tests/test_ggsql.py::TestValidate::test_valid_query_with_visualise -v
+
+# Rust checks (CI runs these)
+cargo fmt -- --check
+cargo clippy -- -D warnings
+```
+
+To pick up a new version of the upstream `ggsql` Rust crate, bump its version in `Cargo.toml` and re-run `maturin develop`.
+
+## Architecture
+
+**Rust layer** (`src/lib.rs`): Single-file PyO3 module exposing `DuckDBReader`, `VegaLiteWriter`, `Validated`, `Spec`, `validate()`, and `execute()` to Python. Data crosses the Rust↔Python boundary via Arrow IPC serialization (`arrow::ipc::StreamWriter`/`StreamReader` on the Rust side, `pyarrow.ipc` on the Python side). The `py_to_df` helper accepts any object that `pyarrow.table()` can convert (pyarrow Tables, polars DataFrames, pandas DataFrames, etc.). Custom Python readers are bridged to the Rust `Reader` trait via `PyReaderBridge`; native readers (currently just `DuckDBReader`) use a fast path that skips the bridge (see `try_native_readers!` macro).
+
+**Python layer** (`python/ggsql/__init__.py`): Re-exports Rust bindings and adds `render_altair()` (convenience function that registers a DataFrame, executes, and returns an Altair chart) and a Python `VegaLiteWriter` wrapper that adds `render_chart()`. The `_json_to_altair_chart()` helper dispatches to the correct Altair chart class based on the Vega-Lite spec structure (layer, facet, concat, etc.).
+
+**Key design pattern**: Two-stage API — `reader.execute(query) -> Spec`, then `writer.render(spec) -> str` or `writer.render_chart(spec) -> AltairChart`. The `render_altair()` shortcut collapses both stages.
+
+## `.cargo/config.toml`
+
+Sets `GGSQL_SKIP_GENERATE=1` so tree-sitter uses its pre-generated parser rather than regenerating from `grammar.js`. Don't remove this.

README.md

@@ -44,21 +44,21 @@ pip install target/wheels/ggsql-*.whl
 
 ### Simple Usage with `render_altair`
 
-For quick visualizations, use the `render_altair` convenience function:
+For quick visualizations, use the `render_altair` convenience function. It accepts any narwhals-compatible DataFrame (polars, pandas, pyarrow, etc.):
 
 ```python
 import ggsql
-import polars as pl
+import pyarrow as pa
 
-# Create a DataFrame
-df = pl.DataFrame({
+# Create a table
+table = pa.table({
     "x": [1, 2, 3, 4, 5],
     "y": [10, 20, 15, 30, 25],
     "category": ["A", "B", "A", "B", "A"]
 })
 
 # Render to Altair chart
-chart = ggsql.render_altair(df, "VISUALISE x, y DRAW point")
+chart = ggsql.render_altair(table, "VISUALISE x, y DRAW point")
 
 # Display or save
 chart.display()  # In Jupyter
@@ -71,18 +71,18 @@ For more control, use the two-stage API with explicit reader and writer:
 
 ```python
 import ggsql
-import polars as pl
+import pyarrow as pa
 
 # 1. Create a DuckDB reader
 reader = ggsql.DuckDBReader("duckdb://memory")
 
-# 2. Register your DataFrame as a table
-df = pl.DataFrame({
+# 2. Register your data as a table (accepts pyarrow, polars, pandas, etc.)
+table = pa.table({
     "date": ["2024-01-01", "2024-01-02", "2024-01-03"],
     "revenue": [100, 150, 120],
     "region": ["North", "South", "North"]
 })
-reader.register("sales", df)
+reader.register("sales", table)
 
 # 3. Execute the ggsql query
 spec = reader.execute(
@@ -102,7 +102,7 @@ print(f"Layers: {spec.layer_count()}")
 # 5. Inspect SQL/VISUALISE portions and data
 print(f"SQL: {spec.sql()}")
 print(f"Visual: {spec.visual()}")
-print(spec.layer_data(0))  # Returns polars DataFrame
+print(spec.layer_data(0))  # Returns pyarrow.Table
 
 # 6. Render to Vega-Lite JSON
 writer = ggsql.VegaLiteWriter()
@@ -125,9 +125,9 @@ reader = ggsql.DuckDBReader("duckdb:///path/to/file.db")  # File database
 
 **Methods:**
 
-- `register(name: str, df: polars.DataFrame, replace: bool = False)` - Register a DataFrame as a queryable table
+- `register(name: str, table, replace: bool = False)` - Register data as a queryable table (accepts `pyarrow.Table`, `polars.DataFrame`, `pandas.DataFrame`, etc.)
 - `unregister(name: str)` - Unregister a previously registered table
-- `execute_sql(sql: str) -> polars.DataFrame` - Execute SQL and return results
+- `execute_sql(sql: str) -> pyarrow.Table` - Execute SQL and return results
 
 #### `VegaLiteWriter()`
 
@@ -161,9 +161,9 @@ Result of `reader.execute()`, containing resolved visualization ready for render
 - `sql() -> str` - The executed SQL query
 - `visual() -> str` - The VISUALISE clause
 - `layer_count() -> int` - Number of DRAW layers
-- `data() -> polars.DataFrame | None` - Main query result DataFrame
-- `layer_data(index: int) -> polars.DataFrame | None` - Layer-specific data (if filtered)
-- `stat_data(index: int) -> polars.DataFrame | None` - Statistical transform data
+- `data() -> pyarrow.Table | None` - Main query result data
+- `layer_data(index: int) -> pyarrow.Table | None` - Layer-specific data (if filtered)
+- `stat_data(index: int) -> pyarrow.Table | None` - Statistical transform data
 - `layer_sql(index: int) -> str | None` - Layer filter SQL
 - `stat_sql(index: int) -> str | None` - Stat transform SQL
 - `warnings() -> list[dict]` - Validation warnings from execution
@@ -205,51 +205,54 @@ Convenience function to render a DataFrame with a VISUALISE spec to an Altair ch
 **Returns:** An Altair chart object (Chart, LayerChart, FacetChart, etc.)
 
 ```python
-import polars as pl
+import pyarrow as pa
 import ggsql
 
-df = pl.DataFrame({"x": [1, 2, 3], "y": [10, 20, 30]})
-chart = ggsql.render_altair(df, "VISUALISE x, y DRAW point")
+table = pa.table({"x": [1, 2, 3], "y": [10, 20, 30]})
+chart = ggsql.render_altair(table, "VISUALISE x, y DRAW point")
 ```
 
 ## Examples
 
 ### Mapping Styles
 
 ```python
-df = pl.DataFrame({"x": [1, 2, 3], "y": [10, 20, 30], "category": ["A", "B", "A"]})
+import pyarrow as pa
+
+table = pa.table({"x": [1, 2, 3], "y": [10, 20, 30], "category": ["A", "B", "A"]})
 
 # Explicit mapping
-ggsql.render_altair(df, "VISUALISE x AS x, y AS y DRAW point")
+ggsql.render_altair(table, "VISUALISE x AS x, y AS y DRAW point")
 
 # Implicit mapping (column name = aesthetic name)
-ggsql.render_altair(df, "VISUALISE x, y DRAW point")
+ggsql.render_altair(table, "VISUALISE x, y DRAW point")
 
 # Wildcard mapping (map all matching columns)
-ggsql.render_altair(df, "VISUALISE * DRAW point")
+ggsql.render_altair(table, "VISUALISE * DRAW point")
 
 # With color encoding
-ggsql.render_altair(df, "VISUALISE x, y, category AS color DRAW point")
+ggsql.render_altair(table, "VISUALISE x, y, category AS color DRAW point")
 ```
 
 ### Custom Readers
 
-You can use any Python object with an `execute_sql(sql: str) -> polars.DataFrame` method as a reader. This enables integration with any data source.
+You can use any Python object with an `execute_sql(sql: str)` method as a reader. The method should return a `pyarrow.Table` (or any type that `pyarrow.table()` can convert, such as a `polars.DataFrame`).
 
 ```python
 import ggsql
-import polars as pl
+import pyarrow as pa
+import pyarrow.csv
 
 class CSVReader:
     """Custom reader that loads data from CSV files."""
 
     def __init__(self, data_dir: str):
         self.data_dir = data_dir
 
-    def execute_sql(self, sql: str) -> pl.DataFrame:
+    def execute_sql(self, sql: str) -> pa.Table:
         # Simple implementation: ignore SQL and return fixed data
         # A real implementation would parse SQL to determine which file to load
-        return pl.read_csv(f"{self.data_dir}/data.csv")
+        return pyarrow.csv.read_csv(f"{self.data_dir}/data.csv")
 
 # Use custom reader with ggsql.execute()
 reader = CSVReader("/path/to/data")
@@ -263,7 +266,7 @@ json_output = writer.render(spec)
 
 **Additional methods** for custom readers:
 
-- `register(name: str, df: polars.DataFrame, replace: bool = False) -> None` - Register a DataFrame as a queryable table (required)
+- `register(name: str, table, replace: bool = False) -> None` - Register data as a queryable table (required). Receives a `pyarrow.Table`.
 - `unregister(name: str) -> None` - Unregister a previously registered table (optional)
 
 ```python
@@ -273,12 +276,12 @@ class AdvancedReader:
     def __init__(self):
         self.tables = {}
 
-    def execute_sql(self, sql: str) -> pl.DataFrame:
+    def execute_sql(self, sql: str) -> pa.Table:
         # Your SQL execution logic here
         ...
 
-    def register(self, name: str, df: pl.DataFrame, replace: bool = False) -> None:
-        self.tables[name] = df
+    def register(self, name: str, table: pa.Table, replace: bool = False) -> None:
+        self.tables[name] = table
 
     def unregister(self, name: str) -> None:
         del self.tables[name]
@@ -292,7 +295,7 @@ Native readers like `DuckDBReader` use an optimized fast path, while custom Pyth
 
 ```python
 import ggsql
-import polars as pl
+import pyarrow as pa
 import ibis
 
 class IbisReader:
@@ -305,22 +308,22 @@ class IbisReader:
             self.con = ibis.sqlite.connect()
         # Add other backends as needed
 
-    def execute_sql(self, sql: str) -> pl.DataFrame:
-        return self.con.con.execute(sql).pl()
+    def execute_sql(self, sql: str) -> pa.Table:
+        return self.con.con.execute(sql).arrow()
 
-    def register(self, name: str, df: pl.DataFrame, replace: bool = False) -> None:
-        self.con.create_table(name, df.to_arrow(), overwrite=replace)
+    def register(self, name: str, table: pa.Table, replace: bool = False) -> None:
+        self.con.create_table(name, table, overwrite=replace)
 
     def unregister(self, name: str) -> None:
         self.con.drop_table(name)
 
 # Usage
 reader = IbisReader()
-df = pl.DataFrame({
+table = pa.table({
     "date": ["2024-01-01", "2024-01-02", "2024-01-03"],
     "revenue": [100, 150, 120],
 })
-reader.register("sales", df)
+reader.register("sales", table)
 
 spec = ggsql.execute(
     "SELECT * FROM sales VISUALISE date AS x, revenue AS y DRAW line",
@@ -356,7 +359,7 @@ pytest tests/ -v
 - Python >= 3.10
 - altair >= 5.0
 - narwhals >= 2.15
-- polars >= 1.0
+- pyarrow >= 14.0
 
 ## License