Add Redash HTTP API as DB_MODE backend#87
Open
chrisophus wants to merge 1 commit into
Open
Conversation
Introduce RedashRunner for ad-hoc SQL via the Redash REST API, wire it through settings/config/doctor, and improve web startup UX for slow remote schema introspection during auto-load.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| await load_project(project_path, _state) | ||
| logger.info(f"Auto-loaded project: {project_path}") | ||
| except ProjectError as e: | ||
| err_msg = str(e) |
| err_msg = str(e) | ||
| logger.error(f"Failed to auto-load project {project_path}: {e}") | ||
| except Exception as e: | ||
| err_msg = str(e) |
There was a problem hiding this comment.
Pull request overview
Adds a new DB_MODE=redash backend that executes ad-hoc SQL through Redash’s REST API, wires it through settings/config/doctor/export scaffolding, and improves web UI boot behavior for slow remote schema introspection.
Changes:
- Introduces
RedashRunner(HTTPPOST /api/query_results, job polling, result fetch) and addsredashto settings/config/doctor wiring. - Extends dialect hinting + SQL validation for additional warehouse dialects (e.g. mysql/bigquery/snowflake) used behind Redash.
- Improves startup UX: background project auto-load,
/api/projectloading state, and frontend boot hints; plus better handling of stale Plotly chunks.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_settings.py | Adds Settings coverage for DB_MODE=redash env parsing + validation. |
| tests/test_redash_runner.py | Adds mocked HTTP tests for Redash query execution, polling, and timeout/error paths. |
| tests/test_export.py | Verifies Python export renders a Redash-specific stub scaffold. |
| tests/test_config_extra.py | Adds create_sql_runner("redash", …) branch tests and missing-config error cases. |
| src/datasight/web/app.py | Background auto-load on startup + /api/project returns loading state; disables caching for SPA shell. |
| src/datasight/templates/html/export_session_python.mustache | Adds an explicit Redash export stub explaining why secrets aren’t embedded. |
| src/datasight/templates/env.template | Documents Redash env vars and adds redash to DB_MODE help text. |
| src/datasight/sql_validation.py | Expands sqlglot dialect mapping (mysql/spark/bigquery/snowflake). |
| src/datasight/settings.py | Adds Redash settings, env parsing, validation, and DBMode literal update. |
| src/datasight/runner.py | Updates module docstring to include Redash as a supported backend. |
| src/datasight/redash_runner.py | New Redash REST API runner implementation (httpx + polling). |
| src/datasight/prompts.py | Adds dialect hint blocks (mysql/bigquery/snowflake) and a safer fallback hint path. |
| src/datasight/export.py | Treats redash as a documented non-file backend in Python export rendering. |
| src/datasight/explore.py | Updates file-import fallback docs to include redash among non-file backends. |
| src/datasight/config.py | Adds redash runner factory branch and settings wiring for timeouts/poll interval. |
| src/datasight/cli_commands/doctor.py | Updates connectivity probe to async-close HTTP clients safely (Redash) under asyncio.run. |
| src/datasight/agent.py | Adds MySQL-specific error hinting. |
| frontend/src/lib/components/PlotlyChart.svelte | Adds one-time reload on stale dynamic import failures for Plotly chunks. |
| frontend/src/lib/api/projects.ts | Adds loading to ProjectStatus and polling helper for background auto-load completion. |
| frontend/src/App.svelte | Uses project auto-load polling and shows boot hints/loading UI states. |
| docs/reference/configuration.md | Documents DB_MODE=redash and related env vars + behavior. |
| AGENTS.md | Updates project overview to include Redash backend. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+39
to
+49
| def _query_result_to_dataframe(query_result: dict[str, Any]) -> pd.DataFrame: | ||
| data = query_result.get("data") or {} | ||
| cols_meta = data.get("columns") or [] | ||
| col_names = [ | ||
| str(c.get("name") or c.get("friendly_name") or f"col_{i}") | ||
| for i, c in enumerate(cols_meta) | ||
| ] | ||
| rows = data.get("rows") or [] | ||
| if not rows: | ||
| return pd.DataFrame(columns=col_names) | ||
| return pd.DataFrame(rows) |
Comment on lines
+85
to
+94
| if client is None: | ||
| self._client = httpx.AsyncClient( | ||
| base_url=self._base_url, | ||
| headers=self._headers, | ||
| timeout=http_timeout, | ||
| verify=verify_ssl, | ||
| ) | ||
| else: | ||
| self._client = client | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RedashRunnerto execute ad-hoc SQL via the Redash REST API (POST /api/query_results, job polling, result fetch)DB_MODE=redashwith config:REDASH_BASE_URL,REDASH_API_KEY,REDASH_DATA_SOURCE_ID,REDASH_SQL_DIALECT, timeoutsdoctor, export stubs, and MySQL dialect hints in prompts/api/project, frontend boot hintTest plan
pytest tests/test_redash_runner.py tests/test_config_extra.py tests/test_settings.py tests/test_export.py -m "not integration"datasight doctorwithDB_MODE=redashand valid env varsdatasight run --project-dir <project>against a Redash data source withschema.yamltable filtering