HyMIR

HyMIR is a BI-inspired hybrid intermediate representation for Text-to-SQL. It uses a typed JSON plan for schema-level decisions and a deterministic compiler to reconstruct executable SQL from the plan.

Paper: HyMIR: A BI-Inspired Hybrid Intermediate Representation for Schema-Agnostic Text-to-SQL, IEEE Access, vol. 14, pp. 110097-110108, 2026.

This public bundle contains the core HyMIR encoder/compiler/evaluation code, selected runnable scripts, prompt documentation, and a small synthetic sample file. It intentionally does not include BIRD, Spider, experiment logs, model checkpoints, private adapters, or paper build artifacts.

Contents

hymir/
├── src/                         # Core HyMIR, compiler, evaluation modules
├── scripts/
│   ├── run_experiment.py         # Main BIRD/Spider evaluation runner
│   ├── smoke_compile.py          # No-data compiler smoke test
│   ├── sweep_nbest.sh            # N-best sweep runner (N=1/5/11)
│   ├── n-sweep.sh                # Alias for sweep_nbest.sh
│   ├── convert_dsl.py            # Convert JSONL SQL targets to HyMIR
│   └── validate_encoder_decoder.py
├── tests/                       # Lightweight smoke tests
├── data/sample_dev.json          # Toy sample data only
├── docs/PROMPTS_PUBLIC.md
├── docs/REPORTED_EXPERIMENTS.md
├── docs/SYNTHETIC_SFT_PROTOCOL.md
├── configs/                    # Reported experiment env files
├── .env.example
├── requirements.txt
├── THIRD_PARTY_NOTICES.md
└── LICENSE

Installation

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

For the reported-style setup, serve your model with a local OpenAI-compatible server such as vLLM or Ollama, then edit .env and set OPENAI_BASE_URL and LLM_MODEL. A real API key is usually not needed for local servers. For local fine-tuned Qwen evaluation without an API server, set QWEN_BASE_MODEL and QWEN_LORA_PATH.

Data

Benchmark datasets are not redistributed in this repository.

• Download BIRD from the official BIRD benchmark website.
• Download Spider from the official Spider dataset release if needed.
• Place BIRD dev files under data/bird/dev/:

data/bird/dev/
├── dev.json
├── dev_tables.json
└── dev_databases/
    └── <db_id>/<db_id>.sqlite

The paired SFT corpus is released as a separate Hugging Face dataset:

• https://huggingface.co/datasets/jumperchen/hymir-bird-sft

It is a dataset artifact derived from BIRD train seeds and is distributed under CC BY-SA 4.0 with attribution to the BIRD benchmark (see the dataset card).

Smoke Tests

python -m compileall -q src scripts tests
PYTHONPATH=. python scripts/smoke_compile.py
PYTHONPATH=. python -m unittest discover -s tests

Example Scripts

Validate compiler round-trip coverage on a BIRD-style dev file:

python scripts/validate_encoder_decoder.py \
  --data-path data/bird/dev/dev.json \
  --db-dir data/bird/dev/dev_databases \
  --tables data/bird/dev/dev_tables.json \
  --limit 20

Convert a JSONL file with SQL targets to HyMIR:

python scripts/convert_dsl.py input.jsonl output.jsonl \
  --tables-json data/bird/train/train_tables.json

Run the main experiment runner:

python scripts/run_experiment.py --dataset bird --limit 10

Reproducing the Reported BIRD Results

The method configuration that produces the reported numbers (value-augmented schema, soft DSL validation, trusted fine-tuned DSL, no post-hoc SQL rewriting) ships in .env.example; copying it to .env during installation applies it automatically. The per-arm N-best sweep parameters (mode, n-values, repeats, model aliases) live in configs/. Do not remove the method flags from .env when editing it — they are required to reproduce the reported accuracy.

1. Serve your fine-tuned checkpoints with a local OpenAI-compatible server (vLLM/Ollama) and note the model aliases and base URL.
2. Place BIRD dev under data/bird/dev/ (see Data above).
3. Edit the arm configs in configs/ so the model alias and base URL match your server (PROPOSED_DSL_LLM_MODEL, PROPOSED_DSL_OPENAI_BASE_URL, BASELINE_FT_LLM_MODEL, BASELINE_FT_OPENAI_BASE_URL).
4. Run the arm wrappers (each sources configs/reported_common.env + its arm config and calls sweep_nbest.sh with --n-values "1 5 11" --repeats 10):

scripts/reported/run_proposed_7b.sh      # HyMIR SFT  (mode=proposed)
scripts/reported/run_baseline_ft_7b.sh   # Standard SQL SFT (mode=baseline_ft)
scripts/reported/run_baseline_1p5b.sh    # 1.5B SQL SFT

The manual equivalent and the exact flags are in docs/REPORTED_EXPERIMENTS.md.

Expected numbers (BIRD dev, 7B backbone, avg. of 10 runs)

Bundle mode	Paper arm	N=1	N=5	N=11
baseline_ft	Standard SQL SFT	52.50%	54.58%	56.81%
proposed	HyMIR SFT	55.48%	56.78%	58.44%

Across both arms, the run-to-run standard deviation over 10 decoding seeds is 0.20–0.44 points (the per-cell values are reported in the paper). Full cross-scale (1.5B/3B/7B), ablation, and Spider numbers are in the paper.

The compiler-only oracle round-trip (the IR's representational ceiling, not model accuracy) reaches 92.83% on BIRD dev; validate_encoder_decoder.py measures this compiler round-trip coverage (run without --limit for the full set).

Training Data Protocol

The paired SFT corpus (48,504 execution-verified examples) is available at https://huggingface.co/datasets/jumperchen/hymir-bird-sft. It is seeded exclusively from BIRD train and built in two execution-verified stages: a reference SQL is first established by cross-model agreement between a teacher (Qwen-2.5-Coder-32B) and an independent verifier (OmniSQL-32B), both open models; the verified SQL is then back-translated into the paired HyMIR target by the deterministic oracle encoder, and the pair is kept only when a compiler round-trip reproduces the same result rows. The SQL-SFT and HyMIR-SFT arms are trained on the same accepted programs; only the output representation differs. See docs/SYNTHETIC_SFT_PROTOCOL.md for the full protocol.

No BIRD dev/test examples are used to construct the SFT corpus.

Citation

If you use this code, please cite the HyMIR paper:

@article{chen2026hymir,
  title   = {{HyMIR}: A {BI}-Inspired Hybrid Intermediate Representation for Schema-Agnostic Text-to-{SQL}},
  author  = {Chen, Wei-Heng and Chien, Ying-Ren},
  journal = {IEEE Access},
  year    = {2026},
  volume  = {14},
  pages   = {110097--110108},
  doi     = {10.1109/ACCESS.2026.3712824}
}

Please also cite the benchmark datasets used in your experiments (BIRD, Spider).