From d6dd79d0a750c6e5a61eaafb3b2413cabafbac77 Mon Sep 17 00:00:00 2001 From: tpp Date: Sun, 15 Mar 2026 22:49:33 -0700 Subject: [PATCH] docs: add how-to guides for TiDB session variable tuning Add 7 scenario-based how-to guides covering all non-cost-factor session variables, grouped by tuning concern: query execution, cost model, statistics collection, stats loading, memory/resources, TiFlash/MPP, and plan baselines. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../how-to/cost-model-and-index-selection.md | 104 ++++++++++ .../how-to/memory-and-resource-control.md | 137 +++++++++++++ .../how-to/plan-stability-and-baselines.md | 186 ++++++++++++++++++ .../how-to/query-execution-tuning.md | 145 ++++++++++++++ .../statistics-collection-and-auto-analyze.md | 184 +++++++++++++++++ .../statistics-loading-and-freshness.md | 110 +++++++++++ .../how-to/tiflash-and-mpp-execution.md | 156 +++++++++++++++ 7 files changed, 1022 insertions(+) create mode 100644 skills/tidb-query-tuning/how-to/cost-model-and-index-selection.md create mode 100644 skills/tidb-query-tuning/how-to/memory-and-resource-control.md create mode 100644 skills/tidb-query-tuning/how-to/plan-stability-and-baselines.md create mode 100644 skills/tidb-query-tuning/how-to/query-execution-tuning.md create mode 100644 skills/tidb-query-tuning/how-to/statistics-collection-and-auto-analyze.md create mode 100644 skills/tidb-query-tuning/how-to/statistics-loading-and-freshness.md create mode 100644 skills/tidb-query-tuning/how-to/tiflash-and-mpp-execution.md diff --git a/skills/tidb-query-tuning/how-to/cost-model-and-index-selection.md b/skills/tidb-query-tuning/how-to/cost-model-and-index-selection.md new file mode 100644 index 0000000..44accae --- /dev/null +++ b/skills/tidb-query-tuning/how-to/cost-model-and-index-selection.md @@ -0,0 +1,104 @@ +# How To: Tune the Cost Model and Index Selection + +These variables control the foundational cost model that the optimizer uses to compare plans, and how it evaluates index access paths. Adjusting them changes how the optimizer weighs sequential vs random I/O, which cost model version is active, and how it handles edge cases in index selection. + +## Variables at a glance + +| Variable | Default | What it controls | +|----------|---------|-----------------| +| `tidb_cost_model_version` | 2 | Which cost model the optimizer uses | +| `tidb_opt_seek_factor` | 20 | Cost multiplier for random I/O seeks | +| `tidb_opt_ordering_index_selectivity_threshold` | 0 | Threshold for preferring an ordering index | +| `tidb_opt_range_max_size` | 67108864 (64 MB) | Max memory for range construction | + +## When to use each variable + +### Plans are suboptimal after upgrading from an old TiDB version + +**Symptom:** After upgrading, some queries get worse plans. The cluster may still be using cost model v1. + +**What to do:** + +```sql +-- Check current version +SELECT @@tidb_cost_model_version; + +-- Switch to v2 (default since TiDB 6.2) +SET GLOBAL tidb_cost_model_version = 2; +``` + +**Benefit:** Cost model v2 is more accurate for modern TiDB. It better estimates the cost of TiKV vs TiFlash access, handles wide tables more accurately, and produces better plans for most workloads. + +**Caution:** Switching cost model versions can change many query plans at once. Test on a non-production environment first, or roll out gradually with session-level testing on critical queries. + +### The optimizer keeps choosing index lookups over full scans on large analytical queries + +**Symptom:** For queries that scan most of a table, the optimizer still picks an index lookup path. The random I/O from index lookups is slower than a sequential full scan would be. + +**What to do:** + +```sql +-- Increase seek factor to make random I/O more expensive +SET tidb_opt_seek_factor = 40; -- default is 20 +``` + +**Benefit:** A higher seek factor tells the optimizer that random I/O is relatively more expensive compared to sequential I/O. This pushes it toward sequential scans and away from index lookups when a large portion of the table is being read. + +**When to lower it:** On fast SSD/NVMe storage where random I/O is nearly as fast as sequential, you might lower the seek factor (e.g., 5-10) to encourage more index-based access paths. + +### ORDER BY ... LIMIT queries pick a bad ordering index + +**Symptom:** A query like `SELECT ... FROM t WHERE ORDER BY col LIMIT 10` uses an index on `col` to avoid sorting, but the filter is poorly selective on that index, causing it to scan many rows before finding matches. + +**What to do:** + +```sql +-- Set a selectivity threshold; the optimizer will only use the ordering index +-- if it estimates the filter selectivity is better than this threshold +SET tidb_opt_ordering_index_selectivity_threshold = 0.1; -- 10% selectivity +``` + +**Benefit:** Prevents the optimizer from choosing an ordering index when the filter would require scanning too many rows through that index. Instead, it will use a more selective index and add an explicit sort, which is faster overall. + +**Typical values:** Start with 0 (disabled, the default) and increase gradually. Values between 0.01 and 0.2 are common. The right value depends on your data distribution and query patterns. + +### Queries with huge IN lists fall back to full index scans + +**Symptom:** A query like `SELECT * FROM t WHERE id IN (1, 2, 3, ..., 100000)` unexpectedly uses an IndexFullScan instead of point lookups. The optimizer ran out of memory constructing the range. + +**What to do:** + +```sql +-- Increase the memory budget for range construction +SET tidb_opt_range_max_size = 134217728; -- 128 MB (default is 64 MB) +``` + +**Benefit:** Allows the optimizer to construct larger range sets, so queries with many IN-list values or complex range conditions can still use efficient point lookups instead of falling back to a full scan. + +**Caution:** Increasing this too much can cause high memory usage during query optimization. Only raise it if you have specific queries that need it, and monitor TiDB memory usage. + +### Queries with large IN lists on non-critical paths + +If raising `tidb_opt_range_max_size` globally is too risky, use per-query scope: + +```sql +SELECT /*+ SET_VAR(tidb_opt_range_max_size=134217728) */ * +FROM orders WHERE id IN (/* large list */); +``` + +## Diagnostic workflow + +1. **Check cost model version first** — this is the single highest-impact variable. If you're on v1, switch to v2 before tuning anything else. + +2. **Look at EXPLAIN output for unexpected access paths:** + - IndexLookUp on analytical queries → raise `tidb_opt_seek_factor` + - Ordering index with poor filter → set `tidb_opt_ordering_index_selectivity_threshold` + - IndexFullScan with large IN list → check `tidb_opt_range_max_size` + +3. **Validate with EXPLAIN ANALYZE** — compare actual rows scanned, execution time, and memory usage before and after the change. + +## General guidance + +- `tidb_cost_model_version` should be set globally and consistently across the cluster. Mixed versions make diagnosis harder. +- `tidb_opt_seek_factor` reflects your storage characteristics. Set it once based on your hardware and leave it. +- `tidb_opt_ordering_index_selectivity_threshold` and `tidb_opt_range_max_size` are best applied per-query or per-session for specific problem queries rather than globally. diff --git a/skills/tidb-query-tuning/how-to/memory-and-resource-control.md b/skills/tidb-query-tuning/how-to/memory-and-resource-control.md new file mode 100644 index 0000000..0e24577 --- /dev/null +++ b/skills/tidb-query-tuning/how-to/memory-and-resource-control.md @@ -0,0 +1,137 @@ +# How To: Control Memory and Resource Usage + +These variables control how much memory and parallelism TiDB uses when executing queries. Use them to prevent OOM kills, manage resource contention between queries, and tune throughput for batch workloads. + +## Variables at a glance + +| Variable | Default | What it controls | +|----------|---------|-----------------| +| `tidb_mem_quota_query` | 1073741824 (1 GB) | Memory limit per query | +| `tidb_max_chunk_size` | 1024 | Max rows per execution chunk | +| `tidb_distsql_scan_concurrency` | 15 | Parallel TiKV coprocessor scan requests | + +## When to use each variable + +### Queries are being killed with "Out Of Memory" errors + +**Symptom:** Queries fail with `Out Of Memory Quota!` errors. The TiDB log shows memory quota exceeded. + +**What to do:** + +```sql +-- Option 1: Increase the per-query memory limit +SET tidb_mem_quota_query = 2147483648; -- 2 GB + +-- Option 2: Increase globally for all connections +SET GLOBAL tidb_mem_quota_query = 2147483648; +``` + +**Benefit:** Allows memory-intensive queries (large joins, aggregations, sorts) to complete without hitting the memory limit. + +**Caution:** Raising this too high risks TiDB node OOM if multiple large queries run concurrently. Calculate the safe limit as: + +``` +safe_limit = (TiDB_available_memory * 0.7) / max_concurrent_heavy_queries +``` + +**Better alternatives to consider first:** +- Optimize the query to use less memory (better indexes, push-down filters) +- Use TiFlash for analytical queries that process large datasets +- Add `ORDER BY ... LIMIT` to reduce result set size + +### Queries need more memory temporarily for a batch job + +**Symptom:** A scheduled batch job needs more memory than the default allows, but you don't want to raise the limit cluster-wide. + +**What to do:** + +```sql +-- Set for this session only +SET tidb_mem_quota_query = 4294967296; -- 4 GB + +-- Run the batch job +INSERT INTO summary_table SELECT ... FROM large_table GROUP BY ...; + +-- Reset (or just close the connection) +SET tidb_mem_quota_query = 1073741824; +``` + +**Benefit:** Gives the batch job the memory it needs without affecting other connections. + +### Scan-heavy queries are too slow + +**Symptom:** Analytical queries or batch jobs that scan large tables are slow. TiDB is not fully utilizing TiKV read bandwidth. `EXPLAIN ANALYZE` shows long scan times. + +**What to do:** + +```sql +-- Increase scan parallelism +SET tidb_distsql_scan_concurrency = 30; -- default is 15 +``` + +**Benefit:** More concurrent coprocessor requests to TiKV means faster data retrieval. Each request scans a different range of data in parallel. + +**When to increase:** When TiKV has spare read bandwidth and the query is scan-bound (most time is spent in TableReader/IndexReader). + +**When to decrease:** When TiKV is under pressure from too many concurrent scan requests, or when you want to limit the impact of one query on others. For OLTP workloads with many concurrent small queries, the default of 15 is usually sufficient. + +### Queries use too much memory on wide tables + +**Symptom:** Queries on tables with many columns or large column values use more memory than expected, even with reasonable result set sizes. + +**What to do:** + +```sql +-- Reduce chunk size to lower per-batch memory usage +SET tidb_max_chunk_size = 256; -- default is 1024 +``` + +**Benefit:** Smaller chunks mean less data is held in memory at once during execution. Each processing step works on fewer rows, reducing peak memory usage. + +**Trade-off:** Smaller chunks mean more processing iterations, which can reduce throughput. Only lower this when memory is the constraint, not speed. + +### High-throughput batch processing needs larger chunks + +**Symptom:** A batch ETL job or analytical query has plenty of memory headroom but throughput is lower than expected. CPU utilization is moderate. + +**What to do:** + +```sql +SET tidb_max_chunk_size = 4096; -- or higher +SET tidb_distsql_scan_concurrency = 30; +``` + +**Benefit:** Larger chunks amortize per-batch overhead and improve CPU efficiency. Combined with higher scan concurrency, this maximizes throughput for data-intensive operations. + +## Common tuning recipes + +### Conservative OLTP session (protect the cluster) + +```sql +SET tidb_mem_quota_query = 536870912; -- 512 MB +SET tidb_max_chunk_size = 512; +SET tidb_distsql_scan_concurrency = 10; +``` + +### Aggressive batch/ETL session (maximize throughput) + +```sql +SET tidb_mem_quota_query = 4294967296; -- 4 GB +SET tidb_max_chunk_size = 4096; +SET tidb_distsql_scan_concurrency = 30; +``` + +### Mixed workload — limit impact of one heavy query + +```sql +-- Per-query via SET_VAR +SELECT /*+ SET_VAR(tidb_mem_quota_query=2147483648) SET_VAR(tidb_distsql_scan_concurrency=20) */ * +FROM large_table WHERE ...; +``` + +## General guidance + +- `tidb_mem_quota_query` is your primary defense against runaway queries. Don't raise it globally without understanding your concurrency profile. +- `tidb_distsql_scan_concurrency` directly affects TiKV read pressure. Monitor TiKV CPU and I/O when increasing it. +- `tidb_max_chunk_size` is a throughput-vs-memory trade-off. The default of 1024 is a good balance for most workloads. +- In connection-pooled environments, session-level changes persist for the life of the connection. Reset variables after batch jobs or use per-query `SET_VAR`. diff --git a/skills/tidb-query-tuning/how-to/plan-stability-and-baselines.md b/skills/tidb-query-tuning/how-to/plan-stability-and-baselines.md new file mode 100644 index 0000000..0f824c4 --- /dev/null +++ b/skills/tidb-query-tuning/how-to/plan-stability-and-baselines.md @@ -0,0 +1,186 @@ +# How To: Manage Plan Stability with SQL Plan Baselines + +These variables control TiDB's SQL Plan Management (SPM) feature, which lets you lock query plans to prevent regressions. Use them when you need to guarantee that a critical query always uses a known-good plan, regardless of stats changes, upgrades, or optimizer behavior changes. + +## Variables at a glance + +| Variable | Default | What it controls | +|----------|---------|-----------------| +| `tidb_enable_plan_baselines` | ON | Capture plan baselines automatically | +| `tidb_use_plan_baseline` | ON | Use captured baselines when available | +| `tidb_mem_quota_binding_cache` | 67108864 (64 MB) | Memory limit for the binding cache | + +## How SQL Plan Baselines work + +1. **Capture**: When `tidb_enable_plan_baselines = ON`, TiDB records the execution plan for each query the first time it's seen. +2. **Bind**: The captured plan becomes a baseline. You can also create bindings manually. +3. **Enforce**: When `tidb_use_plan_baseline = ON`, the optimizer checks for a matching baseline before planning. If one exists, it uses the bound plan instead of re-optimizing. + +This means: +- New plans are captured as they appear +- Existing baselines prevent plan drift +- You can manually bind a specific plan to override the optimizer + +## When to use each variable + +### Prevent plan regressions after stats refresh or upgrade + +**Symptom:** A critical query's plan changes after `ANALYZE TABLE` runs or after a TiDB version upgrade. The new plan is slower. + +**What to do:** + +```sql +-- 1. Ensure baselines are active (these are the defaults) +SET GLOBAL tidb_enable_plan_baselines = ON; +SET GLOBAL tidb_use_plan_baseline = ON; + +-- 2. Manually create a binding for the critical query using the known-good plan +CREATE GLOBAL BINDING FOR + SELECT * FROM orders WHERE status = 'pending' ORDER BY created_at +USING + SELECT /*+ USE_INDEX(orders, idx_status_created) */ * FROM orders WHERE status = 'pending' ORDER BY created_at; +``` + +**Benefit:** The optimizer will always use the bound plan for this query, regardless of cost estimation changes. This is the strongest guarantee of plan stability. + +### Temporarily disable baselines to debug a plan issue + +**Symptom:** You suspect a baseline is forcing a suboptimal plan. You want to see what the optimizer would choose without baselines. + +**What to do:** + +```sql +-- Disable baseline usage for this session +SET tidb_use_plan_baseline = OFF; + +-- Check the plan the optimizer would choose naturally +EXPLAIN ANALYZE SELECT ...; + +-- Compare with the baseline plan +SET tidb_use_plan_baseline = ON; +EXPLAIN ANALYZE SELECT ...; +``` + +**Benefit:** Lets you compare the baseline plan with the optimizer's current best plan to determine if the baseline is still optimal. + +### Global bindings are overriding per-query hints + +**Symptom:** You've added hints to a specific query, but they're being ignored. A global binding for the same query pattern takes precedence. + +**What to do:** + +```sql +-- Check existing bindings +SHOW GLOBAL BINDINGS WHERE Original_sql LIKE '%orders%'; + +-- Option 1: Drop the conflicting binding +DROP GLOBAL BINDING FOR SELECT * FROM orders WHERE ...; + +-- Option 2: Disable baselines for this session to let hints work +SET tidb_enable_plan_baselines = OFF; +``` + +**Benefit:** Removes the conflict between manual hints and automatic bindings. Per-query hints are more precise and should take precedence for targeted tuning. + +### The binding cache is too small for a large number of bindings + +**Symptom:** Some bindings are being evicted from cache. The log shows binding cache quota warnings. Queries intermittently don't use their expected plans. + +**What to do:** + +```sql +-- Increase the binding cache quota +SET GLOBAL tidb_mem_quota_binding_cache = 134217728; -- 128 MB (default is 64 MB) +``` + +**Benefit:** More bindings can be cached in memory, ensuring they're available when queries are compiled. + +**When to increase:** When you have many distinct query patterns with bindings (hundreds or thousands). Check the binding count: + +```sql +SELECT COUNT(*) FROM mysql.bind_info WHERE status = 'enabled'; +``` + +### Stop capturing new baselines but keep using existing ones + +**Symptom:** You have a stable set of bindings and don't want new ones to be created automatically, which could lock in a bad plan. + +**What to do:** + +```sql +-- Stop automatic capture but keep enforcing existing baselines +SET GLOBAL tidb_enable_plan_baselines = OFF; +SET GLOBAL tidb_use_plan_baseline = ON; +``` + +**Benefit:** Existing bindings continue to stabilize plans, but no new plans are automatically captured. New query patterns get the optimizer's natural cost-based selection. + +## Diagnostic workflow + +### 1. Check if a binding exists for your query + +```sql +SHOW GLOBAL BINDINGS WHERE Original_sql LIKE '%your_table%'; +``` + +### 2. Check if the binding is being used + +```sql +EXPLAIN SELECT ...; +-- Look for "Using binding: true" in the output +``` + +### 3. Check binding cache health + +```sql +-- Check memory usage +SHOW GLOBAL BINDINGS; + +-- Check for eviction warnings in TiDB logs +``` + +### 4. Evolve a binding to a better plan + +```sql +-- If the optimizer finds a better plan, you can evolve the binding: +-- First, drop the old binding +DROP GLOBAL BINDING FOR SELECT ...; + +-- Then either let auto-capture create a new one, or manually bind the better plan +CREATE GLOBAL BINDING FOR + SELECT ... +USING + SELECT /*+ desired hints */ ...; +``` + +## Common tuning recipes + +### Lock down critical queries before an upgrade + +```sql +-- Before upgrading, create explicit bindings for your top critical queries +CREATE GLOBAL BINDING FOR USING ; +CREATE GLOBAL BINDING FOR USING ; + +-- Verify they're active +SHOW GLOBAL BINDINGS; +``` + +### Clean up stale bindings + +```sql +-- Find bindings that haven't been used recently +SELECT original_sql, update_time FROM mysql.bind_info +WHERE status = 'enabled' ORDER BY update_time ASC; + +-- Drop ones that are no longer needed +DROP GLOBAL BINDING FOR ; +``` + +## General guidance + +- Baselines are most valuable for critical, high-frequency queries where plan stability matters more than plan optimality. +- For exploratory or ad-hoc queries, baselines can prevent the optimizer from finding better plans as data changes. Consider disabling capture for analytical workloads. +- Manually created bindings override auto-captured ones. Use manual bindings for your most critical queries. +- After a major TiDB upgrade, review your bindings — the optimizer may now produce better plans that your bindings are preventing. +- The binding cache is shared across all connections. Size it based on your total number of active bindings, not per-connection. diff --git a/skills/tidb-query-tuning/how-to/query-execution-tuning.md b/skills/tidb-query-tuning/how-to/query-execution-tuning.md new file mode 100644 index 0000000..fa714d1 --- /dev/null +++ b/skills/tidb-query-tuning/how-to/query-execution-tuning.md @@ -0,0 +1,145 @@ +# How To: Tune Query Execution (Joins, Scans & Rewrites) + +These variables control how TiDB executes joins, scans, and internal query rewrites. Use them when the optimizer picks the wrong join strategy, join order, or fails to apply beneficial rewrites. + +## Variables at a glance + +| Variable | Default | What it controls | +|----------|---------|-----------------| +| `tidb_opt_prefer_range_scan` | OFF | Prefer range scans over full table scans | +| `tidb_opt_enable_hash_join` | ON | Master switch for hash join | +| `tidb_index_join_batch_size` | 25000 | Batch size for index join outer side | +| `tidb_hash_join_concurrency` | 5 | Parallel hash join workers | +| `tidb_opt_insubq_to_join_and_agg` | ON | Rewrite IN subqueries to joins | +| `tidb_opt_derive_topn` | ON | Push TopN through joins/projections | +| `tidb_opt_enable_late_materialization` | ON | Delay column reads until after filtering | +| `tidb_opt_advanced_join_hint` | OFF (older clusters) | Enable advanced join hint syntax | +| `tidb_opt_join_reorder_threshold` | 0 | Join count threshold for greedy vs DP reorder | +| `tidb_enable_outer_join_reorder` | ON | Allow outer join reordering | + +## When to use each variable + +### The optimizer picks hash join but index join would be faster + +**Symptom:** A join query builds a large hash table in memory when the inner table has a selective index on the join key. `EXPLAIN ANALYZE` shows HashJoin with high build time. + +**What to do:** + +```sql +-- Option 1: Increase index join batch size to improve throughput +SET tidb_index_join_batch_size = 50000; + +-- Option 2: Disable hash join entirely (use sparingly) +SET tidb_opt_enable_hash_join = OFF; +``` + +**Benefit:** Index join probes the inner table via its index for each batch of outer rows. When the inner index is selective, this avoids building a large hash table and uses less memory. + +**Caution:** Disabling hash join globally is rarely the right answer. Prefer per-query hints (`INL_JOIN`) or cost factors instead. Only use `tidb_opt_enable_hash_join = OFF` when you're certain no query in the session benefits from hash join. + +### Hash joins are slow because of low parallelism + +**Symptom:** Hash join is the right strategy, but execution is slower than expected. The TiDB node has idle CPU during the join phase. + +**What to do:** + +```sql +SET tidb_hash_join_concurrency = 8; -- or higher, up to available cores +``` + +**Benefit:** More workers build and probe the hash table in parallel, improving throughput on multi-core machines with large joins. + +### The optimizer underestimates filter selectivity and chooses full scans + +**Symptom:** A query with a WHERE clause that filters most rows still uses a TableFullScan instead of a range scan. The stats may not reflect recent data changes. + +**What to do:** + +```sql +SET tidb_opt_prefer_range_scan = ON; +``` + +**Benefit:** Forces the optimizer to prefer range scans even when the cost estimate says a full scan is cheaper. This is a useful safety net when statistics are known to be imprecise. + +**When to turn it off:** Once statistics are refreshed (`ANALYZE TABLE`), re-evaluate whether this is still needed. It can hurt queries where a full scan genuinely is cheaper. + +### IN subquery rewrite produces a worse plan + +**Symptom:** A query with `WHERE col IN (SELECT ...)` gets rewritten to a join with aggregation, but the rewritten plan is slower — for example, the join explodes the row count before aggregation reduces it. + +**What to do:** + +```sql +SET tidb_opt_insubq_to_join_and_agg = OFF; +``` + +**Benefit:** Keeps the subquery as a semi-join or correlated subquery, which can be more efficient when the subquery result set is small. + +### Late materialization hurts a scan-heavy query + +**Symptom:** Rarely, delaying column reads until after filtering can be counterproductive — for example, when the filter eliminates very few rows and the extra read step adds overhead. + +**What to do:** + +```sql +SET tidb_opt_enable_late_materialization = OFF; +``` + +**Benefit:** Fetches all columns in a single pass. Only disable this if you've confirmed via `EXPLAIN ANALYZE` that late materialization adds overhead for your specific query pattern. + +### Join order is wrong for multi-table queries + +**Symptom:** The optimizer picks a join order that processes the largest table first, or pairs tables in an order that produces large intermediate results. + +**What to do:** + +```sql +-- Lower the threshold to use dynamic-programming (DP) reorder for more joins +-- DP is more thorough but slower to plan; greedy is faster but less optimal +SET tidb_opt_join_reorder_threshold = 0; -- 0 = always use DP (default) + +-- If an upgrade caused a regression with outer join reordering: +SET tidb_enable_outer_join_reorder = OFF; +``` + +**Benefit:** DP-based reorder evaluates more join orderings and finds better plans for complex multi-join queries. Disabling outer join reorder can fix regressions where the optimizer incorrectly reorders outer joins after a version upgrade. + +### Advanced join hints don't work on an upgraded cluster + +**Symptom:** You're using advanced join hint syntax (e.g., `LEADING`, `HASH_JOIN_BUILD`) but they're ignored. The cluster was upgraded from an older version where `tidb_opt_advanced_join_hint` defaults to OFF. + +**What to do:** + +```sql +-- Enable globally after upgrade +SET GLOBAL tidb_opt_advanced_join_hint = ON; + +-- Or per-query +SELECT /*+ SET_VAR(tidb_opt_advanced_join_hint=1) LEADING(t1, t2) */ ... +``` + +**Benefit:** Unlocks the full set of join hints including `LEADING()`, `HASH_JOIN_BUILD()`, and `HASH_JOIN_PROBE()`. + +## Common tuning recipes + +### High-throughput batch join session + +```sql +SET tidb_hash_join_concurrency = 8; +SET tidb_index_join_batch_size = 50000; +SET tidb_distsql_scan_concurrency = 30; +``` + +### OLTP session where range scans should always win + +```sql +SET tidb_opt_prefer_range_scan = ON; +SET tidb_opt_enable_late_materialization = ON; +``` + +## General guidance + +- Always validate changes with `EXPLAIN ANALYZE` before and after. +- Prefer per-query hints or `SET_VAR` over session variables when the fix is query-specific. +- Session variables affect all queries in the connection — be careful in connection-pooled environments. +- These variables interact with cost factors: if you've also adjusted `tidb_opt_*_cost_factor` variables, test the combined effect. diff --git a/skills/tidb-query-tuning/how-to/statistics-collection-and-auto-analyze.md b/skills/tidb-query-tuning/how-to/statistics-collection-and-auto-analyze.md new file mode 100644 index 0000000..731c344 --- /dev/null +++ b/skills/tidb-query-tuning/how-to/statistics-collection-and-auto-analyze.md @@ -0,0 +1,184 @@ +# How To: Tune Statistics Collection and Auto Analyze + +These variables control how TiDB collects, refreshes, and maintains table statistics. Good statistics are the foundation of good query plans — if the optimizer doesn't know your data distribution, it can't pick the right join order, access path, or execution engine. + +## Variables at a glance + +| Variable | Default | What it controls | +|----------|---------|-----------------| +| `tidb_analyze_version` | 2 | Stats collection algorithm (v1 or v2) | +| `tidb_analyze_column_options` | PREDICATE | Which columns to collect stats for | +| `tidb_analyze_skip_column_types` | blob,mediumblob,longblob,json | Column types to exclude from stats | +| `tidb_auto_analyze_ratio` | 0.5 | Row-change threshold to trigger auto analyze | +| `tidb_auto_analyze_concurrency` | 1 (v8.5+) | Parallel auto analyze jobs | +| `tidb_auto_build_stats_concurrency` | 1 | Parallel stats building tasks | +| `tidb_build_sampling_stats_concurrency` | 2 | Parallel column sampling workers | +| `tidb_auto_analyze_partition_batch_size` | 128 | Partitions per analyze batch | +| `tidb_merge_partition_stats_concurrency` | 1 | Parallel global stats merge workers | +| `tidb_analyze_distsql_scan_concurrency` | 2 | Scan concurrency for analyze | +| `tidb_sysproc_scan_concurrency` | 1 | System process scan concurrency | +| `tidb_enable_async_merge_global_stats` | OFF | Async merge for partitioned table stats | +| `tidb_enable_historical_stats` | OFF | Store historical stats snapshots | +| `tidb_enable_stats_owner` | ON | Whether this TiDB node runs analyze jobs | +| `tidb_max_auto_analyze_time` | 43200 (12h) | Max time for a single auto analyze job | + +## Common scenarios + +### Stats are going stale faster than auto analyze can refresh them + +**Symptom:** `SHOW STATS_HEALTHY` shows tables in the unhealthy range ([0, 60]) and the count is growing over time. Dashboard shows Auto Analyze Query Per Minute frequently drops to zero while stale tables exist. + +**What to do:** + +```sql +-- Step 1: Lower the trigger threshold so analyze starts sooner +SET GLOBAL tidb_auto_analyze_ratio = 0.3; -- trigger at 30% row change (default 0.5) +``` + +**Benefit:** Tables become eligible for auto analyze earlier, preventing stats from decaying to the point where plans degrade. + +**Caution:** Lowering this too aggressively (e.g., 0.1) means more frequent analyze runs. Only lower it when TiDB and TiKV still have headroom. + +### Auto analyze is triggered but finishes too slowly + +**Symptom:** `SHOW ANALYZE STATUS` shows long-running analyze jobs. The backlog keeps growing despite analyze tasks running. + +**What to do:** + +```sql +-- Increase parallel analyze jobs (v8.5+) +SET GLOBAL tidb_auto_analyze_concurrency = 3; + +-- Increase parallel stats building +SET GLOBAL tidb_auto_build_stats_concurrency = 4; + +-- Increase scan concurrency for analyze +SET GLOBAL tidb_analyze_distsql_scan_concurrency = 4; +``` + +**Benefit:** More parallel work means higher analyze throughput. Multiple tables or partitions can be analyzed simultaneously. + +**Important:** These parameters are multiplicative. `auto_analyze_concurrency * auto_build_stats_concurrency * build_sampling_stats_concurrency` is the effective parallelism. Keep this within what TiDB CPU can sustain — overdoing it creates contention and makes analyze slower. + +### Wide tables are slow to analyze + +**Symptom:** Tables with many columns take a long time to analyze, even though most columns aren't used in WHERE clauses. + +**What to do:** + +```sql +-- Only collect stats for columns that appear in predicates +SET GLOBAL tidb_analyze_column_options = 'PREDICATE'; + +-- Skip large, low-value column types +SET GLOBAL tidb_analyze_skip_column_types = 'json,blob,mediumblob,longblob,text,mediumtext,longtext'; + +-- Increase column sampling parallelism for the remaining columns +SET GLOBAL tidb_build_sampling_stats_concurrency = 4; +``` + +**Benefit:** Reduces the volume of data that analyze must process. Skipping JSON/blob columns avoids reading large values that don't contribute to plan quality. + +**When to use ALL instead of PREDICATE:** If your workload is analytical (OLAP) with varied query shapes, use `tidb_analyze_column_options = 'ALL'` to ensure the optimizer has full coverage. `PREDICATE` works best for stable OLTP workloads. + +### Partitioned tables dominate the analyze backlog + +**Symptom:** Tables with hundreds or thousands of partitions take a long time because each partition is analyzed individually, then global stats are merged. + +**What to do:** + +```sql +-- Increase partition batch size +SET GLOBAL tidb_auto_analyze_partition_batch_size = 256; + +-- Enable async global stats merge (reduces memory spikes) +SET GLOBAL tidb_enable_async_merge_global_stats = ON; + +-- Increase merge concurrency +SET GLOBAL tidb_merge_partition_stats_concurrency = 4; + +-- Increase partition-level build concurrency +SET GLOBAL tidb_auto_build_stats_concurrency = 4; +``` + +**Benefit:** Larger batches reduce the number of merge rounds. Async merge prevents memory spikes. Higher concurrency processes more partitions in parallel. + +### Analyze jobs interfere with business traffic + +**Symptom:** TiKV latency or CPU increases when analyze is running. Business queries slow down during analyze windows. + +**What to do:** + +```sql +-- Option 1: Dedicate specific TiDB nodes to stats work +-- On business-serving TiDB nodes: +SET GLOBAL tidb_enable_stats_owner = OFF; + +-- On dedicated stats TiDB nodes: +SET GLOBAL tidb_enable_stats_owner = ON; +``` + +```sql +-- Option 2: Reduce scan concurrency to limit TiKV pressure +SET GLOBAL tidb_analyze_distsql_scan_concurrency = 1; +SET GLOBAL tidb_sysproc_scan_concurrency = 1; +``` + +**Benefit:** Stats-owner isolation ensures analyze work doesn't compete with latency-sensitive SQL. Reducing scan concurrency limits the read bandwidth analyze consumes from TiKV. + +**Important:** Do not disable `tidb_enable_stats_owner` on ALL TiDB nodes — this effectively disables cluster-wide auto analyze. + +### Cluster is on stats v1 or mixed versions + +**Symptom:** `SELECT @@tidb_analyze_version` returns 1, or different tables have stats collected with different versions. + +**What to do:** + +```sql +-- Switch to v2 +SET GLOBAL tidb_analyze_version = 2; + +-- Re-analyze critical tables +ANALYZE TABLE db.table1; +ANALYZE TABLE db.table2; +``` + +**Benefit:** Stats v2 has better estimation for skewed data, better out-of-range handling, and more efficient sampling. It's been the default since TiDB 6.5. + +### You want to cap how long a single analyze job can run + +**What to do:** + +```sql +-- Default is 43200 seconds (12 hours) +SET GLOBAL tidb_max_auto_analyze_time = 7200; -- 2 hours +``` + +**Benefit:** Prevents a single runaway analyze job from blocking the analyze queue for hours. Jobs that exceed the limit are cancelled and retried later. + +### Historical stats are consuming too much memory + +**What to do:** + +```sql +SET GLOBAL tidb_enable_historical_stats = OFF; +``` + +**Benefit:** Stops TiDB from storing historical stats snapshots, reducing memory pressure. Only enable this if you have a specific operational need to compare stats across time periods. + +## Recommended tuning order + +1. **Confirm stats v2** — highest-impact single change +2. **Set column coverage policy** — `PREDICATE` for OLTP, `ALL` for OLAP +3. **Exclude memory-heavy column types** — `tidb_analyze_skip_column_types` +4. **Tune the trigger threshold** — `tidb_auto_analyze_ratio` +5. **Tune concurrency** — only after policy choices are correct +6. **Handle partitioned tables** — async merge, partition batch size +7. **Isolate stats work** — dedicated TiDB nodes if needed + +## General guidance + +- Change one variable at a time and observe the effect over hours, not minutes. +- Use dashboard trends (`Stats Healthy Distribution`, `Auto Analyze Duration`) to judge improvement. +- Always add TiKV background read rate limiting before aggressively increasing analyze scan concurrency. +- Re-check query plans after major stats tuning changes — plan choices can legitimately change. diff --git a/skills/tidb-query-tuning/how-to/statistics-loading-and-freshness.md b/skills/tidb-query-tuning/how-to/statistics-loading-and-freshness.md new file mode 100644 index 0000000..f79d368 --- /dev/null +++ b/skills/tidb-query-tuning/how-to/statistics-loading-and-freshness.md @@ -0,0 +1,110 @@ +# How To: Manage Statistics Loading and Freshness + +These variables control how TiDB handles missing or outdated statistics during query optimization. They determine whether the optimizer waits for complete stats before planning, and what happens when stats are stale. + +## Variables at a glance + +| Variable | Default | What it controls | +|----------|---------|-----------------| +| `tidb_stats_load_sync_wait` | 100 (ms) | Max wait time for on-demand stats loading during optimization | +| `tidb_enable_pseudo_for_outdated_stats` | OFF | Whether to fall back to pseudo stats when real stats are outdated | + +## When to use each variable + +### Query plans are unstable after TiDB restart + +**Symptom:** The first wave of queries after a TiDB restart gets different (often worse) plans than steady-state traffic. Plans stabilize after a few minutes as stats are loaded into memory. + +**What to do:** + +```sql +-- Increase the sync-load wait budget so the optimizer waits longer for stats +SET GLOBAL tidb_stats_load_sync_wait = 200; -- 200ms (default is 100ms) +``` + +**Benefit:** The optimizer waits longer for complete column statistics to load before compiling a plan. This produces more stable plans during warm-up at the cost of slightly higher compilation latency. + +**When to increase further:** If `Statistics -> Sync Load QPS` on the dashboard shows frequent timeouts, the wait budget is too short. Increase gradually (200 → 500 → 1000ms) until timeouts are rare. + +**When to decrease:** If compilation latency is a concern and you're running a recent TiDB version (v8.2+) with adaptive stats-load concurrency, the loading path may already be fast enough to use a lower budget. + +### Sync load timeouts are frequent and sustained + +**Symptom:** The dashboard shows a high sustained rate of sync-load timeouts, not just occasional spikes. Plans are frequently compiled against incomplete stats. + +**What to do:** + +```sql +-- First, increase the wait budget +SET GLOBAL tidb_stats_load_sync_wait = 500; +``` + +If that's not enough: +- On versions before v8.2: increase `stats-load-concurrency` in the TiDB configuration file +- On v8.2+: set `stats-load-concurrency = 0` to enable adaptive concurrency based on CPU cores + +**Benefit:** More worker capacity means stats are loaded faster, so the optimizer is less likely to time out waiting. + +### Plans silently degrade when stats become stale + +**Symptom:** After bulk data changes (large INSERTs, DELETEs, or partition operations), query plans degrade but there are no warnings or errors. The optimizer silently uses pseudo statistics. + +**What to do:** + +```sql +-- Ensure pseudo stats are disabled (this is the default since newer versions) +SET GLOBAL tidb_enable_pseudo_for_outdated_stats = OFF; +``` + +**Benefit:** When OFF, TiDB uses the real (albeit outdated) statistics rather than falling back to pseudo stats. This is almost always better because: +- Real outdated stats are still closer to reality than pseudo stats +- The optimizer produces warnings about stale stats, making the problem visible +- You can then address the root cause (run `ANALYZE TABLE` or tune auto analyze) + +**When you might set it ON:** Almost never. The only case is if outdated stats are so wrong that they produce worse plans than pseudo stats — this is extremely rare. + +## Diagnostic workflow + +### 1. Check if stats freshness is causing plan issues + +```sql +-- Check stats health for the relevant tables +SHOW STATS_HEALTHY WHERE Db_name = 'mydb' AND Table_name = 'mytable'; + +-- If health is low, refresh stats +ANALYZE TABLE mydb.mytable; + +-- Check if the plan improves +EXPLAIN ANALYZE SELECT ...; +``` + +### 2. Check if sync load is a bottleneck + +Look at the TiDB dashboard: +- `Statistics -> Sync Load QPS` — if timeouts are frequent, increase `tidb_stats_load_sync_wait` +- `Statistics -> Sync Load Duration` — if durations are near the timeout, the budget is too tight + +### 3. After restart, verify warm-up behavior + +```sql +-- Immediately after restart, check a critical query's plan +EXPLAIN SELECT ...; + +-- Wait 30 seconds, check again +EXPLAIN SELECT ...; + +-- If plans differ, the warm-up period is too long — increase sync wait or upgrade +``` + +## Interaction with other settings + +- **`tidb_auto_analyze_ratio`**: Controls when auto analyze triggers. If stats decay faster than auto analyze runs, you'll see more stale-stats issues regardless of these loading variables. See the [Statistics Collection guide](statistics-collection-and-auto-analyze.md). +- **force-init-stats**: A TiDB configuration option (not a session variable) that makes TiDB wait for full stats initialization before serving traffic. Use it when the first queries after restart must be stable. +- **Stats version**: These loading variables work with both stats v1 and v2, but v2 loads more efficiently. + +## General guidance + +- Keep `tidb_enable_pseudo_for_outdated_stats = OFF` — this is the safe default. +- Start with `tidb_stats_load_sync_wait = 100` and only increase if you see sync-load timeouts. +- If you're on an older TiDB version and seeing restart-time plan drift, upgrading is usually a better fix than tuning these variables. +- After changing stats loading behavior, validate first-query latency and plan stability for your critical queries. diff --git a/skills/tidb-query-tuning/how-to/tiflash-and-mpp-execution.md b/skills/tidb-query-tuning/how-to/tiflash-and-mpp-execution.md new file mode 100644 index 0000000..b637542 --- /dev/null +++ b/skills/tidb-query-tuning/how-to/tiflash-and-mpp-execution.md @@ -0,0 +1,156 @@ +# How To: Control TiFlash and MPP Execution + +These variables control whether and how TiDB uses TiFlash (the columnar storage engine) and MPP (Massively Parallel Processing) for query execution. Use them to route analytical queries to TiFlash, force MPP execution, or prevent TiFlash from being used when it's not appropriate. + +## Variables at a glance + +| Variable | Default | What it controls | +|----------|---------|-----------------| +| `tidb_allow_mpp` | ON | Master switch for MPP execution | +| `tidb_enforce_mpp` | OFF | Force all eligible queries to use MPP | +| `tidb_isolation_read_engines` | tikv,tiflash,tidb | Which storage engines the optimizer can use | +| `tidb_enable_tiflash_cop` | ON | Allow TiFlash coprocessor (non-MPP) requests | + +## How TiFlash execution works + +TiDB can read from TiFlash in two modes: + +1. **MPP mode**: TiFlash nodes collaborate to execute the query in parallel (shuffles, joins, aggregations happen on TiFlash). Best for analytical queries. +2. **Cop mode**: TiDB sends coprocessor requests to TiFlash, similar to how it reads from TiKV. Simpler but less powerful — no cross-node shuffles. + +The optimizer chooses between TiKV and TiFlash based on cost estimation. These variables let you override that choice. + +## When to use each variable + +### Route an analytical session entirely to TiFlash + +**Symptom:** You're running analytical queries (aggregations, joins on large tables, GROUP BY) and want to ensure they use TiFlash's columnar storage and MPP execution. + +**What to do:** + +```sql +-- Force all reads to TiFlash and enable MPP +SET tidb_isolation_read_engines = 'tiflash'; +SET tidb_enforce_mpp = ON; +``` + +**Benefit:** All table reads go to TiFlash, and the optimizer uses MPP execution for joins, aggregations, and shuffles. This can be orders of magnitude faster for analytical queries on large datasets. + +**Caution:** This will fail for tables that don't have TiFlash replicas. Only use this in sessions where you know all queried tables are replicated to TiFlash. + +### Let the optimizer choose, but allow TiFlash + +**Symptom:** You want the optimizer to consider both TiKV and TiFlash and pick the cheaper option. This is the default behavior. + +**What to do:** + +```sql +-- These are the defaults — set them explicitly if a previous session changed them +SET tidb_isolation_read_engines = 'tikv,tiflash,tidb'; +SET tidb_allow_mpp = ON; +SET tidb_enforce_mpp = OFF; +``` + +**Benefit:** The optimizer uses cost-based selection. Small point queries go to TiKV; large analytical queries go to TiFlash when the cost is lower. + +### Prevent TiFlash from being used + +**Symptom:** TiFlash is deployed in the cluster but you want certain sessions (e.g., OLTP application connections) to never use it, to avoid unexpected latency from TiFlash access. + +**What to do:** + +```sql +-- Remove TiFlash from the engine list +SET tidb_isolation_read_engines = 'tikv,tidb'; +``` + +**Benefit:** Guarantees all reads go to TiKV. Useful for OLTP sessions where you want predictable latency and TiFlash access would be suboptimal. + +### TiFlash cop requests are overloading TiFlash + +**Symptom:** High-frequency small queries are sent to TiFlash as cop (coprocessor) requests instead of MPP. TiFlash CPU spikes from handling many individual cop requests. Dashboard shows high TiFlash cop request volume. + +**What to do:** + +```sql +-- Disable TiFlash cop mode — queries will either use MPP or fall back to TiKV +SET GLOBAL tidb_enable_tiflash_cop = OFF; +``` + +**Benefit:** Prevents TiDB from sending lightweight coprocessor requests to TiFlash. Queries that would have used TiFlash in cop mode will either: +- Use MPP if eligible (better for analytical queries) +- Fall back to TiKV (better for small OLTP queries) + +**When to keep it ON:** If you have queries that benefit from TiFlash's columnar reads but don't qualify for MPP (e.g., simple single-table scans without joins or aggregations). + +### Force MPP for a specific analytical query + +**What to do:** + +```sql +-- Per-query approach +SELECT /*+ SET_VAR(tidb_enforce_mpp=ON) */ + region, SUM(amount), COUNT(*) +FROM orders +GROUP BY region; +``` + +**Benefit:** Forces MPP execution for this single query without affecting other queries in the session. + +## Common tuning recipes + +### Dedicated analytical session + +```sql +SET tidb_isolation_read_engines = 'tiflash'; +SET tidb_enforce_mpp = ON; +SET tidb_mem_quota_query = 4294967296; -- 4 GB for large analytical queries +``` + +### Mixed HTAP workload — optimizer chooses + +```sql +SET tidb_isolation_read_engines = 'tikv,tiflash,tidb'; +SET tidb_allow_mpp = ON; +SET tidb_enforce_mpp = OFF; +``` + +### Pure OLTP session — no TiFlash + +```sql +SET tidb_isolation_read_engines = 'tikv,tidb'; +``` + +## Diagnostic workflow + +### 1. Check if TiFlash is being used + +```sql +EXPLAIN ANALYZE SELECT ...; +-- Look for "tiflash" in the store column or "ExchangeSender/ExchangeReceiver" operators (MPP) +``` + +### 2. Check if TiFlash replicas exist + +```sql +SELECT * FROM information_schema.tiflash_replica WHERE TABLE_SCHEMA = 'mydb'; +``` + +### 3. If TiFlash is available but not chosen + +```sql +-- Check current engine settings +SELECT @@tidb_isolation_read_engines; +SELECT @@tidb_allow_mpp; + +-- Try forcing it to compare performance +SET tidb_enforce_mpp = ON; +EXPLAIN ANALYZE SELECT ...; +``` + +## General guidance + +- `tidb_allow_mpp = ON` is required for any MPP execution. If it's OFF, `tidb_enforce_mpp` has no effect. +- `tidb_enforce_mpp` forces MPP but won't make unsupported operations work on TiFlash. If a query uses functions or operators not supported by TiFlash, it falls back. +- `tidb_isolation_read_engines` is the most powerful control — it completely removes engines from consideration. Use it for session-level workload isolation. +- TiFlash is almost always better for large scans, aggregations, and joins. TiKV is almost always better for point lookups and small range scans. Let the optimizer choose unless you have a clear reason to override.