From b0c84c72d130e887d2b2b0b3467f696fafc6971b Mon Sep 17 00:00:00 2001 From: Jianjun Liao Date: Sun, 26 Jan 2025 15:31:20 +0800 Subject: [PATCH 1/7] add unrecoverable system tables list Signed-off-by: Jianjun Liao --- br/br-system-tables.md | 62 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 br/br-system-tables.md diff --git a/br/br-system-tables.md b/br/br-system-tables.md new file mode 100644 index 0000000000000..f7fa4f966a30e --- /dev/null +++ b/br/br-system-tables.md @@ -0,0 +1,62 @@ +--- +title: the unrecoverable system tables by snapshot restore +summary: list the unrecoverable system tables by snapshot restore +--- + +# the unrecoverable system tables by snapshot restore + +When the cluster is in snapshot backup, BR backs up system tables as tables with the database name prefix with `__TiDB_BR_Temporary`. For example, the table `mysql.user` will be backed up as `__TiDB_BR_Temporary_mysql.user`. + +When the cluster is in snapshot restore, BR at first restores tables with the database name prefix with `__TiDB_BR_Temporary`, in order to avoid conflicts with existing system tables in the cluster. Then when BR starts to restore the system tables, it inserts the data from tables with the database name prefix with `__TiDB_BR_Temporary` to the corresponding system tables through the `REPLACE INTO` SQL. + +The following lists the system tables that cannot be restored by the above methods for snapshot restore. + +* `mysql` + * `advisory_locks` + * `analyze_jobs` + * `analyze_options` + * `capture_plan_baselines_blacklist` + * `column_stats_usage` + * `dist_framework_meta` + * `gc_delete_range` + * `gc_delete_range_done` + * `global_variables` + * `help_topic` + * `index_advisor_results` + * `plan_replayer_status` + * `plan_replayer_task` + * `request_unit_by_group` + * `stats_buckets` + * `stats_extended` + * `stats_feedback` + * `stats_fm_sketch` + * `stats_histograms` + * `stats_history` + * `stats_meta` + * `stats_meta_history` + * `stats_table_locked` + * `stats_top_n` + * `table_cache_meta` + * `tidb` + * `tidb_background_subtask` + * `tidb_background_subtask_history` + * `tidb_ddl_history` + * `tidb_ddl_job` + * `tidb_ddl_notifier` + * `tidb_ddl_reorg` + * `tidb_global_task` + * `tidb_global_task_history` + * `tidb_import_jobs` + * `tidb_mdl_info` + * `tidb_mdl_view` + * `tidb_pitr_id_map` + * `tidb_runaway_queries` + * `tidb_runaway_watch` + * `tidb_runaway_watch_done` + * `tidb_timers` + * `tidb_ttl_job_history` + * `tidb_ttl_table_status` + * `tidb_ttl_task` + +* `sys` + * `schema_unused_indexes` From 2f2713ece105d9e8d14633481b0d449ae93d1637 Mon Sep 17 00:00:00 2001 From: Jianjun Liao Date: Mon, 3 Mar 2025 11:10:39 +0800 Subject: [PATCH 2/7] move to another place Signed-off-by: Jianjun Liao --- br/br-snapshot-guide.md | 35 +++++++++++++++++++- br/br-system-tables.md | 62 ----------------------------------- faq/backup-and-restore-faq.md | 2 +- 3 files changed, 35 insertions(+), 64 deletions(-) delete mode 100644 br/br-system-tables.md diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 4ef4a957630bc..fb749ce22c0b0 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -129,6 +129,10 @@ tiup br restore full \ ### Restore tables in the `mysql` schema +When the cluster is in snapshot backup, BR backs up system tables as tables with the database name prefix with `__TiDB_BR_Temporary`. For example, the table `mysql.user` will be backed up as `__TiDB_BR_Temporary_mysql.user`. + +When the cluster is in snapshot restore, BR at first restores tables with the database name prefix with `__TiDB_BR_Temporary`, in order to avoid conflicts with existing system tables in the cluster. Then when BR starts to restore the system tables, it inserts the data from tables with the database name prefix with `__TiDB_BR_Temporary` to the corresponding system tables through the `REPLACE INTO` SQL. + - Starting from BR v5.1.0, when you back up snapshots, BR automatically backs up the **system tables** in the `mysql` schema, but does not restore these system tables by default. - Starting from v6.2.0, BR lets you specify `--with-sys-table` to restore **data in some system tables**. - Starting from v7.6.0, BR enables `--with-sys-table` by default, which means that BR restores **data in some system tables** by default. @@ -153,15 +157,24 @@ tiup br restore full \ - Statistics tables (`mysql.stat_*`). But statistics can be restored. See [Back up statistics](/br/br-snapshot-manual.md#back-up-statistics). - System variable tables (`mysql.tidb` and `mysql.global_variables`) -- [Other system tables](https://github.com/pingcap/tidb/blob/master/br/pkg/restore/snap_client/systable_restore.go#L31) +- [Other system tables](https://github.com/pingcap/tidb/blob/master/br/pkg/restore/snap_client/systable_restore.go#L66) ``` +-----------------------------------------------------+ +| advisory_locks | +| analyze_jobs | +| analyze_options | | capture_plan_baselines_blacklist | | column_stats_usage | +| dist_framework_meta | | gc_delete_range | | gc_delete_range_done | | global_variables | +| help_topic | +| index_advisor_results | +| plan_replayer_status | +| plan_replayer_task | +| request_unit_by_group | | stats_buckets | | stats_extended | | stats_feedback | @@ -172,7 +185,27 @@ tiup br restore full \ | stats_meta_history | | stats_table_locked | | stats_top_n | +| table_cache_meta | | tidb | +| tidb_background_subtask | +| tidb_background_subtask_history | +| tidb_ddl_history | +| tidb_ddl_job | +| tidb_ddl_notifier | +| tidb_ddl_reorg | +| tidb_global_task | +| tidb_global_task_history | +| tidb_import_jobs | +| tidb_mdl_info | +| tidb_mdl_view | +| tidb_pitr_id_map | +| tidb_runaway_queries | +| tidb_runaway_watch | +| tidb_runaway_watch_done | +| tidb_timers | +| tidb_ttl_job_history | +| tidb_ttl_table_status | +| tidb_ttl_task | +-----------------------------------------------------+ ``` diff --git a/br/br-system-tables.md b/br/br-system-tables.md deleted file mode 100644 index f7fa4f966a30e..0000000000000 --- a/br/br-system-tables.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -title: the unrecoverable system tables by snapshot restore -summary: list the unrecoverable system tables by snapshot restore ---- - -# the unrecoverable system tables by snapshot restore - -When the cluster is in snapshot backup, BR backs up system tables as tables with the database name prefix with `__TiDB_BR_Temporary`. For example, the table `mysql.user` will be backed up as `__TiDB_BR_Temporary_mysql.user`. - -When the cluster is in snapshot restore, BR at first restores tables with the database name prefix with `__TiDB_BR_Temporary`, in order to avoid conflicts with existing system tables in the cluster. Then when BR starts to restore the system tables, it inserts the data from tables with the database name prefix with `__TiDB_BR_Temporary` to the corresponding system tables through the `REPLACE INTO` SQL. - -The following lists the system tables that cannot be restored by the above methods for snapshot restore. - -* `mysql` - * `advisory_locks` - * `analyze_jobs` - * `analyze_options` - * `capture_plan_baselines_blacklist` - * `column_stats_usage` - * `dist_framework_meta` - * `gc_delete_range` - * `gc_delete_range_done` - * `global_variables` - * `help_topic` - * `index_advisor_results` - * `plan_replayer_status` - * `plan_replayer_task` - * `request_unit_by_group` - * `stats_buckets` - * `stats_extended` - * `stats_feedback` - * `stats_fm_sketch` - * `stats_histograms` - * `stats_history` - * `stats_meta` - * `stats_meta_history` - * `stats_table_locked` - * `stats_top_n` - * `table_cache_meta` - * `tidb` - * `tidb_background_subtask` - * `tidb_background_subtask_history` - * `tidb_ddl_history` - * `tidb_ddl_job` - * `tidb_ddl_notifier` - * `tidb_ddl_reorg` - * `tidb_global_task` - * `tidb_global_task_history` - * `tidb_import_jobs` - * `tidb_mdl_info` - * `tidb_mdl_view` - * `tidb_pitr_id_map` - * `tidb_runaway_queries` - * `tidb_runaway_watch` - * `tidb_runaway_watch_done` - * `tidb_timers` - * `tidb_ttl_job_history` - * `tidb_ttl_table_status` - * `tidb_ttl_task` - -* `sys` - * `schema_unused_indexes` diff --git a/faq/backup-and-restore-faq.md b/faq/backup-and-restore-faq.md index 3055c24ac4006..0ce1283292dd9 100644 --- a/faq/backup-and-restore-faq.md +++ b/faq/backup-and-restore-faq.md @@ -276,7 +276,7 @@ Note that even if you configures [table filter](/table-filter.md#syntax), **BR d - Statistics tables (`mysql.stat_*`). But statistics can be restored. See [Back up statistics](/br/br-snapshot-manual.md#back-up-statistics). - System variable tables (`mysql.tidb`, `mysql.global_variables`) -- [Other system tables](https://github.com/pingcap/tidb/blob/master/br/pkg/restore/snap_client/systable_restore.go#L31) +- [Other system tables](https://github.com/pingcap/tidb/blob/master/br/pkg/restore/snap_client/systable_restore.go#L66) ### How to deal with the error of `cannot find rewrite rule` during restoration? From 13296cb8ee8329119187c855a430a2e2ae2f8ea9 Mon Sep 17 00:00:00 2001 From: Aolin Date: Thu, 6 Mar 2025 15:45:11 +0800 Subject: [PATCH 3/7] revise --- br/br-snapshot-guide.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index fb749ce22c0b0..95ae52f483e90 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -129,9 +129,7 @@ tiup br restore full \ ### Restore tables in the `mysql` schema -When the cluster is in snapshot backup, BR backs up system tables as tables with the database name prefix with `__TiDB_BR_Temporary`. For example, the table `mysql.user` will be backed up as `__TiDB_BR_Temporary_mysql.user`. - -When the cluster is in snapshot restore, BR at first restores tables with the database name prefix with `__TiDB_BR_Temporary`, in order to avoid conflicts with existing system tables in the cluster. Then when BR starts to restore the system tables, it inserts the data from tables with the database name prefix with `__TiDB_BR_Temporary` to the corresponding system tables through the `REPLACE INTO` SQL. +When you perform a snapshot backup, BR backs up system tables as tables with the `__TiDB_BR_Temporary_` prefix added to the database name. For example, BR backs up the `mysql.user` table as `__TiDB_BR_Temporary_mysql.user`. During the snapshot restore, BR first restores these system tables with the `__TiDB_BR_Temporary_` prefix to avoid conflicts with existing system table data in the target cluster. While restoring system tables, BR writes the data from the tables with the `__TiDB_BR_Temporary_` prefix into the corresponding system tables using the `REPLACE INTO` statement. - Starting from BR v5.1.0, when you back up snapshots, BR automatically backs up the **system tables** in the `mysql` schema, but does not restore these system tables by default. - Starting from v6.2.0, BR lets you specify `--with-sys-table` to restore **data in some system tables**. From c9418c6328d2cdc5311258b444b6a4eebd5d3e62 Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 10 Mar 2025 11:23:29 +0800 Subject: [PATCH 4/7] Apply suggestions from code review Co-authored-by: Grace Cai --- br/br-snapshot-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 95ae52f483e90..e930894f95127 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -129,7 +129,7 @@ tiup br restore full \ ### Restore tables in the `mysql` schema -When you perform a snapshot backup, BR backs up system tables as tables with the `__TiDB_BR_Temporary_` prefix added to the database name. For example, BR backs up the `mysql.user` table as `__TiDB_BR_Temporary_mysql.user`. During the snapshot restore, BR first restores these system tables with the `__TiDB_BR_Temporary_` prefix to avoid conflicts with existing system table data in the target cluster. While restoring system tables, BR writes the data from the tables with the `__TiDB_BR_Temporary_` prefix into the corresponding system tables using the `REPLACE INTO` statement. +When you perform a snapshot backup, BR backs up system tables as tables with the `__TiDB_BR_Temporary_` prefix added to the database name. For example, BR backs up the `mysql.user` table as `__TiDB_BR_Temporary_mysql.user`. During the snapshot restore, BR first restores these tables with the `__TiDB_BR_Temporary_` prefix to avoid conflicts with existing system table data in the target cluster. While restoring system tables, BR writes the data from the tables with the `__TiDB_BR_Temporary_` prefix into the corresponding system tables using the `REPLACE INTO` statement. - Starting from BR v5.1.0, when you back up snapshots, BR automatically backs up the **system tables** in the `mysql` schema, but does not restore these system tables by default. - Starting from v6.2.0, BR lets you specify `--with-sys-table` to restore **data in some system tables**. From 656ef1f0cabaf8ede0ac39ce2ad7a6067ad4f18d Mon Sep 17 00:00:00 2001 From: Jianjun Liao Date: Fri, 21 Mar 2025 10:06:58 +0800 Subject: [PATCH 5/7] fix link Signed-off-by: Jianjun Liao --- br/br-snapshot-guide.md | 2 +- faq/backup-and-restore-faq.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index e930894f95127..d2a6d2cd4339a 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -155,7 +155,7 @@ When you perform a snapshot backup, BR backs up system tables as tables with the - Statistics tables (`mysql.stat_*`). But statistics can be restored. See [Back up statistics](/br/br-snapshot-manual.md#back-up-statistics). - System variable tables (`mysql.tidb` and `mysql.global_variables`) -- [Other system tables](https://github.com/pingcap/tidb/blob/master/br/pkg/restore/snap_client/systable_restore.go#L66) +- Other system tables. See [Unrecoverable system tables](/br/br-snapshot-guide.md#restore-tables-in-the-mysql-schema) ``` +-----------------------------------------------------+ diff --git a/faq/backup-and-restore-faq.md b/faq/backup-and-restore-faq.md index 0ce1283292dd9..0c76ac8c6d734 100644 --- a/faq/backup-and-restore-faq.md +++ b/faq/backup-and-restore-faq.md @@ -276,7 +276,7 @@ Note that even if you configures [table filter](/table-filter.md#syntax), **BR d - Statistics tables (`mysql.stat_*`). But statistics can be restored. See [Back up statistics](/br/br-snapshot-manual.md#back-up-statistics). - System variable tables (`mysql.tidb`, `mysql.global_variables`) -- [Other system tables](https://github.com/pingcap/tidb/blob/master/br/pkg/restore/snap_client/systable_restore.go#L66) +- Other system tables ### How to deal with the error of `cannot find rewrite rule` during restoration? From b405fdd9d5c086874e8ec1c34a4c1db05c69184d Mon Sep 17 00:00:00 2001 From: Jianjun Liao Date: Fri, 21 Mar 2025 10:14:42 +0800 Subject: [PATCH 6/7] fix link Signed-off-by: Jianjun Liao --- br/br-snapshot-guide.md | 2 +- faq/backup-and-restore-faq.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index d2a6d2cd4339a..9b216120d6f33 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -155,7 +155,7 @@ When you perform a snapshot backup, BR backs up system tables as tables with the - Statistics tables (`mysql.stat_*`). But statistics can be restored. See [Back up statistics](/br/br-snapshot-manual.md#back-up-statistics). - System variable tables (`mysql.tidb` and `mysql.global_variables`) -- Other system tables. See [Unrecoverable system tables](/br/br-snapshot-guide.md#restore-tables-in-the-mysql-schema) +- Other system tables ``` +-----------------------------------------------------+ diff --git a/faq/backup-and-restore-faq.md b/faq/backup-and-restore-faq.md index 0c76ac8c6d734..1d358ea0e4e6a 100644 --- a/faq/backup-and-restore-faq.md +++ b/faq/backup-and-restore-faq.md @@ -276,7 +276,7 @@ Note that even if you configures [table filter](/table-filter.md#syntax), **BR d - Statistics tables (`mysql.stat_*`). But statistics can be restored. See [Back up statistics](/br/br-snapshot-manual.md#back-up-statistics). - System variable tables (`mysql.tidb`, `mysql.global_variables`) -- Other system tables +- Other system tables. See [Unrecoverable system tables](/br/br-snapshot-guide.md#restore-tables-in-the-mysql-schema) ### How to deal with the error of `cannot find rewrite rule` during restoration? From 152c64debbf818ebea616ee5d32c2d21d3e4e0af Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 21 Mar 2025 10:33:01 +0800 Subject: [PATCH 7/7] fix typo --- faq/backup-and-restore-faq.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/faq/backup-and-restore-faq.md b/faq/backup-and-restore-faq.md index 1d358ea0e4e6a..38c7d51d183ce 100644 --- a/faq/backup-and-restore-faq.md +++ b/faq/backup-and-restore-faq.md @@ -272,11 +272,11 @@ If you only need to restore `mysql.usertable`, run the following command: br restore full -f 'mysql.usertable' -s $external_storage_url --with-sys-table ``` -Note that even if you configures [table filter](/table-filter.md#syntax), **BR does not restore the following system tables**: +Note that even if you configure [table filter](/table-filter.md#syntax), **BR does not restore the following system tables**: - Statistics tables (`mysql.stat_*`). But statistics can be restored. See [Back up statistics](/br/br-snapshot-manual.md#back-up-statistics). - System variable tables (`mysql.tidb`, `mysql.global_variables`) -- Other system tables. See [Unrecoverable system tables](/br/br-snapshot-guide.md#restore-tables-in-the-mysql-schema) +- Other system tables. For more details, see [Restore tables in the `mysql` schema system tables](/br/br-snapshot-guide.md#restore-tables-in-the-mysql-schema). ### How to deal with the error of `cannot find rewrite rule` during restoration?