feat(backup): auto-include local tables of distributed tables by donhardman · Pull Request #136 · manticoresoftware/manticoresearch-backup

donhardman · 2026-05-26T16:48:17Z

Implement expandDistributedLocals to include child tables in backups
Add getDistributedLocals to retrieve local children of distributed tables
Update validateTables to trigger automatic expansion of distributed tables
Fix indentation and whitespace in ManticoreBackup and ManticoreClient

- Implement expandDistributedLocals to include child tables in backups - Add getDistributedLocals to retrieve local children of distributed tables - Update validateTables to trigger automatic expansion of distributed tables - Fix indentation and whitespace in ManticoreBackup and ManticoreClient

github-actions · 2026-05-26T16:49:33Z

clt

❌ CLT tests in test/clt-tests/
✅ OK: 1
❌ Failed: 7
⏳ Duration: 54s
👉 Check Action Results for commit 83b4a5e

Failed tests:

test/clt-tests/backup-all-ok.rec

––– input –––
cp ${MANTICORESEARCH_CONFIG:-./test/clt-tests/config/manticore.conf} /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
stdbuf -oL searchd
––– output: contains –––
listening on
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 < test/clt-tests/dumps/tables.sql
––– output –––
+ ERROR 1064 (42000) at line 23: error adding table 'test': knn library not loaded
––– input –––
mysql -h0 -P9306 -e 'show tables'
––– output –––
- +---------------------+-------------+
+ +---------------------+------+
- | Table               | Type        |
+ | Table               | Type |
- +---------------------+-------------+
+ +---------------------+------+
- | distributed_index   | distributed |
+ | rt_with_columnar    | rt   |
- | rt_with_columnar    | rt          |
+ | rt_without_columnar | rt   |
- | rt_without_columnar | rt          |
+ +---------------------+------+
- | test                | rt          |
- +---------------------+-------------+
––– input –––
mysql -h0 -P9306 -e 'show create table distributed_index'
––– output –––
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+ ERROR 1146 (42S02) at line 1: no such table 'distributed_index'
- | Table             | Create Table                                                                                                                                                              |
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
- | distributed_index | CREATE TABLE distributed_index type='distributed' local='rt_with_columnar' local='rt_without_columnar' agent='127.0.0.1:9312:plain_with_columnar, plain_without_columnar' |
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
––– input –––
mysql -h0 -P9306 -e 'show create table rt_with_columnar'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e 'show create table rt_without_columnar'
––– output –––
OK
––– input –––
ls -lt /tmp/backup-*
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "flush ramchunk test; select * from test.@files;"
––– output –––
- +------+---------------------------------------+---------------------------------------+--------+
+ ERROR 1064 (42000) at line 1: FLUSH RAMCHUNK requires an existing RT table
- | id   | file                                  | normalized                            | size   |
- +------+---------------------------------------+---------------------------------------+--------+
- |    2 | /var/lib/manticore/test/test.settings | /var/lib/manticore/test/test.settings |      %{NUMBER} |
- |    3 | /var/lib/manticore/test/test.0.sph    | /var/lib/manticore/test/test.0.sph    |    %{NUMBER} |
- |    4 | /var/lib/manticore/test/test.0.spd    | /var/lib/manticore/test/test.0.spd    |     %{NUMBER} |
- |    5 | /var/lib/manticore/test/test.0.spp    | /var/lib/manticore/test/test.0.spp    |      %{NUMBER} |
- |    6 | /var/lib/manticore/test/test.0.spe    | /var/lib/manticore/test/test.0.spe    |      %{NUMBER} |
- |    7 | /var/lib/manticore/test/test.0.spi    | /var/lib/manticore/test/test.0.spi    |     %{NUMBER} |
- |    8 | /var/lib/manticore/test/test.0.spm    | /var/lib/manticore/test/test.0.spm    |      %{NUMBER} |
- |    9 | /var/lib/manticore/test/test.0.sphi   | /var/lib/manticore/test/test.0.sphi   |     %{NUMBER} |
- |   10 | /var/lib/manticore/test/test.0.spds   | /var/lib/manticore/test/test.0.spds   |     %{NUMBER} |
- |   11 | /var/lib/manticore/test/test.0.spknn  | /var/lib/manticore/test/test.0.spknn  |    %{NUMBER} |
- |   12 | /var/lib/manticore/test/test.0.spt    | /var/lib/manticore/test/test.0.spt    |     %{NUMBER} |
- |   13 | /var/lib/manticore/test/test.0.spidx  | /var/lib/manticore/test/test.0.spidx  |    %{NUMBER} |
- |   14 | /var/lib/manticore/test/test.0.spa    | /var/lib/manticore/test/test.0.spa    |     %{NUMBER} |
- |   15 | /var/lib/manticore/test/test.0.spb    | /var/lib/manticore/test/test.0.spb    | %{NUMBER} |
- |    1 | /var/lib/manticore/test/test.meta     | /var/lib/manticore/test/test.meta     |    %{NUMBER} |
- +------+---------------------------------------+---------------------------------------+--------+
––– input –––
manticore-backup --backup-dir=/tmp
––– output –––
Copyright (c) %{YEAR}-%{YEAR}, Manticore Software LTD (https://manticoresearch.com)
Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: all tables
Backup dir: /tmp
Manticore config
  endpoint =  http://%{IPADDR}:9312
Manticore versions:
  manticore: %{VERSION}
  columnar: %{VERSION}
  secondary: %{VERSION}
  knn: %{VERSION}
  buddy: %{VERSION}
%{DATETIME} [Info] Starting the backup...
%{DATETIME} [Info] Backing up config files...
%{DATETIME} [Info]   config files - OK
%{DATETIME} [Info] Backing up global state files...
%{DATETIME} [Info]   global state files – OK
%{DATETIME} [Info] Backing up tables...
- %{DATETIME} [Info]   distributed_index (distributed)...
+ 2026-05-26 16:49:13 [Info]   rt_with_columnar (rt) [929B]...
- %{DATETIME} [Info]   SKIP
+ 2026-05-26 16:49:13 [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:13 [Info]   rt_without_columnar (rt) [907B]...
%{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:13 [Info] Running sync
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:13 [Info]  OK
- %{DATETIME} [Info]   test (rt) [#!/[0-9\.]+K/!#]...
+ 2026-05-26 16:49:13 [Info] You can find backup here: /tmp/backup-20260526164913
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:13 [Info] Elapsed time: 0.01s
- %{DATETIME} [Info] Running sync
+ 2026-05-26 16:49:14 [Info] Done
- %{DATETIME} [Info]  OK
+ 2026-05-26 16:49:14 [Info]
- %{DATETIME} [Info] You can find backup here: /tmp/backup-#!/[0-9]{14}/!#
+ Unfreezing all tables...
- %{DATETIME} [Info] Elapsed time: #!/[0-9]+/!#.#!/[0-9]+/!#s
+ 2026-05-26 16:49:14 [Info]   rt_with_columnar...
- %{DATETIME} [Info] Done
+ 2026-05-26 16:49:14 [Info]    OK
- %{DATETIME} [Info]
+ 2026-05-26 16:49:14 [Info]   rt_without_columnar...
- Unfreezing all tables...
+ 2026-05-26 16:49:14 [Info]    OK
- %{DATETIME} [Info]   distributed_index...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   test...
- %{DATETIME} [Info]    OK
––– input –––
ls -lh /tmp/backup-* | wc -l
––– output –––
OK
––– input –––
ls -1v /tmp/backup-*/data/test/
––– output –––
- test.meta
+ ls: cannot access '/tmp/backup-*/data/test/': No such file or directory
- test.settings
- test.0.spa
- test.0.spb
- test.0.spd
- test.0.spds
- test.0.spe
- test.0.sph
- test.0.sphi
- test.0.spi
- test.0.spidx
- test.0.spknn
- test.0.spm
- test.0.spp
- test.0.spt

test/clt-tests/backup-restore-ok.rec

––– input –––
cp ${MANTICORESEARCH_CONFIG:-./test/clt-tests/config/manticore.conf} /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
stdbuf -oL searchd
––– output: contains –––
listening on
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 < test/clt-tests/dumps/tables.sql
––– output –––
+ ERROR 1064 (42000) at line 23: error adding table 'test': knn library not loaded
––– input –––
touch /usr/share/manticore/{abc,zed}.pak
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e 'show tables'
––– output –––
- +---------------------+-------------+
+ +---------------------+------+
- | Table               | Type        |
+ | Table               | Type |
- +---------------------+-------------+
+ +---------------------+------+
- | distributed_index   | distributed |
+ | rt_with_columnar    | rt   |
- | rt_with_columnar    | rt          |
+ | rt_without_columnar | rt   |
- | rt_without_columnar | rt          |
+ +---------------------+------+
- | test                | rt          |
- +---------------------+-------------+
––– input –––
manticore-backup --backup-dir=/tmp
––– output –––
Copyright (c) %{YEAR}-%{YEAR}, Manticore Software LTD (https://manticoresearch.com)
Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: all tables
Backup dir: /tmp
Manticore config
  endpoint =  http://%{IPADDR}:9312
Manticore versions:
  manticore: %{VERSION}
  columnar: %{VERSION}
  secondary: %{VERSION}
  knn: %{VERSION}
  buddy: %{VERSION}
%{DATETIME} [Info] Starting the backup...
%{DATETIME} [Info] Backing up config files...
%{DATETIME} [Info]   config files - OK
%{DATETIME} [Info] Backing up global state files...
%{DATETIME} [Info]   global state files – OK
%{DATETIME} [Info] Backing up tables...
- %{DATETIME} [Info]   distributed_index (distributed)...
+ 2026-05-26 16:49:20 [Info]   rt_with_columnar (rt) [929B]...
- %{DATETIME} [Info]   SKIP
+ 2026-05-26 16:49:20 [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:20 [Info]   rt_without_columnar (rt) [907B]...
%{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:20 [Info] Running sync
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:20 [Info]  OK
- %{DATETIME} [Info]   test (rt) [%{NUMBER}B]...
+ 2026-05-26 16:49:20 [Info] You can find backup here: /tmp/backup-20260526164920
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:20 [Info] Elapsed time: 0.01s
- %{DATETIME} [Info] Running sync
+ 2026-05-26 16:49:21 [Info] Done
- %{DATETIME} [Info]  OK
+ 2026-05-26 16:49:21 [Info]
- %{DATETIME} [Info] You can find backup here: /tmp/backup-%{NUMBER}
+ Unfreezing all tables...
- %{DATETIME} [Info] Elapsed time: #!/([0-9]+\.[0-9]+|0)/!#s
+ 2026-05-26 16:49:21 [Info]   rt_with_columnar...
- %{DATETIME} [Info] Done
+ 2026-05-26 16:49:21 [Info]    OK
- %{DATETIME} [Info]
+ 2026-05-26 16:49:21 [Info]   rt_without_columnar...
- Unfreezing all tables...
+ 2026-05-26 16:49:21 [Info]    OK
- %{DATETIME} [Info]   distributed_index...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   test...
- %{DATETIME} [Info]    OK
––– input –––
mv /tmp/backup-* /tmp/backup-111
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp --restore
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp --restore=backup-111
––– output –––
OK
––– input –––
pgrep -f searchd | xargs kill -9
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp --restore=backup-111
––– output –––
OK
––– input –––
rm /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp --restore=backup-111
––– output –––
OK
––– input –––
rm -fr /var/lib/manticore && mkdir -p /var/lib/manticore
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp --restore=backup-111
––– output –––
OK
––– input –––
cp ${MANTICORESEARCH_CONFIG:-./test/clt-tests/config/manticore.conf} /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
stdbuf -oL searchd
––– output: contains –––
listening on
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e 'show tables'
––– output –––
- +---------------------+-------------+
+ +---------------------+------+
- | Table               | Type        |
+ | Table               | Type |
- +---------------------+-------------+
+ +---------------------+------+
- | distributed_index   | distributed |
+ | rt_with_columnar    | rt   |
- | rt_with_columnar    | rt          |
+ | rt_without_columnar | rt   |
- | rt_without_columnar | rt          |
+ +---------------------+------+
- | test                | rt          |
- +---------------------+-------------+
––– input –––
ls /usr/share/manticore/{abc,zed}.pak
––– output –––
OK

test/clt-tests/backup-fails.rec

––– input –––
cp ${MANTICORESEARCH_CONFIG:-./test/clt-tests/config/manticore.conf} /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
stdbuf -oL searchd
––– output: contains –––
listening on
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 < test/clt-tests/dumps/tables.sql
––– output –––
+ ERROR 1064 (42000) at line 23: error adding table 'test': knn library not loaded
––– input –––
mysql -h0 -P9306 -e 'show tables'
––– output –––
- +---------------------+-------------+
+ +---------------------+------+
- | Table               | Type        |
+ | Table               | Type |
- +---------------------+-------------+
+ +---------------------+------+
- | distributed_index   | distributed |
+ | rt_with_columnar    | rt   |
- | rt_with_columnar    | rt          |
+ | rt_without_columnar | rt   |
- | rt_without_columnar | rt          |
+ +---------------------+------+
- | test                | rt          |
- +---------------------+-------------+
––– input –––
mysql -h0 -P9306 -e 'show create table distributed_index'
––– output –––
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+ ERROR 1146 (42S02) at line 1: no such table 'distributed_index'
- | Table             | Create Table                                                                                                                                                              |
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
- | distributed_index | CREATE TABLE distributed_index type='distributed' local='rt_with_columnar' local='rt_without_columnar' agent='127.0.0.1:9312:plain_with_columnar, plain_without_columnar' |
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
––– input –––
mysql -h0 -P9306 -e 'show create table rt_with_columnar'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e 'show create table rt_without_columnar'
––– output –––
OK
––– input –––
ls -lt /tmp/non-exist
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp/non-exist
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp --tables=none
––– output –––
OK
––– input –––
pgrep -f searchd | xargs kill
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp
––– output –––
OK

test/clt-tests/backup-single-table-ok.rec

––– input –––
cp ${MANTICORESEARCH_CONFIG:-./test/clt-tests/config/manticore.conf} /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
stdbuf -oL searchd
––– output: contains –––
listening on
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 < test/clt-tests/dumps/tables.sql
––– output –––
+ ERROR 1064 (42000) at line 23: error adding table 'test': knn library not loaded
––– input –––
mysql -h0 -P9306 -e 'show tables'
––– output –––
- +---------------------+-------------+
+ +---------------------+------+
- | Table               | Type        |
+ | Table               | Type |
- +---------------------+-------------+
+ +---------------------+------+
- | distributed_index   | distributed |
+ | rt_with_columnar    | rt   |
- | rt_with_columnar    | rt          |
+ | rt_without_columnar | rt   |
- | rt_without_columnar | rt          |
+ +---------------------+------+
- | test                | rt          |
- +---------------------+-------------+
––– input –––
mysql -h0 -P9306 -e 'show create table distributed_index'
––– output –––
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+ ERROR 1146 (42S02) at line 1: no such table 'distributed_index'
- | Table             | Create Table                                                                                                                                                              |
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
- | distributed_index | CREATE TABLE distributed_index type='distributed' local='rt_with_columnar' local='rt_without_columnar' agent='127.0.0.1:9312:plain_with_columnar, plain_without_columnar' |
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
––– input –––
mysql -h0 -P9306 -e 'show create table rt_with_columnar'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e 'show create table rt_without_columnar'
––– output –––
OK
––– input –––
ls -lt /tmp/backup-*
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp --tables=rt_without_columnar
––– output –––
Copyright (c) %{YEAR}-%{YEAR}, Manticore Software LTD (https://manticoresearch.com)
Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: rt_without_columnar
Backup dir: /tmp
Manticore config
  endpoint =  http://%{IPADDR}:9312
Manticore versions:
  manticore: %{VERSION}
  columnar: %{VERSION}
  secondary: %{VERSION}
  knn: %{VERSION}
  buddy: %{VERSION}
%{DATETIME} [Info] Starting the backup...
%{DATETIME} [Info] Backing up config files...
%{DATETIME} [Info]   config files - OK
%{DATETIME} [Info] Backing up tables...
- %{DATETIME} [Info]   rt_without_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:26 [Info]   rt_without_columnar (rt) [907B]...
%{DATETIME} [Info]    OK
%{DATETIME} [Info] Running sync
%{DATETIME} [Info]  OK
%{DATETIME} [Info] You can find backup here: /tmp/backup-#!/[0-9]{14}/!#
%{DATETIME} [Info] Elapsed time: #!/[0-9]+\.[0-9]+|0/!#s
%{DATETIME} [Info] Done
%{DATETIME} [Info]
Unfreezing all tables...
- %{DATETIME} [Info]   distributed_index...
+ 2026-05-26 16:49:26 [Info]   rt_with_columnar...
%{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar...
+ 2026-05-26 16:49:26 [Info]   rt_without_columnar...
%{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   test...
- %{DATETIME} [Info]    OK
––– input –––
ls -lh /tmp/backup-* | wc -l
––– output –––
OK
––– input –––
ls -lh /tmp/backup-*/data/*
––– output –––
total 12K
- %{FILEPERMS} 1 root root  %{NUMBER} #!/[a-zA-Z]{3}/!# #!/[0-9]{1,2}/!# #!/[0-9]{2}:[0-9]{2}/!# rt_without_columnar.meta
+ -rw------- 1 root root 884 May 26 16:49 rt_without_columnar.meta
- %{FILEPERMS} 1 root root #!/[0-9]{1}\.[0-9]{1}/!#K #!/[a-zA-Z]{3}/!# #!/[0-9]{1,2}/!# #!/[0-9]{2}:[0-9]{2}/!# rt_without_columnar.ram
+ -rw------- 1 root root  20 May 26 16:49 rt_without_columnar.ram
- %{FILEPERMS} 1 root root    %{NUMBER} #!/[a-zA-Z]{3}/!# #!/[0-9]{1,2}/!# #!/[0-9]{2}:[0-9]{2}/!# rt_without_columnar.settings
+ -rw------- 1 root root   3 May 26 16:49 rt_without_columnar.settings

test/clt-tests/backup-shebang-config-ok.rec

––– input –––
cp test/clt-tests/config/manticore.sh /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
cp ${MANTICORESEARCH_CONFIG:-./test/clt-tests/config/manticore.conf} /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
stdbuf -oL searchd
––– output: contains –––
listening on
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 < test/clt-tests/dumps/tables.sql
––– output –––
+ ERROR 1064 (42000) at line 23: error adding table 'test': knn library not loaded
––– input –––
ls -lt /tmp/backup-*
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp
––– output –––
Copyright (c) %{YEAR}-%{YEAR}, Manticore Software LTD (https://manticoresearch.com)
Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: all tables
Backup dir: /tmp
Manticore config
  endpoint =  http://%{IPADDR}:9312
Manticore versions:
  manticore: %{VERSION}
  columnar: %{VERSION}
  secondary: %{VERSION}
  knn: %{VERSION}
  buddy: %{VERSION}
%{DATETIME} [Info] Starting the backup...
%{DATETIME} [Info] Backing up config files...
%{DATETIME} [Info]   config files - OK
%{DATETIME} [Info] Backing up global state files...
%{DATETIME} [Info]   global state files – OK
%{DATETIME} [Info] Backing up tables...
- %{DATETIME} [Info]   distributed_index (distributed)...
+ 2026-05-26 16:49:24 [Info]   rt_with_columnar (rt) [929B]...
- %{DATETIME} [Info]   SKIP
+ 2026-05-26 16:49:24 [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:24 [Info]   rt_without_columnar (rt) [907B]...
%{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:24 [Info] Running sync
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:24 [Info]  OK
- %{DATETIME} [Info]   test (rt) [%{NUMBER}B]...
+ 2026-05-26 16:49:24 [Info] You can find backup here: /tmp/backup-20260526164924
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:24 [Info] Elapsed time: 0.01s
- %{DATETIME} [Info] Running sync
+ 2026-05-26 16:49:24 [Info] Done
- %{DATETIME} [Info]  OK
+ 2026-05-26 16:49:24 [Info]
- %{DATETIME} [Info] You can find backup here: /tmp/backup-%{NUMBER}
+ Unfreezing all tables...
- %{DATETIME} [Info] Elapsed time: #!/([0-9]+\.[0-9]+|0)/!#s
+ 2026-05-26 16:49:24 [Info]   rt_with_columnar...
- %{DATETIME} [Info] Done
+ 2026-05-26 16:49:24 [Info]    OK
- %{DATETIME} [Info]
+ 2026-05-26 16:49:24 [Info]   rt_without_columnar...
- Unfreezing all tables...
+ 2026-05-26 16:49:24 [Info]    OK
- %{DATETIME} [Info]   distributed_index...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   test...
- %{DATETIME} [Info]    OK
––– input –––
ls -lh /tmp/backup-* | wc -l
––– output –––
OK

test/clt-tests/backup-works-with-max-connections.rec

––– input –––
export MANTICORESEARCH_CONFIG="./test/clt-tests/config/manticore-vip.conf"
––– output –––
OK
––– input –––
cp ${MANTICORESEARCH_CONFIG:-./test/clt-tests/config/manticore.conf} /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
stdbuf -oL searchd
––– output: contains –––
listening on
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 < test/clt-tests/dumps/tables.sql
––– output –––
+ ERROR 1064 (42000) at line 23: error adding table 'test': knn library not loaded
––– input –––
curl -s localhost:9312/sql?mode=raw -d 'query=debug sleep 10' &
––– output –––
OK
––– input –––
mysql -h0 -P9307 -e 'show tables'
––– output –––
- +---------------------+-------------+
+ +---------------------+------+
- | Table               | Type        |
+ | Table               | Type |
- +---------------------+-------------+
+ +---------------------+------+
- | distributed_index   | distributed |
+ | rt_with_columnar    | rt   |
- | rt_with_columnar    | rt          |
+ | rt_without_columnar | rt   |
- | rt_without_columnar | rt          |
+ +---------------------+------+
- | test                | rt          |
- +---------------------+-------------+
––– input –––
mysql -h0 -P9307 -e 'show create table distributed_index'
––– output –––
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+ ERROR 1146 (42S02) at line 1: no such table 'distributed_index'
- | Table             | Create Table                                                                                                                                                              |
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
- | distributed_index | CREATE TABLE distributed_index type='distributed' local='rt_with_columnar' local='rt_without_columnar' agent='127.0.0.1:9312:plain_with_columnar, plain_without_columnar' |
- +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
––– input –––
mysql -h0 -P9307 -e 'show create table rt_with_columnar'
––– output –––
OK
––– input –––
mysql -h0 -P9307 -e 'show create table rt_without_columnar'
––– output –––
OK
––– input –––
ls -lt /tmp/backup-*
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp
––– output –––
Copyright (c) %{YEAR}-%{YEAR}, Manticore Software LTD (https://manticoresearch.com)
Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: all tables
Backup dir: /tmp
Manticore config
  endpoint =  http://%{IPADDR}:9313
Manticore versions:
  manticore: %{VERSION}
  columnar: %{VERSION}
  secondary: %{VERSION}
  knn: %{VERSION}
  buddy: %{VERSION}
%{DATETIME} [Info] Starting the backup...
%{DATETIME} [Info] Backing up config files...
%{DATETIME} [Info]   config files - OK
%{DATETIME} [Info] Backing up global state files...
%{DATETIME} [Info]   global state files – OK
%{DATETIME} [Info] Backing up tables...
- %{DATETIME} [Info]   distributed_index (distributed)...
+ 2026-05-26 16:49:28 [Info]   rt_with_columnar (rt) [929B]...
- %{DATETIME} [Info]   SKIP
+ 2026-05-26 16:49:28 [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:28 [Info]   rt_without_columnar (rt) [907B]...
%{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:28 [Info] Running sync
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:28 [Info]  OK
- %{DATETIME} [Info]   test (rt) [%{NUMBER}B]...
+ 2026-05-26 16:49:28 [Info] You can find backup here: /tmp/backup-20260526164928
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:28 [Info] Elapsed time: 0.01s
- %{DATETIME} [Info] Running sync
+ 2026-05-26 16:49:29 [Info] Done
- %{DATETIME} [Info]  OK
- %{DATETIME} [Info] You can find backup here: /tmp/backup-%{NUMBER}
- %{DATETIME} [Info] Elapsed time: #!/([0-9]+\.[0-9]+|0)/!#s
- %{DATETIME} [Info] Done
––– input –––
ls -lh /tmp/backup-* | wc -l
––– output –––
OK

test/clt-tests/backup-conf.rec

––– input –––
cp /etc/manticoresearch/manticore.conf /tmp/
––– output –––
OK
––– input –––
cp ./test/clt-tests/config/manticore.conf /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
cp ${MANTICORESEARCH_CONFIG:-./test/clt-tests/config/manticore.conf} /etc/manticoresearch/manticore.conf
––– output –––
OK
––– input –––
stdbuf -oL searchd
––– output: contains –––
listening on
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 < test/clt-tests/dumps/tables.sql
––– output –––
+ ERROR 1064 (42000) at line 23: error adding table 'test': knn library not loaded
––– input –––
manticore-backup --version
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e 'show tables'
––– output –––
- +---------------------+-------------+
+ +---------------------+------+
- | Table               | Type        |
+ | Table               | Type |
- +---------------------+-------------+
+ +---------------------+------+
- | distributed_index   | distributed |
+ | rt_with_columnar    | rt   |
- | rt_with_columnar    | rt          |
+ | rt_without_columnar | rt   |
- | rt_without_columnar | rt          |
+ +---------------------+------+
- | test                | rt          |
- +---------------------+-------------+
––– input –––
manticore-backup --config=/etc/manticoresearch/manticore.conf --config=/tmp/manticore.conf --backup-dir=/tmp/
––– output –––
Copyright (c) %{YEAR}-%{YEAR}, Manticore Software LTD (https://manticoresearch.com)
Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: all tables
Backup dir: /tmp/
Manticore config
  endpoint =  http://%{IPADDR}:9312
Manticore config
  endpoint =  http://%{IPADDR}:9312
Manticore versions:
  manticore: %{VERSION}
  columnar: %{VERSION}
  secondary: %{VERSION}
  knn: %{VERSION}
  buddy: %{VERSION}
%{DATETIME} [Info] Starting the backup...
%{DATETIME} [Info] Backing up config files...
%{DATETIME} [Info]   config files - OK
%{DATETIME} [Info] Backing up global state files...
%{DATETIME} [Info]   global state files – OK
%{DATETIME} [Info] Backing up tables...
- %{DATETIME} [Info]   distributed_index (distributed)...
+ 2026-05-26 16:49:15 [Info]   rt_with_columnar (rt) [929B]...
- %{DATETIME} [Info]   SKIP
+ 2026-05-26 16:49:15 [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:15 [Info]   rt_without_columnar (rt) [907B]...
%{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar (rt) [#!/[0-9]{1}\.[0-9]{3}/!#K]...
+ 2026-05-26 16:49:15 [Info] Running sync
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:15 [Info]  OK
- %{DATETIME} [Info]   test (rt) [%{NUMBER}B]...
+ 2026-05-26 16:49:15 [Info] You can find backup here: /tmp/backup-20260526164915
- %{DATETIME} [Info]    OK
+ 2026-05-26 16:49:15 [Info] Elapsed time: 0.01s
- %{DATETIME} [Info] Running sync
+ 2026-05-26 16:49:16 [Info] Done
- %{DATETIME} [Info]  OK
+ 2026-05-26 16:49:16 [Info]
- %{DATETIME} [Info] You can find backup here: /tmp/backup-%{NUMBER}
+ Unfreezing all tables...
- %{DATETIME} [Info] Elapsed time: #!/([0-9]+\.[0-9]+|0)/!#s
+ 2026-05-26 16:49:16 [Info]   rt_with_columnar...
- %{DATETIME} [Info] Done
+ 2026-05-26 16:49:16 [Info]    OK
- %{DATETIME} [Info]
+ 2026-05-26 16:49:16 [Info]   rt_without_columnar...
- Unfreezing all tables...
+ 2026-05-26 16:49:16 [Info]    OK
- %{DATETIME} [Info]   distributed_index...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_with_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   rt_without_columnar...
- %{DATETIME} [Info]    OK
- %{DATETIME} [Info]   test...
- %{DATETIME} [Info]    OK
––– input –––
rm -rf /etc/manticoresearch/manticore.conf /tmp/manticore.conf /var/lib/manticore; mkdir /var/lib/manticore;
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp/ --restore
––– output –––
OK
––– input –––
ls /tmp/
––– output –––
OK
––– input –––
export BACKUP_DATE=$(ls /tmp/)
––– output –––
OK
––– input –––
manticore-backup --backup-dir=/tmp/ --restore=$BACKUP_DATE
––– output –––
OK
––– input –––
ls /tmp/manticore.conf
––– output –––
OK
––– input –––
ls /etc/manticoresearch/manticore.conf
––– output –––
OK

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(backup): auto-include local tables of distributed tables#136

feat(backup): auto-include local tables of distributed tables#136
donhardman wants to merge 1 commit into
mainfrom
fix/distributed-table-backup

donhardman commented May 26, 2026

Uh oh!

github-actions Bot commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

donhardman commented May 26, 2026

Uh oh!

github-actions Bot commented May 26, 2026

clt

Failed tests:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant