Minic for ESP32 and other microcontrollers

[vovagorodok]

There is any possibility to use Minic engine for non Windows/Linux/Mac operation systems like FreeRTOS?
Cheap hardware that I'm trying to use ESP32-WROOM-32D DevKitc: 2x240MHz CPU, 320KB RAM, 4MB Flash
320KB RAM enough for that?
Software possibilities that I have here: gcc with std=c++2a and almost full STL with dynamic allocation and threads (Newlib)
As I correctly understand engine works like separate process. Can it be used as library instead?

Thanks!

[tryingsomestuff]

Hi, thanks for your interest in Minic,
Based on (almost) full stl support and a recent g++ this might be feasible.

UCI engine are more meant to communicate with pipes (stdin, stdout) than being libraries but it would be very easy to handle calls like a library would indeed.

Wanna play with a NOKIA 5110 ?

[vovagorodok]

Wanna play with a NOKIA 5110 ?

Almost :p I'm trying to find brain to some cheap electronic chess board, prototype is in progress. I can record movie at the and.

As I understand all is configured like that

// *** options
#define WITH_UCI    // include or not UCI protocol support
// #define WITH_XBOARD // include or not XBOARB protocol support
// #define WITH_MAGIC  // use magic bitboard or HB ones
// #define WITH_NNUE   // include or not NNUE support
// #define WITH_STATS  // produce or not search statitics
//#define WITH_MPI    // support "distributed" version or not
#ifndef _MSC_VER
#define WITH_SYZYGY // include or not syzgy ETG support
#endif

#define WITHOUT_FILESYSTEM              // some compiler don't support whole std:filesystem

First compilation error is:

lib/Minic/src/definition.hpp:643:21: error: 'aligned_alloc' is not a member of 'std'
    void* ret = std::aligned_alloc(alignment, size);

adding:

#elif defined(INC_FREERTOS_H)
  return malloc(size);

fixes that. Next issue:

lib/Minic/src/attack.hpp:59:161: error: 'empty' was not declared in this scope
 template < Piece > [[nodiscard]] inline BitBoard coverage      (const Square x, const BitBoard occupancy = 0, const Color c = Co_White) { assert(false); return empty; }

A correctly I understand we should prepare another set of flags inside definition.hpp and add #ifdef INC_FREERTOS_H at OS dependent places?

[tryingsomestuff]

Yes your on the right track indeed.

the "empty" missing is a mistake on my side, as I not often compile the non- magic bitboard attack code. It should be "emptyBitBoard" instead.

-template < Piece > [[nodiscard]] inline BitBoard coverage      (const Square x, const BitBoard occupancy = 0, const Color c = Co_White) { assert(false); return empty; }
+template < Piece > [[nodiscard]] inline BitBoard coverage      (const Square x, const BitBoard occupancy = 0, const Color c = Co_White) { assert(false); return emptyBitBoard; }

[vovagorodok]

Thanks, fixed!
Looks that it can be last compilation issue:

Dependency Graph
|-- <NimBLE-Arduino> 1.3.8
|-- <StepperDriver> 1.3.1
|-- <Minic>
Building in release mode
Compiling .pio/build/esp32dev/lib71f/Minic/attack.cpp.o
Compiling .pio/build/esp32dev/lib71f/Minic/definition.cpp.o
Compiling .pio/build/esp32dev/lib71f/Minic/distributed.cpp.o
Compiling .pio/build/esp32dev/lib71f/Minic/dynamicConfig.cpp.o
In file included from lib/Minic/src/bitboard.hpp:3,
                 from lib/Minic/src/bitboardTools.hpp:3,
                 from lib/Minic/src/attack.hpp:3,
                 from lib/Minic/src/attack.cpp:1:
lib/Minic/src/attack.cpp: In function 'BitBoard BBTools::rankAttack(BitBoard, Square)':
lib/Minic/src/attack.cpp:139:25: error: 'x' was not declared in this scope
    const int f = SQFILE(x);
                         ^
lib/Minic/src/definition.hpp:187:51: note: in definition of macro 'SQFILE'
 #define SQFILE(s)              static_cast<File>((s)&7)
                                                   ^
*** [.pio/build/esp32dev/lib71f/Minic/attack.cpp.o] Error 1

As I understand x is globał and we should include that

[tryingsomestuff]

Very sorry again, I don't compile HQBB code very often and some refactoring were thus not well wrote.

Here it's simply SQFILE(s) (in BitBoard rankAttack(const BitBoard occupancy, const Square s) )

BTW, why are you not using magic bitboard ?

[vovagorodok]

Unfortunately :):

Dependency Graph
|-- <Minic>
Building in release mode
Compiling .pio/build/esp32dev/libc2b/Minic/egt.cpp.o
Compiling .pio/build/esp32dev/libc2b/Minic/extendedPosition.cpp.o
Compiling .pio/build/esp32dev/libc2b/Minic/material.cpp.o
Compiling .pio/build/esp32dev/libc2b/Minic/moveGen.cpp.o
lib/Minic/Source/egt.cpp:11:10: fatal error: tbprobe.h: No such file or directory

*****************************************************************
* Looking for tbprobe.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:tbprobe.h"
* Web  > https://registry.platformio.org/search?q=header:tbprobe.h
*
*****************************************************************

 #include "tbprobe.h"
          ^~~~~~~~~~~
compilation terminated.
*** [.pio/build/esp32dev/libc2b/Minic/egt.cpp.o] Error 1
lib/Minic/Source/material.cpp: In function 'ScoreType MaterialHash::helperKPK(const Position&, Color, ScoreType, DepthType)':
lib/Minic/Source/material.cpp:191:11: error: 'DynamicConfig' has not been declared
       if (DynamicConfig::armageddon) {
           ^~~~~~~~~~~~~
*** [.pio/build/esp32dev/libc2b/Minic/material.cpp.o] Error 1
lib/Minic/Source/extendedPosition.cpp: In lambda function:
lib/Minic/Source/extendedPosition.cpp:257:13: error: 'NNUEEvaluator' was not declared in this scope
             NNUEEvaluator    evaluator;
             ^~~~~~~~~~~~~
lib/Minic/Source/extendedPosition.cpp:258:18: error: 'struct ExtendedPosition' has no member named 'associateEvaluator'
             extP.associateEvaluator(evaluator);
                  ^~~~~~~~~~~~~~~~~~
lib/Minic/Source/extendedPosition.cpp:258:37: error: 'evaluator' was not declared in this scope
             extP.associateEvaluator(evaluator);
                                     ^~~~~~~~~
lib/Minic/Source/extendedPosition.cpp:258:37: note: suggested alternative: 'evalDanger'
             extP.associateEvaluator(evaluator);
                                     ^~~~~~~~~
                                     evalDanger
lib/Minic/Source/extendedPosition.cpp:259:18: error: 'struct ExtendedPosition' has no member named 'resetNNUEEvaluator'
             extP.resetNNUEEvaluator(extP.evaluator());
                  ^~~~~~~~~~~~~~~~~~
lib/Minic/Source/extendedPosition.cpp:259:42: error: 'struct ExtendedPosition' has no member named 'evaluator'
             extP.resetNNUEEvaluator(extP.evaluator());
                                          ^~~~~~~~~
*** [.pio/build/esp32dev/libc2b/Minic/extendedPosition.cpp.o] Error 1

How configure definition.hpp in order to have minimal version without any dependencies?

Here are draft changes that are done until now:
vovagorodok@fff591d
And here is esp32 project where Minic is submodule in lib folder:
https://github.com/vovagorodok/minic_esp32

[tryingsomestuff]

First, about tbprobe, you'll have to get submodules (git submodule update --init) in order to have fathom, or you can disable (comment) #define WITH_SYZYGY

Then, for some reason I don't get yet, #include "dynamicConfig.hpp" may be missing in egt.cpp, I really wonder how it compiles everyday, but anyway.

And finally, a use of NNUEEvaluator around line 257 in extendedPosition.cpp is not protected by a #ifdef. Should reads like this

#ifdef WITH_NNUE            
            NNUEEvaluator    evaluator;
            extP.associateEvaluator(evaluator);
            extP.resetNNUEEvaluator(extP.evaluator());
#endif

I'll push those little fixes on master tomorrow probably.

[vovagorodok]

Nice!
Additional fixes was added. List of all chenges:
master...vovagorodok:master

Compilation success! Now I have linking issues:

|-- <Minic>
Building in release mode
Linking .pio/build/esp32dev/firmware.elf
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/libc2b/libMinic.a(minic.cpp.o): in function `init(int, char**)':
/home/vova/PlatformIO/Projects/minic_esp32/lib/Minic/Source/minic.cpp:30: multiple definition of `init(int, char**)'; .pio/build/esp32dev/src/main.cpp.o:/home/vova/PlatformIO/Projects/minic_esp32/src/main.cpp:32: first defined here
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/libc2b/libMinic.a(minic.cpp.o): in function `finalize()':
/home/vova/PlatformIO/Projects/minic_esp32/lib/Minic/Source/minic.cpp:55: multiple definition of `finalize()'; .pio/build/esp32dev/src/main.cpp.o:/home/vova/PlatformIO/Projects/minic_esp32/src/main.cpp:57: first defined here
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/firmware.elf section `.dram0.bss' will not fit in region `dram0_0_seg'
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: DRAM segment data does not fit.
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: DRAM segment data does not fit.
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: section .iram0.vectors VMA [0000000040080000,0000000040080402] overlaps section .dram0.bss VMA [000000003ffc1780,0000000041b4c537]
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: region `dram0_0_seg' overflowed by 28771128 bytes
collect2: error: ld returned 1 exit status
*** [.pio/build/esp32dev/firmware.elf] Error 1

Looks that compiled code take a lot of static memory (overflowed by 28771128 bytes). I have ~1.2MB flash and ~320KB RAM. ~29MB instead 320KB, we almost there :) What can I additionally disable/optiomize/(cat arrays size)/(recursions depth) in order to achieve using less than 320K RAM?

[tryingsomestuff]

Ok so the link issues are likely related to the fact you are trying to build Minic as a lib (libMinic.a), maybe we'll work on that later.

Using only 320K of RAM is a real challenge. By default Minic is using

Size of TT 96Mb (dynamic allocation)
Size of Pawn TT 10752Kb (dynamic allocation)
Material hash size : 27Mb (static allocation)
KPK table size : 24Kb (static allocation)

Minic can totally disable transposition table and pawn table (I just need to expose this to the user).
Let's say KPK table is small enough. But disabling the 27Mb Material hash table is not that easy ... I need a few hours to think of that.

Is 320K, the total RAM or the size available for static allocation ?

[vovagorodok]

Wow! great analysis. And have the same numbers as in linking error.
If it will succeed, we will have first strong chess engine/library ported to such tinny arduino devices.

I'm using this developing board. It has 512KB RAM and by default 320K allocated for static. System allocates around 30KB with Bluetooth/Wifi stack.

[tryingsomestuff]

ok so with 512K RAM, I really need to make material table optionnal. I'll work on that this week end.

[tryingsomestuff]

ok got it to work without the material table. The risk is that it'd be unable to play simple KRK or KBNK end-game so i'll also find a way to patch for those two ...

[vovagorodok]

Perfecto. I'll align to master then and will see whan happens ..

[tryingsomestuff]

ok I pushed to master, I think most of your discoveries were fixed in those various commits.
You only have to tune definition.hpp to fit your need, and we'll have to discuss how to use Minic in as library in the next days I guess. Feel free to reach on discord (SF, TCEC or openbench server for instance).

[vovagorodok]

Aligned!

Linking .pio/build/esp32dev/firmware.elf
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/firmware.elf section `.dram0.bss' will not fit in region `dram0_0_seg'
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: DRAM segment data does not fit.
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: DRAM segment data does not fit.
/home/vova/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: region `dram0_0_seg' overflowed by 115104 bytes
collect2: error: ld returned 1 exit status
*** [.pio/build/esp32dev/firmware.elf] Error 1

Wow, overhead just 115KB it means that Minic with OS now use only ~435KB of static memory
Current diff:
master...vovagorodok:master
For this moment I just rename minic.cpp to minic.hpp in order to not compile it
main.cpp now is 99% copy-pasta of minic.cpp

I think now if we use it as library instead whole project we will achieve success.
Currently I'm not discord user, I'll register there