Multiprocess and Model API Pipe

[apollo-time]

Chess-Zero/src/chess_zero/worker/self_play.py

Lines 33 to 41 in 90a5aad

	self.cur_pipes = self.m.list([self.current_model.get_pipes(self.config.play.search_threads) for _ in range(self.config.play.max_processes)])
	def start(self):
	self.buffer = []
	futures = deque()
	with ProcessPoolExecutor(max_workers=self.config.play.max_processes) as executor:
	for game_idx in range(self.config.play.max_processes):
	futures.append(executor.submit(self_play_buffer, self.config, cur=self.cur_pipes))

I see you create max_processes pipe list of search_threads pipes and max_processes execute self play processes with the pipe list.
A self player use own pipe list by pop from it.

Chess-Zero/src/chess_zero/worker/self_play.py

Lines 86 to 87 in 90a5aad

	def self_play_buffer(config, cur) -> (ChessEnv, list):
	pipes = cur.pop() # borrow

More ever you keep it going continuously the same pipe list.

Chess-Zero/src/chess_zero/worker/self_play.py

Line 56 in 90a5aad

futures.append(executor.submit(self_play_buffer, self.config, cur=self.cur_pipes)) # Keep it going

Is it ok multiple pop from the same list?
Why create the shared list by calling Manager.list and send the list to player, instead of sending pipe list itself?
like this:

futures.append(executor.submit(self_play_buffer, self.config, cur=self.current_model.get_pipes(self.config.play.search_threads)))
# and
def self_play_buffer(config, cur) -> (ChessEnv, list):
    pipes = cur # without pop

[Akababa]

I hear you; I was trying to do that for the longest time before giving up and using shared manager.

The processpool will only have 3 running at a time, so it's safe to pop once for each (I append it back in the end). If I'm not mistaken calling get_pipes every time will leak pipes.

[apollo-time]

Oh, I see append pipe back in the end.

Chess-Zero/src/chess_zero/worker/self_play.py

Line 118 in 90a5aad

cur.append(pipes)

And I think python is not enough implement search tree using multi processing.
Zeta36#13
Is the good result train using SL or RL?

[Akababa]

Yeah python is slow but since I already maxed out my GPU usage I will just keep using it for now. When we do distributed self play we will most likely need C++ like leela zero though (@benediamond has already started implementing this part)
Supervised, although it may be just luck as I'm having trouble reproducing it. I tried a whole bunch of stuff along the way and sometimes the most unintuitive things just work for no reason.

[apollo-time]

Thanks reply.
I try multi processing search tree select_action_q_and_u part without expanding, is not use GPU and use only CPU.
So if has multiple core CPU, this search can be faster in multiprocessing.
And are you believe the residual model can be train chess moves is not in the training data with SL?
More ever for RL, the residual model can be train chess moves is not passed ever in self playing?

[Akababa]

Nice, I look forward to seeing it!
I'm not sure what your question is. The SL policy is just 1 for the human move and 0 for everything else. And chess has no "passing" allowed.

[apollo-time]

My question is that the residual model can be train chess moves on the status, this board status is not in the training data.

[Akababa]

https://github.com/Akababa/Chess-Zero/wiki#model-diagram
Oh I think I understand now. The board status (except for 3 times repetition) can be inferred from the moves leading up to it. I left out the history planes for simplicity and to combat overfitting.