Deadlock with std::function argument

[davidhewitt]

Issue description

I can easily reproduce a deadlock which seems to be related to gil_scoped_acquire and std::function<> arguments, when being called from a thread other than the main thread.

Reproducible example code

In particular, this binding code:

#include <pybind11/pybind11.h>
#include <pybind11/functional.h>

PYBIND11_MODULE(deadlock, m)
{
  m.def("deadlock", [] (std::function<void ()> f) { f(); });
}

Deadlocks with the following Python code:

import deadlock
from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor(max_workers=2) as e:
    e.submit(deadlock.deadlock, lambda: None)
    e.submit(deadlock.deadlock, lambda: None)

Or also with this:

import deadlock
from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor(max_workers=1) as e:
    e.submit(deadlock.deadlock, lambda: None)

with ThreadPoolExecutor(max_workers=1) as e:
    e.submit(deadlock.deadlock, lambda: None)

Something about two calls seems significant, I can't reproduce with just one.

I can spend some time on the weekend writing a patch if you are able to point me to what is probably going wrong :)

Thanks
David

[davidhewitt]

I'm still very concerned about this. Has anyone got a feeling what is going on here? If I attempted a PR, would it be accepted?

[lambdaknight]

You might check out PR #1595. That fixes a deadlock issue I was getting with functions getting called from a non-main thread. Maybe it'll fix yours?

[davidhewitt]

Thanks, I think that is tackling a different problem. I'll post on that PR.

[davidhewitt]

I've just seen #1211 has been merged, and I can confirm that the examples I give above now work correctly without deadlocking on master. Exciting!