[BUG]: segfault at exit since upgrading to 3.0.x, probably due to "Make wrapped C++ functions pickleable" (https://github.com/pybind/pybind11/pull/5580)

[HMRWork42]

EDIT (2026-03-27): Same root cause as #5976. Partially addressed by #6010, but that PR masks the bug by skipping destruction during finalization (leaking all function records). See #5486 for how the bug was introduced.

---

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

3.0.2

Problem description

I am one of the co-maintainers of skia-python . We have had segfaults at end of CI pytest since 3.0. c1 if built against pybind11 3.x. I I have finally got round to do a debug build and seems to get python's finalizer involving pybind11 code here in this valgrind trace:

==2980== Invalid free() / delete / delete[] / realloc()
==2980==    at 0x4847E43: free (vg_replace_malloc.c:990)
==2980==    by 0x5A0E488: pybind11::cpp_function::destruct(pybind11::detail::function_record*, bool) (pybind11.h:824)
==2980==    by 0x5A0E375: pybind11::detail::function_record_PyTypeObject_methods::tp_dealloc_impl(_object*) (pybind11.h:1325)
==2980==    by 0x49AD620: _Py_Dealloc (object.c:3072)
==2980==    by 0x49C82BC: Py_DECREF (refcount.h:421)
==2980==    by 0x49C82BC: Py_XDECREF (refcount.h:514)
==2980==    by 0x49C82BC: Py_XDECREF (refcount.h:511)
==2980==    by 0x49C82BC: meth_dealloc.lto_priv.0 (methodobject.c:179)
==2980==    by 0x49AD620: _Py_Dealloc (object.c:3072)
==2980==    by 0x4ACB77A: property_dealloc.lto_priv.0 (descrobject.c:1638)
==2980==    by 0x49C1631: _Py_Dealloc (object.c:3072)
==2980==    by 0x49C1631: Py_DECREF (refcount.h:421)
==2980==    by 0x49C1631: Py_XDECREF (refcount.h:514)
==2980==    by 0x49C1631: dictkeys_decref.part.0.constprop.0 (dictobject.c:463)
==2980==    by 0x4A7A739: PyDict_Clear (dictobject.c:2932)
==2980==    by 0x4A7A739: type_clear.lto_priv.0 (typeobject.c:6620)
==2980==    by 0x4A79F56: delete_garbage (gc.c:1141)
==2980==    by 0x4A79F56: gc_collect_region.lto_priv.0 (gc.c:1761)
==2980==    by 0x4AC84C1: gc_collect_full (gc.c:1681)
==2980==    by 0x4AC84C1: _PyGC_Collect (gc.c:2045)
==2980==    by 0x4AEAD2A: _Py_Finalize.constprop.0 (pylifecycle.c:2144)
==2980==  Address 0x6d55c3e is in a r-- mapped file /home/HMRWork42/.local/lib/python3.14/site-packages/skia.cpython-314-x86_64-linux-gnu.so segment

The code around pybind11.h:1325 in 3.0.2 was introduced in "Make wrapped C++ functions pickleable" (#5580) in 3.0 rc1, which seems to match the time frame.

Would like some suggestions to look further.

downstream issue skia-python/skia-python#350

Reproducible example code

Unfortunately skia-python is extremely large and interconnected. I seems to have isolated the segfault at anything, exit (a simple one-line "import skia; .... " segfaults) to src/skia/GrContext* (which is still a thousand lines).

Is this a regression? Put the last known working version here if it is.

2.13.6

[rwgk]

Unfortunately skia-python is extremely large and interconnected. I seems to have isolated the segfault at anything, exit (a simple one-line "import skia; .... " segfaults) to src/skia/GrContext* (which is still a thousand lines).

Could you please try to use codex or claude code or similar to reduce?

This

==2980==    by 0x4AEAD2A: _Py_Finalize.constprop.0 (pylifecycle.c:2144)

makes me think the crash might happen during Python finalization (not sure though). Possibly we need to test _Py_IsFinalizing() to skip the cleanup.

which seems to match the time frame.

That's too vague to be actionable; and even if 5580 triggers the crash, it could just be unmasking a latent bug.

==2980== by 0x4AEAD2A: _Py_Finalize.constprop.0 (pylifecycle.c:2144)

What Python version and build options?

The only way to be sure about the root cause is to reduce. Guessing hardly ever works.

[HMRWork42]

The segfault has been happening on all 3 platforms (windows, mac , linux) and all python versions (CI pytest abort on first failure, so we have at least 3.8, 3.10, 3.13 and now 3.14) with pybind11 3.0+ (See #5694 (comment) ) for 6 months, and blocking newer skia-python releases. ( all the CI failures in the last 6 months in https://github.com/kyamagu/skia-python/pulls ).

@rwgk I am hoping for a tips - say, additional snipplet of code inside pybind11 - to print some info per destruction - so at least I can elimit those which don't segfault. At the moment, I am finding that not calling initGrContext (in src/skia/GrContext*) from skia python's main init, and disabling that particularly set of functionality - plus commenting out a few other things elsewhere to allow the whole thing to low - get around the segfault at exit issue.

[rwgk]

I cannot think of anything better than suggesting again to reduce. At a couple occasions I had good results using codex to do that mostly independently.

Or insert _Py_IsFinalizing() in pybind11::cpp_function::destruct() to see if that helps as a quick-guess-fix.

[HMRWork42]

Okay, I added printf("\targ.name \"%s\" \"%s\" \t \t \t 0x%p 0x%p\n", arg.name, arg.descr, arg.name, arg.descr); just above pybind11.h:824 , and it is giving me a crash at a "self" which differs from the other in the address. So it looks like I am looking at something similar / related to #5976 - there are many arg.name with "self" on the heap, but one is elsewhere.

So I am wondering about the interaction of py::enum_ with py::arithmetic() . The latter involves py::self. However, even with that combinations, it is not obvious why some is more troublesome than others: There are 114 of those in skia-python, and 26 of which has a py::arithmetic() in its definitions. 17 of them are in the src/skia/Gr*.cpp source files, and 12 of those with py::arithmetic(). I suspect it is one or more of the 17 which is/are troublesome, but it is not clear why these 17 and not the other (114 - 17 = 97 elsewhere. Since we are talking about method registrations, I am guessing it probably matters whether these enums are used in method signatures (arguments or return types) or not. So that goes back to the "pickleable" function #5580, and "feat: rework of arg/return type hints to support .noconvert()" #5486 referenced in #5976 .

[HMRWork42]

When I convert a few of those to native_enum, I got exactly one error - arg(): could not convert default argument 'renderable: skgpu::Renderable' in method '<class 'skia.GrRecordingContext'>.defaultBackendFormat' into a Python object (type not registered yet?). So it does look like it is related to arg/return type hints and method signatures.

[rwgk]

Someone will have to systematically reduce (or carefully debug) to get to the bottom of this; guessing seems hopeless in this case. Currently I'm not aware of anyone else stumbling over 5580 in the same way. That code has been in production use at Google for over two years.

Or insert _Py_IsFinalizing() in pybind11::cpp_function::destruct() to see if that helps as a quick-guess-fix.

Did you already try that?

[HinTak]

Someone will have to systematically reduce (or carefully debug) to get to the bottom of this; guessing seems hopeless in this case. Currently I'm not aware of anyone else stumbling over 5580 in the same way. That code has been in production use at Google for over two years.

Or insert _Py_IsFinalizing() in pybind11::cpp_function::destruct() to see if that helps as a quick-guess-fix.

Did you already try that?

Yes - if you mean skipping the free's if true.

[rwgk]

Someone will have to systematically reduce (or carefully debug) to get to the bottom of this; guessing seems hopeless in this case. Currently I'm not aware of anyone else stumbling over 5580 in the same way. That code has been in production use at Google for over two years.

Or insert _Py_IsFinalizing() in pybind11::cpp_function::destruct() to see if that helps as a quick-guess-fix.

Did you already try that?

Yes - if you mean skipping the free's if true.

Yes (see "skip the cleanup" here).

Did it resolve your issue?

[HMRWork42]

Yes, if I put a if (Py_IsFinalizing()) { ...} around the free's, it does not crash. However, ask I said above, only one in just under a thousand of that free() with "self" has the wrong address not on the heap.

[rwgk]

Yes, if I put a if (Py_IsFinalizing()) { ...} around the free's, it does not crash. However, ask I said above, only one in just under a thousand of that free() with "self" has the wrong address not on the heap.

That's plausible during finalization, although it'd be ideal to understand why.

Leaking certainly isn't great, but it's actually very common in the context of Python finalization AFAIK (e.g. note that the "Per iter" numbers here are far from zero even in the best case).

A good simple experiment would be to measure the leaking with and without the Py_IsFinalizing() guard. How much worse is it? Then decide based on the numbers: is it worth your time digging deeper?

If we can show that the numbers are reasonable, I'd support adding the Py_IsFinalizing() guard.

[HMRWork42]

There is another string literal "self" in the same file, for processing py::arg_v - and we do use those in skia-python recently (see #5994 ) , so I am looking to see which of them is causing the double free's:

pybind11/include/pybind11/attr.h

Line 494 in 45fab40

r->args.emplace_back("self", nullptr, handle(), /*convert=*/true, /*none=*/false);

pybind11/include/pybind11/attr.h

Line 515 in 45fab40

"self", /*descr=*/nullptr, /*parent=*/handle(), /*convert=*/true, /*none=*/false);

[HMRWork42]

In skia-python, we have over a hundred enum's, and at least a quarter of those have py::arithematics(). But I have narrowed the #5991 crash to these two enums:

py::enum_<VkFormat>(m, "VkFormat", py::arithmetic())
    .export_values();

py::implicitly_convertible<int, VkFormat>();

py::enum_<VkImageLayout>(m, "VkImageLayout", py::arithmetic())
    .export_values();

py::implicitly_convertible<int, VkImageLayout>();

This code has been in there for years, but the related APIs aren't really used much by users. We were being a bit lazy: there are 300+ values for one, and 40+ for the other, and not all of them useful/commonly used, so we have the implicitly convertible to convert arbitrary new values for those rarely used APIs. If I fill in the 300+ and 40+ values explicitly, remove the py::arithmetic() and py::implicitly_convertible constructs, the segfault goes away.

I think this is a bug in pybind11 - some combination of (a) enum (probably a bare one without a doc string etc), (b) not having any .value() defined (c) relying on py::implicitly_convertible for arbitrary value for it to be useful , (d) having py::arithmetic() (making it a minimal class with some implied methods?) - would go through the literal "self" code path.

I am also a little worried about the other literal "self" on line 515.

Anyway, since having explicit list of symbol enums (rather than relying on implicit conversion) is a generally good direction for skia-python to take, I'll make the change downstream. Hope CI passes and and we can make a release soon; regardless of the status of this.

[rwgk]

Anyway, since having explicit list of symbol enums (rather than relying on implicit conversion) is a generally good direction for skia-python to take, I'll make the change downstream.

Just to be sure you're aware of the alternative: did you consider using py::native_enum?