PEP 835: Address Discourse feedback from encukou and tjreedy

Author: till-varoquaux

peps/pep-0835.rst

@@ -228,46 +228,11 @@ origin:
 Supported Left-Hand Operands
 -----------------------------
 
-The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the
-metatype (``type``) and to several typing-related types. The operator is
-supported for any left-hand operand that currently supports the ``|``
-operator for making a union.
+The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the metatype (``type``) and to several typing-related types. The operator is supported for instances of ``type`` and other typing constructs that currently support the ``|`` operator for unions. The specific handling of ``None`` is currently unsettled (see Open Issues).
 
 For all other left-hand operands, the operator returns ``NotImplemented``,
 allowing normal ``__matmul__`` dispatch to proceed.
 
-Parsing and Grammar
-===================
-
-This proposal requires no changes to the Python grammar. Because ``@`` is
-already a valid operator, it is natively parsed as a binary operation. The
-shorthand is resolved during semantic analysis, entirely bypassing the need
-to patch grammar files or update the parser.
-
-How to Teach This
-=================
-
-In Python, the ``@`` symbol already has an established association with
-metadata through decorators. The annotation shorthand extends this
-intuition to the type system: ``int @ Field(gt=0)`` reads as "``int``,
-decorated with ``Field(gt=0)``."
-
-For beginners, the key rule is simple: **in a type annotation, ``@`` means
-"with this metadata."** The full ``Annotated[int, Field(gt=0)]`` syntax
-remains available and is entirely equivalent for those who find it clearer.
-
-For experienced developers, the precedence rules follow standard Python
-operator precedence (``@`` binds tighter than ``|``), and chaining
-``T @ m1 @ m2`` flattens exactly as nested ``Annotated`` does.
-
-Documentation and teaching materials should introduce the shorthand alongside
-``Annotated``, not as a replacement. The longhand form is still preferred in
-contexts where multiple metadata items are passed as a group and chaining
-would be unwieldy.
-
-Backwards Compatibility
-=======================
-
 Forward References and Deferred Evaluation
 -------------------------------------------
 
@@ -277,8 +242,9 @@ module provides several formats for retrieving annotations:
 - ``Format.VALUE``: Fully evaluates the annotation. Raises ``NameError``
   if any name is unresolvable.
 - ``Format.FORWARDREF``: Wraps unresolvable names in ``ForwardRef`` objects.
-  However, compound expressions using operators (``@``, ``|``) produce an
-  opaque ``ForwardRef`` string containing the entire expression.
+  However, when operators like ``@`` or ``|`` fail to evaluate because of an
+  unresolvable name, this format falls back to returning the entire expression
+  as an opaque string wrapped in a single ``ForwardRef`` object.
 - ``Format.STRING``: Returns the raw source text with no evaluation.
 
 For the ``@`` operator, ``Format.FORWARDREF`` is insufficient. Consider::
@@ -293,8 +259,8 @@ forward reference is resolved. This is a blocking issue for libraries like
 Pydantic and FastAPI, which inspect metadata at class-definition time.
 
 This proposal introduces a new format, ``Format.FORWARDREF_STRUCTURAL``.
-This format assumes typing semantics and evaluates compound type expressions
-**structurally**. It always returns an ``AnnotatedType`` for ``@``, a union
+This format assumes typing semantics and evaluates type expressions involving
+operators (like ``@`` and ``|``) **structurally**. It always returns an ``AnnotatedType`` for ``@``, a union
 for ``|``, and a ``GenericAlias`` for subscripting. When a name cannot be
 resolved, only that name is wrapped in ``ForwardRef``; the surrounding
 operators are still evaluated. The example above produces::
@@ -321,6 +287,38 @@ coarser than from :pep:`749` thunks. When a name is unresolvable, the
 rather than just a name. :pep:`749` provides a strictly better experience and
 is the recommended path forward.
 
+Parsing and Grammar
+===================
+
+This proposal requires no changes to the Python grammar. Because ``@`` is
+already a valid operator, it is natively parsed as a binary operation. The
+shorthand is resolved during semantic analysis, entirely bypassing the need
+to patch grammar files or update the parser.
+
+How to Teach This
+=================
+
+In Python, the ``@`` symbol already has an established association with
+metadata through decorators. The annotation shorthand extends this
+intuition to the type system: ``int @ Field(gt=0)`` reads as "``int``,
+decorated with ``Field(gt=0)``."
+
+For beginners, the key rule is simple: **in a type annotation, ``@`` means
+"with this metadata."** The full ``Annotated[int, Field(gt=0)]`` syntax
+remains available and is entirely equivalent for those who find it clearer.
+
+For experienced developers, the precedence rules follow standard Python
+operator precedence (``@`` binds tighter than ``|``), and chaining
+``T @ m1 @ m2`` flattens exactly as nested ``Annotated`` does.
+
+Documentation and teaching materials should introduce the shorthand alongside
+``Annotated``, not as a replacement. The longhand form is still preferred in
+contexts where multiple metadata items are passed as a group and chaining
+would be unwieldy.
+
+Backwards Compatibility
+=======================
+
 Operator Overloading
 --------------------
 
@@ -354,7 +352,7 @@ metaclass.
 The private ``typing._AnnotatedAlias`` class is retained as a deprecated
 compatibility shim. Code using ``isinstance(x, typing._AnnotatedAlias)``
 will continue to work but emit a ``DeprecationWarning``. The shim is
-scheduled for removal in Python 3.18.
+scheduled for removal in Python 3.21.
 
 Code that should be updated:
 
@@ -481,6 +479,26 @@ References
 - :pep:`727` -- Documentation metadata in typing (Withdrawn)
 - :pep:`749` -- Implementing PEP 649
 
+Open Issues
+===========
+
+Supporting ``None``
+-------------------
+
+While the ``@`` operator naturally applies to instances of ``type``, it is currently unsettled how it should handle ``None``.
+
+There are two primary options, both with notable downsides:
+
+1. **Implement ``__matmul__`` on ``NoneType``:** This provides the best ergonomics, allowing ``None @ Metadata``. However, it introduces a runtime risk. If a user accidentally uses matrix multiplication on a ``None`` value (e.g., ``a @ b`` where ``a`` is ``None`` instead of a numpy array), it will silently return an ``Annotated`` object instead of raising a ``TypeError``.
+2. **Do not support ``None @ Metadata``:** This avoids the runtime risk, but creates an ergonomic cliff. Users would be forced to write ``type(None) @ Metadata`` or fall back to ``typing.Annotated[None, Metadata]``.
+
+The authors of this PEP lean slightly in favor of implementing ``__matmul__`` on ``NoneType`` for its ergonomic benefits, but ultimately defer to the Typing Council and community discussion to decide the best path forward.
+
+Deprecation Timeline
+--------------------
+
+This PEP schedules the removal of the ``typing._AnnotatedAlias`` compatibility shim for Python 3.21, following the standard 5-year deprecation policy (:pep:`387`). However, given the widespread use of ``typing.Annotated``, the exact timeline for removal—or whether the shim should be retained indefinitely to prevent churn in legacy code—remains open for community discussion.
+
 Copyright
 =========