PEP 835: Address Discourse feedback from encukou and tjreedy

Author: till-varoquaux

peps/pep-0835.rst

@@ -229,45 +229,15 @@ Supported Left-Hand Operands
 -----------------------------
 
 The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the
-metatype (``type``) and to several typing-related types. The operator is
+metatype (``type``) and to several types used in type hinting (including
+``NoneType``). The operator is
 supported for any left-hand operand that currently supports the ``|``
-operator for making a union.
+operator for making a union (including ``type(None)``, which acts as the
+canonical equivalent to ``None`` in type hints).
 
 For all other left-hand operands, the operator returns ``NotImplemented``,
 allowing normal ``__matmul__`` dispatch to proceed.
 
-Parsing and Grammar
-===================
-
-This proposal requires no changes to the Python grammar. Because ``@`` is
-already a valid operator, it is natively parsed as a binary operation. The
-shorthand is resolved during semantic analysis, entirely bypassing the need
-to patch grammar files or update the parser.
-
-How to Teach This
-=================
-
-In Python, the ``@`` symbol already has an established association with
-metadata through decorators. The annotation shorthand extends this
-intuition to the type system: ``int @ Field(gt=0)`` reads as "``int``,
-decorated with ``Field(gt=0)``."
-
-For beginners, the key rule is simple: **in a type annotation, ``@`` means
-"with this metadata."** The full ``Annotated[int, Field(gt=0)]`` syntax
-remains available and is entirely equivalent for those who find it clearer.
-
-For experienced developers, the precedence rules follow standard Python
-operator precedence (``@`` binds tighter than ``|``), and chaining
-``T @ m1 @ m2`` flattens exactly as nested ``Annotated`` does.
-
-Documentation and teaching materials should introduce the shorthand alongside
-``Annotated``, not as a replacement. The longhand form is still preferred in
-contexts where multiple metadata items are passed as a group and chaining
-would be unwieldy.
-
-Backwards Compatibility
-=======================
-
 Forward References and Deferred Evaluation
 -------------------------------------------
 
@@ -277,8 +247,9 @@ module provides several formats for retrieving annotations:
 - ``Format.VALUE``: Fully evaluates the annotation. Raises ``NameError``
   if any name is unresolvable.
 - ``Format.FORWARDREF``: Wraps unresolvable names in ``ForwardRef`` objects.
-  However, compound expressions using operators (``@``, ``|``) produce an
-  opaque ``ForwardRef`` string containing the entire expression.
+  However, when operators like ``@`` or ``|`` fail to evaluate because of an
+  unresolvable name, this format falls back to returning the entire expression
+  as an opaque string wrapped in a single ``ForwardRef`` object.
 - ``Format.STRING``: Returns the raw source text with no evaluation.
 
 For the ``@`` operator, ``Format.FORWARDREF`` is insufficient. Consider::
@@ -293,8 +264,8 @@ forward reference is resolved. This is a blocking issue for libraries like
 Pydantic and FastAPI, which inspect metadata at class-definition time.
 
 This proposal introduces a new format, ``Format.FORWARDREF_STRUCTURAL``.
-This format assumes typing semantics and evaluates compound type expressions
-**structurally**. It always returns an ``AnnotatedType`` for ``@``, a union
+This format assumes typing semantics and evaluates type expressions involving
+operators (like ``@`` and ``|``) **structurally**. It always returns an ``AnnotatedType`` for ``@``, a union
 for ``|``, and a ``GenericAlias`` for subscripting. When a name cannot be
 resolved, only that name is wrapped in ``ForwardRef``; the surrounding
 operators are still evaluated. The example above produces::
@@ -304,7 +275,10 @@ operators are still evaluated. The example above produces::
 The metadata is immediately accessible. This format also resolves the
 pre-existing issue with ``|`` unions, where ``"Foo" | int`` under
 ``Format.FORWARDREF`` produces ``ForwardRef('Foo | int')`` instead of the
-structural ``ForwardRef('Foo') | int``.
+structural ``ForwardRef('Foo') | int``. Notably, if ``FORWARDREF_STRUCTURAL``
+becomes the default evaluation strategy for type hints, it could eventually
+reduce or eliminate the need for runtime type objects to implement
+``__matmul__``.
 
 Interaction with PEP 563
 ^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -321,6 +295,38 @@ coarser than from :pep:`749` thunks. When a name is unresolvable, the
 rather than just a name. :pep:`749` provides a strictly better experience and
 is the recommended path forward.
 
+Parsing and Grammar
+===================
+
+This proposal requires no changes to the Python grammar. Because ``@`` is
+already a valid operator, it is natively parsed as a binary operation. The
+shorthand is resolved during semantic analysis, entirely bypassing the need
+to patch grammar files or update the parser.
+
+How to Teach This
+=================
+
+In Python, the ``@`` symbol already has an established association with
+metadata through decorators. The annotation shorthand extends this
+intuition to the type system: ``int @ Field(gt=0)`` reads as "``int``,
+decorated with ``Field(gt=0)``."
+
+For beginners, the key rule is simple: **in a type annotation, ``@`` means
+"with this metadata."** The full ``Annotated[int, Field(gt=0)]`` syntax
+remains available and is entirely equivalent for those who find it clearer.
+
+For experienced developers, the precedence rules follow standard Python
+operator precedence (``@`` binds tighter than ``|``), and chaining
+``T @ m1 @ m2`` flattens exactly as nested ``Annotated`` does.
+
+Documentation and teaching materials should introduce the shorthand alongside
+``Annotated``, not as a replacement. The longhand form is still preferred in
+contexts where multiple metadata items are passed as a group and chaining
+would be unwieldy.
+
+Backwards Compatibility
+=======================
+
 Operator Overloading
 --------------------
 
@@ -354,7 +360,8 @@ metaclass.
 The private ``typing._AnnotatedAlias`` class is retained as a deprecated
 compatibility shim. Code using ``isinstance(x, typing._AnnotatedAlias)``
 will continue to work but emit a ``DeprecationWarning``. The shim is
-scheduled for removal in Python 3.18.
+scheduled for removal in Python 3.21 (following the standard 5-year deprecation
+policy outlined in :pep:`387`).
 
 Code that should be updated: