feat(compiler): mirror memo output paths to Python source modules#6457
Conversation
Memos now compile into a single JSX file per user module at a path that mirrors the module's dotted name, instead of one file per memo under . The page-side import surface matches the source layout, which makes debugging easier and lets Vite group co-defined memos in the same chunk. Memos without a captured source module keep the legacy per-name files and index. A manifest in records emitted paths so stale files from previous compiles get pruned.
Merging this PR will not alter performance
Comparing 
Footnotes
|
Greptile SummaryThis PR changes how memo-compiled JSX files are laid out under
Confidence Score: 5/5The PR is safe to merge. All three issues flagged in prior review threads are addressed in the current implementation. The core path-mirroring logic, manifest-driven stale cleanup, and cross-module symbol disambiguation are all well-tested by the new unit and integration suites. The only new finding is a missing
Important Files Changed
Reviews (7): Last reviewed commit: "fix: don't treat reflex_components_* as ..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Identical memoizable subtrees on pages from different user modules produce the same wrapper tag. The auto-memo registry was keyed by tag alone, so the second registration overwrote the first — only one of the source modules got a mirrored memo file emitted, and the other page imported the tag from a JSX file that never declared it. Vite failed the prod build with MISSING_EXPORT. Key the registry by (tag, source_module) so each module's mirrored file gets its own definition, and add an integration test that builds two pages from distinct user modules sharing a memoizable subtree.
was d, so once a module had been resolved its mirrored path was frozen for the process. A user toggling a module between a regular and a package () during dev reload kept the original origin and emitted memo files to the stale path. Drop the cache and read from first, falling back to only when the module isn't loaded — is rebound on reload, while a cached spec wouldn't be. Also tighten to close the mkstemp fd up front so it can't leak if reopening raises, and re-export from for parity with the rest of the surface.
Memo components now mirror to their source module's combined file, so the self-referencing memo test can no longer find a per-name RecursiveBox.jsx path. Join all emitted code and assert on content.
Auto-memoized (rx.memo) components now compile to .web paths mirroring their defining Python module instead of a shared components.jsx, scoping the memo registry per module so same-named components no longer collide. Adds reflex_base.utils.memo_paths to translate source modules into the mirrored JSX path and $/... specifier.
This is net-new functionality, we don't need to provide compat shims
use more durable module/package names that are less likely to change in the future
1. the import was weird, why not just import directly? 2. the canonical location for _get_memo_component_class has moved to reflex_base
masenf
left a comment
masenf
left a comment
There was a problem hiding this comment.
we need to put the modules under a subdir of .web that is not already used, maybe .web/app_components.
The problem arises when I have an app called app and a module called app.root which exports a memo component... this ends up getting overwritten by the app/root.jsx that the framework emits and then the page breaks when you try to load it.
The user's file tree hierarchy really shouldn't be able to break the app, hence keeping it under a subdirectory that is only used for emitting memo components would protect the rest of the app from unexpected changes.
Mirrored memos could compile to paths that overwrite framework output (a memo in module `app.root` would clobber `.web/app/root.jsx`), and un-mirrorable memos shared `.web/utils/components`. Move all memo output under a reserved `app_components/` tree — mirrored memos at `app_components/<segments>`, un-mirrorable ones at `app_components/_internal/<name>` — so user module paths can never collide with framework files. Add collision detection (reserved internal dir, case-insensitive path clashes) and reject Windows reserved device names so mirroring fails loudly instead of silently overwriting. Reset the memo wrapper class cache each compile so a module flipping to a package across hot reloads re-resolves its library. Move stale-file pruning after the dry-run return so `--dry` never mutates `.web` or the manifest.
The is-absolute guard double-prefixed emitted paths to `.web/.web/...` when get_web_dir() returns the default relative path, so emitted keys never matched the manifest and live files were wrongly pruned. Emitted paths already share the web_dir prefix, so strip it directly.
masenf
left a comment
masenf
left a comment
There was a problem hiding this comment.
One other weird thing i noticed:
import reflex as rx
@rx.memo
def my_routes_func() -> rx.Component:
return rx.text("This is a memoized component in root.py that changed again.")
@rx.memo
def depends_on_shadowed_memo() -> rx.Component:
return rx.el.span(my_routes_func(), rx.text("that depends on the shadowed memo."))
@rx.memo
def my_routes_func() -> rx.Component:
return rx.text("This is a memoized component in root.py that overrides the same name.")When you have a module like this (even if you force eager eval of depends_on_shadowed_memo), it seems to always silently take the overridden memo component; no error, and only a single component emitted in the generated JS.
Another weird one i noticed, was that if i import my_routes_func into a module that also defines my_routes_func, it complains about a name overlap.
import reflex as rx
from .root import my_routes_func as root_routes_func
@rx.memo
def my_routes_func() -> rx.Component:
"""A memoized component that shares a name with one in root.py.
This tests that memos with the same name in different modules don't collide
and that the correct one is used in each place. The one in root.py should
be used in the memo that depends on it, and this one should be used in the
index page.
"""
return rx.text("This is a memoized component in app.py.")I'm assuming this one is because they compile to the same import name, so a single page cannot include both of them?
But the trace doesn't suggest as much, it looks like it caught it way earlier in the process, at definition time
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/multiprocessing/process.py", line 320, in _bootstrap
self.run()
~~~~~~~~^^
File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/masenf/code/reflex-dev/reflex/.venv/lib/python3.14/site-packages/granian/server/mp.py", line 90, in wrapped
callback = callback_loader()
File "/Users/masenf/code/reflex-dev/reflex/.venv/lib/python3.14/site-packages/granian/_internal.py", line 65, in load_target
module = load_module(path)
File "/Users/masenf/code/reflex-dev/reflex/.venv/lib/python3.14/site-packages/granian/_internal.py", line 47, in load_module
__import__(module_name)
~~~~~~~~~~^^^^^^^^^^^^^
File "/Users/masenf/code/reflex-dev/repro-memo-separate-modules/app/app/app.py", line 14, in <module>
@rx.memo
^^^^^^^
File "/Users/masenf/code/reflex-dev/reflex/packages/reflex-base/src/reflex_base/components/memo.py", line 1951, in memo
_register_memo_definition(definition)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/Users/masenf/code/reflex-dev/reflex/packages/reflex-base/src/reflex_base/components/memo.py", line 470, in _register_memo_definition
raise ValueError(msg)
ValueError: Memo name collision for `MyRoutesFunc`: `app.root.my_routes_func` and `app.app.my_routes_func` both compile to the same memo name.
☝️ that error is occuring because the key for checking for duplicate memo definitions does not include the source_file.
if i patch that in, then we hit the compile error that i had in mind
Traceback (most recent call last):
File "/Users/masenf/code/reflex-dev/reflex/reflex/utils/prerequisites.py", line 361, in compile_or_validate_app
get_compiled_app(
~~~~~~~~~~~~~~~~^
check_if_schema_up_to_date=check_if_schema_up_to_date,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
prerender_routes=prerender_routes,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
trigger=trigger,
^^^^^^^^^^^^^^^^
)
^
File "/Users/masenf/code/reflex-dev/reflex/reflex/utils/prerequisites.py", line 273, in get_compiled_app
app._compile(
~~~~~~~~~~~~^
prerender_routes=prerender_routes,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
trigger=trigger,
^^^^^^^^^^^^^^^^
)
^
File "/Users/masenf/code/reflex-dev/reflex/reflex/app.py", line 1329, in _compile
did_real_compile = compiler.compile_app(
self,
...<2 lines>...
use_rich=use_rich,
)
File "/Users/masenf/code/reflex-dev/reflex/reflex/compiler/compiler.py", line 1190, in compile_app
compile_ctx.compile(
~~~~~~~~~~~~~~~~~~~^
evaluate_progress=lambda: progress.advance(task),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
render_progress=lambda: progress.advance(task),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/Users/masenf/code/reflex-dev/reflex/packages/reflex-base/src/reflex_base/plugins/compiler.py", line 858,
in compile
compiler.compile_page_from_context(page_ctx)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/Users/masenf/code/reflex-dev/reflex/reflex/compiler/compiler.py", line 737, in compile_page_from_context
imports=utils.compile_imports(imports),
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/Users/masenf/code/reflex-dev/reflex/reflex/compiler/utils.py", line 127, in compile_imports
validate_imports(collapsed_import_dict)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/masenf/code/reflex-dev/reflex/reflex/compiler/utils.py", line 103, in validate_imports
raise ValueError(msg)
ValueError: Can not compile, the tag MyRoutesFunc is used multiple time from $/app_components/app/root and
$/app_components/app/app
So I think the fix for this one could be including the module in the import alias so we can actually disambiguate these when they're used in the same page.
I feel like the tests should have caught these. There's definitely intention in the test to catch these kind of edge cases. But the tests are too isolated and these kind of problems are occuring at integration points between the functions. We really should spend some time thinking about if the tests are really good, just because there's a lot of them and they are passing doesn't necessarily mean they are exercising the system in ways that will prevent regressions.
…nal dir Framework-defined memos now mirror to their real package name under app_components/ like any other module, instead of being forced into the shared fallback. Un-mirrorable memos (__main__, unsafe names) fall back to the existing utils/components/<name> layout, so the reserved app_components/_internal/ directory and its collision guard are no longer needed.
Two memos sharing a name in different modules mirrored to distinct files but still exported and imported the same JS identifier, so a page using both hit a tag collision. Derive a per-module-unique symbol (name plus a short hash of the dotted module) via the new memo_paths.library_and_symbol, and route every compiled tag/import/name through it. Key the MEMOS registry by (name, source_module) so same-named memos in different modules coexist rather than overwriting each other, while a genuine same-module shadow still resolves last-wins.
The reflex_components_* prefix is the convention community component libraries follow, so prefix-matching it as framework wrongly steered those modules' memos away from their real package name. Match framework packages by exact name or dot boundary only.
2 participants
Memos now compile into a single JSX file per user module at a path that mirrors the module's dotted name, instead of one file per memo under . The page-side import surface matches the source
layout, which makes debugging easier and lets Vite group co-defined memos in the same chunk.
Memos without a captured source module keep the legacy per-name files and index. A manifest in records emitted
paths so stale files from previous compiles get pruned.
All Submissions:
Type of change
Please delete options that are not relevant.
New Feature Submission:
Changes To Core Features:
closes #6218
fixes ENG-9150