Remove restriction on macros in __main__

[gilch]

https://macropy3.readthedocs.io/en/latest/overview.html says,

Note that this means you cannot use macros in a file that is run directly, as it will not be passed through the import hooks.

Sometimes I use Python for big projects, but sometimes I just need a one-page script and just want to import my macros and run it directly. It's a pain to have to write a new launcher to import it every time just so I can have macros.

Writing a macropy launcher aliased to replace python altogether would help some, but it messes up the if __name__ == '__main__': pattern I need to use sometimes. Then I'd have to remember to launch with normal python instead, but only if I didn't use macros...

Anyway, I think we can do better.

---

macros/__init__.py

import macropy.activate
from . import macros_in_main

macros/macros_in_main.py

import ast, importlib, inspect

from macropy.core.macros import ModuleExpansionContext, detect_macros

frame = inspect.currentframe()
while frame.f_globals["__name__"] != "__main__":
    frame = frame.f_back

source = inspect.getsource(frame)

tree = ast.parse(source)
exec(
    compile(
        ast.Module(
            ModuleExpansionContext(
                tree,
                source,
                [
                    (importlib.import_module(mod), bind)
                    for mod, bind in (detect_macros(tree, "__main__"))
                ],
            )
            .expand_macros()
            .body
        ),
        "__main__",
        "exec",
    )
)
raise SystemExit

main.py

import macros

from macropy.case_classes import macros, enum

@enum
class Direction:
    North, South, East, West

print(Direction(name="North")) # Direction.North

---

$ python main.py
Direction.North

This is just a proof of concept. I'm probably missing some edge cases and interactions, but it proves that it's at least possible to have syntactic macros, even in a file that is run "directly".

Maybe there's an easier way to do this, but I didn't see any kind of exec_with_macros() in the library.

Something like the above could perhaps replace the import macropy.activate to allow a script to be both imported and run on its own (the usual use case for the if __name__ == '__main__': construct).

[azazel75]

Another way to achieve the same goal is to use the SaveExporter as step in the testing/packaging of your sources, thus removing the need for macro expansion at runtime, have you tried that? This has the effect of launching the application/script in lesser time and at same time if the need to for debugging emerges you can follow the code like in any normal python script.

[gilch]

Oh, that looks useful actually. I must have missed that when I skimmed the docs the first time. The resulting pre-expanded files should have perfect compatibility with all the other Python tools.

But unless I'm actively debugging the macro itself, I would actually prefer to work with the unexpanded source that uses the macros if possible, even when debugging. It's generally more readable than the sometimes-complex expansion, for the same reason that it's preferable to work with source instead of assembly.

My approach seems to work fine if I add a breakpoint() somewhere in __main__. (When I tried this I also noticed that I used the wrong globals. I probably should have added frame.f_globals to my exec in macros/macros_in_main.py.)

SaveExporter does not really address my primary motivating use case for this issue: one-file scripts. It adds an extra layer of complexity (a compile step and output directory) for what could have been a single seamless import at the top of the script. CPython doesn't actually save a .pyc for __main__, only for imports. This means you can use them like shell scripts--copy and tweak as you need without worrying about an explicit compilation step or cluttering your working directory.

[gilch]

Improved version

import ast, importlib, inspect, sys

from macropy.core.macros import ModuleExpansionContext, detect_macros

__main__ = sys.modules["__main__"]
source = inspect.getsource(__main__)
if "import macros\n" in source:
    tree = ast.parse(source)
    exec(
        compile(
            source=ast.Module(
                ModuleExpansionContext(
                    tree,
                    source,
                    bindings=[
                        (importlib.import_module(mod), bind)
                        for mod, bind in detect_macros(tree, "__main__")
                    ],
                )
                .expand_macros()
                .body
            ),
            filename=__main__.__file__,
            mode="exec",
            dont_inherit=True,
        ),
        __main__.__dict__,
    )
    raise SystemExit

Turns out that we don't need the frame to get the source or the globals. Adding the __main__.__file__ seems to make the PyCharm debugger work too. The if checks that __main__ actually imported macros. If it's to be used in place of activate, we shouldn't re-run __main__ just because some other module imported it. This version just checks that the substring is present in the source, but it would be better to check that it's actually in the ast and not just in a docstring or something.

[vrthra]

You could also use the coding: ... trick. Essentially, register macropy as a codec, and use

# coding: macropy

within the script, which will get Python to decode the source file through macropy. For direct use, you will need to copy the codec to the Python encoder directory. See the readme of the project below.

See this example on how to do it.