diff --git a/ANNOTATION_INFO_REWORK.md b/ANNOTATION_INFO_REWORK.md new file mode 100644 index 000000000..9f9868c29 --- /dev/null +++ b/ANNOTATION_INFO_REWORK.md @@ -0,0 +1,133 @@ +# Annotation info rework — self-contained debug info + +## Problem + +Runtime debug info (`debug_info.h`) leaks AST pointers into the Context: + +- `VarInfo::annotation_arguments : void*` — points at `Structure::FieldDeclaration::annotation` + (an `AnnotationArgumentList` living inside the gc_node AST). +- `StructInfo::annotation_list : void*` — points at `Structure::annotations` (`AnnotationList`). +- `TypeInfo::annotation_or_name : mutable TypeAnnotation*` — tagged pointer: low bit set means + `~`-padded `module::name` string, lazily resolved against the bound environment; otherwise a + direct `TypeAnnotation*`. + +A Context can outlive its Program (program unload while context runs), so the first two dangle +unless the embedder keeps the ProgramPtr alive — which is what `options rtti` signals (it merely +keeps `context.thisProgram` non-null; the lifetime discipline is on the embedder). Function and +Enumeration annotations are not reflected at all. Both non-interpreter tiers already deep-copy: +AOT emits standalone static `AnnotationArguments` objects plus an init-time pointer fixup; JIT +heap-allocates copies via `jit_make_varinfo_annotations` / `jit_free_varinfo_annotations`. + +## Design + +Debug info becomes self-contained: annotation data is deep-copied into the per-context +`DebugInfoAllocator` at `DebugInfoHelper` time, unconditionally. Since that allocator is linear +and never runs destructors, the copies are POD mirrors with `allocateCachedName` strings: + +```cpp +struct AnnotationArgumentInfo { // POD, debug-heap allocated + Type type; // tBool/tInt/tFloat/tString + const char * name; + const char * sValue; + union { bool bValue; int iValue; float fValue; }; +}; +struct AnnotationInfo { // mirrors one AnnotationDeclaration (or a handled-type ref) + const char * name; // annotation name + const char * module_name; // declaring module + AnnotationArgumentInfo * arguments; // flat array + uint32_t count; + mutable Annotation * resolved; // lazy env-lookup cache +}; +``` + +Field changes: + +| struct | before | after | +|---|---|---| +| `VarInfo` | `void * annotation_arguments` | `AnnotationArgumentInfo * annotation_arguments; uint32_t annotation_count` | +| `StructInfo` | `void * annotation_list` | `AnnotationInfo * annotations; uint32_t annotation_count` | +| `FuncInfo` | — | `AnnotationInfo * annotations; uint32_t annotation_count` (NEW) | +| `EnumInfo` | — | `AnnotationInfo * annotations; uint32_t annotation_count` (NEW) | +| `TypeInfo` | `mutable TypeAnnotation * annotation_or_name` (union) | `AnnotationInfo * annotation_info` (union) | + +Globals (`Variable::annotation` is a bare `AnnotationArgumentList`, same shape as struct fields) +are captured into `VarInfo.annotation_arguments` too (NEW). + +Resolution: `TypeInfo::getAnnotation()` (and the new `resolve_annotation` das builtin) look up +`module_name::name` in the bound environment's module list and cache into `resolved` +(`g_resolve_annotations` gates the cache write, as today). The `~`-padding alignment hack and +the low-bit pointer tag in `Module::resolveAnnotation` are deleted; both +`DAS_THREAD_SAFE_ANNOTATIONS` paths unify into always-name-based + lazy resolve. The invariant +"modules outlive contexts" is unchanged; what's removed is the dependence on *Program* lifetime. + +`DebugInfoHelper::rtti` dies. Everything it gated becomes unconditional: annotation capture, +field/global constexpr init-value capture, builtin `cppName`. `options rtti`'s only remaining +effect is keeping `context.thisProgram` for compile-time/macro introspection (`this_program()`, +`ast_typedecl`). Memory cost is near-zero for unannotated entities (null + 0) and dedup of +strings comes free via `allocateCachedName`. + +## das API + +New (rtti module): + +- `each_annotation(s : StructInfo) : iterator` — same for `FuncInfo`, `EnumInfo` +- `each_annotation_argument(v : VarInfo)` / direct indexed access to `annotation_arguments` +- `get_annotation_argument_value(arg : AnnotationArgumentInfo) : RttiValue` +- `resolve_annotation(ann : AnnotationInfo) : Annotation?` — live object, env lookup + cache + +Deprecated (kept working, `[deprecated]`): + +- `structure_for_each_annotation(st, $(ann : Annotation; args : AnnotationArguments))` — shim + reconstructs a stack `AnnotationArgumentList` from the POD copies and env-resolves the + `Annotation*` for the duration of the block invoke. Works without `options rtti` now. + +## Tier impact + +- **AOT** (`daslib/aot_cpp.das`): emits static `AnnotationArgumentInfo` / `AnnotationInfo` + arrays (now also for functions/enums/globals). The init-time field-annotation fixup function + dies — the mirrors are static-initializable, no `std::string` construction at load. Init-time + `resolveAnnotation()` pre-warm for handled types stays. Ctor/layout change makes stale external + AOT files fail loudly at compile (desired). +- **JIT** (`modules/dasLLVM/daslib/llvm_jit.das`): hard-coded VarInfo/StructInfo/EnumInfo/ + TypeInfo layout tables updated; `jit_initialize_varinfo_annotations` / + `jit_free_varinfo_annotations` shims AND the tagged-pointer + `handle_typeinfo_constructor` deleted — `AnnotationInfo` globals are plain static data + (the old ptrtoint|1 scheme existed because wasm32 forbids widening ptrtoint in .data). + JIT *gains* struct/enum annotation lists (previously always null). + `LLVM_JIT_CODEGEN_VERSION` bump. +- **Interpreter**: the fix itself. + +## das-side string fields and AOT + +`AnnotationInfo.name` / `AnnotationArgumentInfo.name` (and `sValue`, `module_name`) are +`const char *` bound as das `string` — same as `VarInfo.name`. Comparisons and string +interpolation work in all tiers, but passing the field *directly as a function argument* +compiles interpreted and fails AOT C++ compilation (`const char*` → `char*`). This is a +pre-existing emitter wart shared by every `const char *` bound field; the in-tree +convention is to interpolate (`"{ann.name}"`) when passing onward. + +## Known pre-existing gap (unchanged) + +AOT-emitted `VarInfo` statics carry `flag_hasInitValue` in flags but never emit the actual +`value`/`sValue` payload (stays zero) — true before this rework under `options rtti`, true +after it for everyone. `get_variable_value` consumers go through helper-built debug info +(fresh from the live Module AST), so nothing in-tree reads the static payload. + +## Affected consumers (in-tree) + +`src/simulate/json_print.cpp`, `src/simulate/json_scan.cpp`, `src/builtin/module_jit.cpp`, +`src/builtin/module_builtin_rtti.cpp`, `daslib/rtti.das`, `daslib/aot_cpp.das`, +`modules/dasLLVM/daslib/llvm_jit.das`, `utils/daslang-live/main.cpp`, +`tutorials/integration/cpp/06_interop.cpp` (+ RST), `tests/language/reflection.das`. + +## Tests + +- function/enum/global annotation reflection (new capability) +- annotations readable WITHOUT `options rtti` +- tests-cpp: read annotations from a live Context after the ProgramPtr is released +- existing reflection.das / json annotation behavior preserved + +## Post-merge + +- daspkg externals sweep for `annotation_arguments` / `annotation_or_name` consumers + (same drill as FIXED_ARRAY_REWORK). diff --git a/daslib/aot_cpp.das b/daslib/aot_cpp.das index fc7e86a83..66ea6a4d7 100644 --- a/daslib/aot_cpp.das +++ b/daslib/aot_cpp.das @@ -639,10 +639,53 @@ class public DebugVarCache { } +def private writeAnnotationArgInit(var sb : StringBuilderWriter; arg : AnnotationArgumentInfo) { + if (arg.basicType == Type.tBool) { + write(sb, "AnnotationArgumentInfo(\"{arg.name}\", {arg.bValue})") + } elif (arg.basicType == Type.tString) { + write(sb, "AnnotationArgumentInfo(\"{arg.name}\", \"{escape(arg.sValue)}\")") + } elif (arg.basicType == Type.tInt) { + write(sb, "AnnotationArgumentInfo(\"{arg.name}\", {arg.iValue})") + } elif (arg.basicType == Type.tFloat) { + write(sb, "AnnotationArgumentInfo(\"{arg.name}\", {to_cpp_float(arg.fValue)})") + } +} + +// emits AnnotationArgumentInfo + AnnotationInfo statics for a Struct/Func/Enum info +def private writeAnnotationList(var writer : StringBuilderWriter?; info; baseName : string) { + if (info.annotation_count == 0u) return ; + for (ai in range(int(info.annotation_count))) { + unsafe { + let ann & = get_annotation(*info, ai) + if (ann.count > 0u) { + let annArgs = build_string() $(var sb) { + for (gi in range(int(ann.count))) { + if (gi != 0) { write(sb, ", "); } + writeAnnotationArgInit(sb, get_annotation_argument(ann, gi)) + } + } + write(*writer, "AnnotationArgumentInfo {baseName}_ann_{ai}_args[{int(ann.count)}] = \{ {annArgs} \};\n") + } + } + } + let anns = build_string() $(var sb) { + for (ai in range(int(info.annotation_count))) { + if (ai != 0) { write(sb, ", "); } + unsafe { + let ann & = get_annotation(*info, ai) + let argsRef = ann.count > 0u ? "{baseName}_ann_{ai}_args" : "nullptr" + write(sb, "AnnotationInfo(\"{ann.name}\", \"{ann.module_name}\", {argsRef}, {int(ann.count)}u)") + } + } + } + write(*writer, "AnnotationInfo {baseName}_ann[{int(info.annotation_count)}] = \{ {anns} \};\n") +} + class public AotDebugInfoHelper { //! Helper for generating debug type and variable information in AOT C++ output. private info2Name = new DebugVarCache() private info2TypeName = new DebugVarCache() + private annInfoNames : table // "module::name" -> emitted AnnotationInfo identifier helper : DebugInfoHelper? = new DebugInfoHelper(uninitialized) cross_platform : bool = false def operator delete { @@ -695,6 +738,7 @@ class public AotDebugInfoHelper { write(writer, "extern TypeInfo {typeInfoName(ti)};\n"); }) write(writer, "\n"); + writeHandledAnnotations(unsafe(addr(writer))); helper |> debug_helper_iter_enums($(_name, ti){ describeCppEnumInfoValues(unsafe(addr(writer)), ti); write(writer, "EnumInfo {enumInfoName(ti)} = \{ {describeCppEnumInfo(ti)} \};\n"); @@ -726,21 +770,61 @@ class public AotDebugInfoHelper { write(writer, " for (auto& ann : annotations) \{\n") write(writer, " ann.resolveAnnotation();\n") write(writer, " \}\n") - helper |> debug_helper_iter_structs($(_name, si) { - if (si.fields == null) return ; - for (fi in range(si.count)) { - let fld & = unsafe(si.fields[fi]) - if (fld.annotation_arguments == null || empty(*fld.annotation_arguments)) continue; - write(writer, " {structInfoName(si)}_field_{fi}.annotation_arguments = &{structInfoName(si)}_field_{fi}_ann;\n") - } - }) write(writer, "\}\n\n") info2Name.clear(); info2TypeName.clear(); + annInfoNames |> clear() } } + def registerHandledAnnotation(var writer : StringBuilderWriter?; info : TypeInfo?) { + if (info._type != Type.tHandle) return ; + let ann = info.annotation + let key = "{ann._module.name}::{ann.name}" + if (annInfoNames |> key_exists(key)) return ; + let ident = "__handled_ann_{length(annInfoNames)}" + annInfoNames[key] = ident + write(*writer, "AnnotationInfo {ident} = \{ \"{ann.name}\", \"{ann._module.name}\", nullptr, 0u \};\n") + } + + // dedup'd AnnotationInfo statics for every handled type referenced from any TypeInfo/VarInfo + def writeHandledAnnotations(var writer : StringBuilderWriter?) { + helper |> debug_helper_iter_types($(_name, ti) { + registerHandledAnnotation(writer, ti); + }) + helper |> debug_helper_iter_structs($(_name, si) { + if (si.fields == null) return ; + for (fi in range(si.count)) { + registerHandledAnnotation(writer, unsafe(si.fields[fi])); + } + }) + helper |> debug_helper_iter_funcs($(_name, fni) { + if (fni.fields == null) return ; + for (fi in range(fni.count)) { + registerHandledAnnotation(writer, unsafe(fni.fields[fi])); + } + }) + } + + def annInfoRef(info : TypeInfo?) : string { + let ann = info.annotation + return "&" + annInfoNames.get_value("{ann._module.name}::{ann.name}") + } + + def writeVarAnnotationArgs(var writer : StringBuilderWriter?; fld : VarInfo?; arrName : string) { + if (fld.annotation_argument_count == 0u) return ; + let annArgs = build_string() $(var sb) { + for (ai in range(int(fld.annotation_argument_count))) { + if (ai != 0) { write(sb, ", "); } + unsafe { + writeAnnotationArgInit(sb, get_annotation_argument(*fld, ai)) + } + } + } + write(*writer, "AnnotationArgumentInfo {arrName}[{int(fld.annotation_argument_count)}] = \{ {annArgs} \};\n") + } + def describeCppVarInfo(struct_name : string; info : VarInfo?; suffix : string) { return build_string() $(writer) { write(writer, "{describeCppTypeInfo(info, suffix)}, \"{info.name}\", "); @@ -761,6 +845,7 @@ class public AotDebugInfoHelper { } } def describeCppStructInfoFields(var writer : StringBuilderWriter?; info : StructInfo?) { + writeAnnotationList(writer, info, structInfoName(info)); if (info.fields == null) return ; for (fi in range(info.count)) { let suffix = "_var_{info.hash:d}"; @@ -769,30 +854,11 @@ class public AotDebugInfoHelper { writeArgTypes(writer, fld, suffix); writeArgNames(writer, fld, suffix); let prefix = (info.module_name |> !empty(info.module_name)) ? "{info.module_name}::" : ""; + writeVarAnnotationArgs(writer, fld, "{structInfoName(info)}_field_{fi}_ann"); + let fldAnnRef = fld.annotation_argument_count > 0u ? ", {structInfoName(info)}_field_{fi}_ann, {int(fld.annotation_argument_count)}u" : "" // info.name is the struct's daslang name; mangle if C++ keyword so offsetof() // in describeCppVarInfo sees the same identifier the struct decl emitted. - write(*writer, "VarInfo {structInfoName(info)}_field_{fi} = \{ {describeCppVarInfo(prefix + aotSuffixNameEx(info.name, "_S"), fld,suffix)} \};\n"); - if (fld.annotation_arguments != null) { - if (length(*fld.annotation_arguments) > 0) { - let annArgs = build_string() $(var sb) { - var first = true - for (arg in *fld.annotation_arguments) { - if (!first) { write(sb, ", "); } - first = false - if (arg.basicType == Type.tBool) { - write(sb, "AnnotationArgument(\"{arg.name}\", {arg.bValue})") - } elif (arg.basicType == Type.tString) { - write(sb, "AnnotationArgument(\"{arg.name}\", string(\"{arg.sValue}\"))") - } elif (arg.basicType == Type.tInt) { - write(sb, "AnnotationArgument(\"{arg.name}\", {arg.iValue})") - } elif (arg.basicType == Type.tFloat) { - write(sb, "AnnotationArgument(\"{arg.name}\", {arg.fValue}f)") - } - } - } - write(*writer, "static AnnotationArguments {structInfoName(info)}_field_{fi}_ann = \{ {annArgs} \};\n") - } - } + write(*writer, "VarInfo {structInfoName(info)}_field_{fi} = \{ {describeCppVarInfo(prefix + aotSuffixNameEx(info.name, "_S"), fld,suffix)}{fldAnnRef} \};\n"); } let fields = (each(range(info.count)) ._select("&{structInfoName(info)}_field_{_}") @@ -810,10 +876,12 @@ class public AotDebugInfoHelper { write(writer, "nullptr, ") } let info_size = cross_platform ? "TypeSize<{helper |> debug_helper_find_struct_cppname(info)}>::size" : "{info.size:d}"; - write(writer, "{info.count:d}, {info_size}, UINT64_C(0x{info.init_mnh:x}), nullptr, UINT64_C(0x{info.hash:x}), {info.firstGcField:d}") + let annRef = info.annotation_count > 0u ? "{structInfoName(info)}_ann" : "nullptr"; + write(writer, "{info.count:d}, {info_size}, UINT64_C(0x{info.init_mnh:x}), {annRef}, {int(info.annotation_count)}u, UINT64_C(0x{info.hash:x}), {info.firstGcField:d}") } } def describeCppFuncInfoFields(var writer : StringBuilderWriter?; info : FuncInfo?) { + writeAnnotationList(writer, info, funcInfoName(info)); if (info.fields == null) return ; for (fi in range(info.count)) { let suffix = "_var_{info.hash:d}"; @@ -821,7 +889,9 @@ class public AotDebugInfoHelper { writeDim(writer, fld, suffix); writeArgTypes(writer, fld, suffix); writeArgNames(writer, fld, suffix); - write(*writer, "VarInfo {funcInfoName(info)}_field_{fi} = \{ {describeCppVarFuncInfo(info.name, fld,suffix)} \};\n"); + writeVarAnnotationArgs(writer, fld, "{funcInfoName(info)}_field_{fi}_ann"); + let fldAnnRef = fld.annotation_argument_count > 0u ? ", {funcInfoName(info)}_field_{fi}_ann, {int(fld.annotation_argument_count)}u" : "" + write(*writer, "VarInfo {funcInfoName(info)}_field_{fi} = \{ {describeCppVarFuncInfo(info.name, fld,suffix)}{fldAnnRef} \};\n"); } let fields = (each(range(info.count)) @@ -839,6 +909,9 @@ class public AotDebugInfoHelper { write(writer, "nullptr, ") } write(writer, "{info.count:d}, {info.stackSize:d}, &{typeInfoName(info.result)}, nullptr,0,UINT64_C(0x{info.hash:x}), 0x{info.flags:x}"); + if (info.annotation_count > 0u) { + write(writer, ", {funcInfoName(info)}_ann, {int(info.annotation_count)}u"); + } } } def describeCppEnumInfoValues(var writer : StringBuilderWriter?; einfo : EnumInfo?) { @@ -866,9 +939,11 @@ class public AotDebugInfoHelper { .to_array() ._fold()) |> join(", ") write(*writer, "EnumValueInfo * {enumInfoName(einfo)}_values [] = \{ {enum_vals} \};\n"); + writeAnnotationList(writer, einfo, enumInfoName(einfo)); } def describeCppEnumInfo(info : EnumInfo?) { - return "\"{info.name}\", \"{info.module_name}\", {enumInfoName(info)}_values, {info.count:d}, UINT64_C(0x{info.hash:x}), {info.flags:d}"; + let annRef = info.annotation_count > 0u ? ", {enumInfoName(info)}_ann, {int(info.annotation_count)}u" : ""; + return "\"{info.name}\", \"{info.module_name}\", {enumInfoName(info)}_values, {info.count:d}, UINT64_C(0x{info.hash:x}), {info.flags:d}{annRef}"; } def describeCppTypeInfo(info : TypeInfo?; suffix : string = "") { return build_string() $(writer) { @@ -891,20 +966,8 @@ class public AotDebugInfoHelper { } write_sep(); if (info._type == Type.tHandle) { - let ann_ptr : void? = info.annotation_or_name - let int_ptr = unsafe(reinterpret(ann_ptr)); - if ((int_ptr & uint64(1)) != uint64(0)) { - let tname = unsafe(reinterpret(int_ptr ^ uint64(1))); // already comes from string allocator - write(writer, "DAS_MAKE_ANNOTATION(\"{tname}\")"); - } else { - // we add ~ at the beginning of the name for padding - // if name is allocated by the compiler, it does not guarantee that it is aligned - // we check if there is a ~ at the beginning of the name, and if it is - we skip it - // that way we can accept both aligned and unaligned names - write(writer, "DAS_MAKE_ANNOTATION(\"~{info.annotation_or_name._module.name}::{info.annotation_or_name.name}\")"); - } + write(writer, annInfoRef(info)); } else { - assert(info._type != Type.tHandle); write(writer, "nullptr"); } write_sep(); @@ -1195,7 +1258,6 @@ class public CppAot : AstVisitor { //! Main AST visitor that generates C++ ahead-of-time compiled code from daslang AST. /* CppAot ( const ProgramPtr & prog, BlockVariableCollector & cl ) : program(prog), collector(cl) { - helper.rtti = program.options.getBoolOption("rtti",false); prologue = program.options.getBoolOption("aot_prologue",false) || program.getDebugger(); solidContext = program.policies.solid_context || program.options.getBoolOption("solid_context",false); @@ -4150,7 +4212,6 @@ def public run_aot(prog : Program?; var pctx : Context?; cop : CodeOfPolicies) : solidContext = program.policies.solid_context || (program._options |> find_arg("solid_context") ?as tBool ?? false), cross_platform = cop.cross_platform) - cpp_aot.helper.helper.rtti = program._options |> find_arg("rtti") ?as tBool ?? false; cpp_aot.helper.cross_platform = cop.cross_platform; make_visitor(*cpp_aot) $(adapter) { cpp_aot.adapter := adapter @@ -4226,7 +4287,6 @@ def public run_aot_function(prog : Program?; var pctx : Context?; cop : CodeOfPo (program._options |> find_arg("solid_context") ?as tBool ?? false), cross_platform = cop.cross_platform, aot_filter_function = func_name) - cpp_aot.helper.helper.rtti = program._options |> find_arg("rtti") ?as tBool ?? false; cpp_aot.helper.cross_platform = cop.cross_platform; make_visitor(*cpp_aot) $(adapter) { cpp_aot.adapter := adapter diff --git a/daslib/aot_standalone.das b/daslib/aot_standalone.das index d9932b01b..47ec32b15 100644 --- a/daslib/aot_standalone.das +++ b/daslib/aot_standalone.das @@ -10,7 +10,6 @@ require daslib/strings_boost require daslib/ast_boost require daslib/templates_boost require daslib/functional -require daslib/algorithm require daslib/ast_print_flags require daslib/aot_constants @@ -31,7 +30,7 @@ def aotFunctionName(str : string) { return replace(str, "`", "__") } -def getInitSemanticHash(ctx : Context) { +def getInitSemanticHash(_ctx : Context) { /* const uint64_t fnv_prime = 1099511628211ul; uint64_t hash = globalsSize ^ sharedSize; @@ -221,7 +220,6 @@ class StandaloneContextGen : CppAot { cross_platform = cp; prologue = program._options |> find_arg("aot_prologue") ?as tBool ?? false; solidContext = program.policies.solid_context || (program._options |> find_arg("solid_context") ?as tBool ?? false); - helper.helper.rtti = program._options |> find_arg("rtti") ?as tBool ?? false; helper.cross_platform = cp; } @@ -309,8 +307,7 @@ def GetFunctionInfo(pfun : Function?, info : string) { } } -def addFunctionInfo(disableInit : bool; rtti : bool, fnn : array, var helper : AotDebugInfoHelper?) { - helper.helper.rtti = rtti; +def addFunctionInfo(fnn : array; var helper : AotDebugInfoHelper?) { var lookupFunctionTable : array> lookupFunctionTable |> reserve(length(fnn)) for (pfun in fnn) { @@ -349,10 +346,7 @@ def genStandaloneSrc(var program : ProgramPtr; program |> visit_module(adapter, program.getThisModule); } - initFunctions = addFunctionInfo(program._options |> find_arg("no_init") ?as tBool ?? program.policies.no_init, - program._options |> find_arg("rtti") ?as tBool ?? program.policies.rtti, - collectProgramUsedFunctions(program, false, false), - gen.helper); + initFunctions = addFunctionInfo(collectProgramUsedFunctions(program, false, false), gen.helper); write(tw, gen.str()); unsafe { delete gen.collector @@ -444,7 +438,7 @@ def public runStandaloneVisitor(var program : ProgramPtr, modules : array split_by_chars("/\\") |> back() diff --git a/daslib/rtti.das b/daslib/rtti.das index 5239a13e8..6b52ed4a5 100644 --- a/daslib/rtti.das +++ b/daslib/rtti.das @@ -10,11 +10,78 @@ require strings typedef FileAccessPtr = smart_ptr -[generic] +//! Deprecated: resolves each AnnotationInfo back to the live Annotation object and +//! reconstructs an AST-shaped argument list per invoke. Use `each_annotation` instead. +[generic, deprecated] def structure_for_each_annotation(st : StructInfo; subexpr : block<(ann : Annotation; args : AnnotationArguments) : void>) { rtti_builtin_structure_for_each_annotation(st, subexpr) } +def private each_annotation_range(pinfo; annotation_count : uint) { + return <- generator { + for (index in range(int(annotation_count))) { + unsafe { + yield get_annotation(*pinfo, index) + } + } + return false + } +} + +//! Iterates annotations (`[name(args)]`) of a structure, function, or enumeration. +[generic] +def each_annotation(info : StructInfo) { + unsafe { + let pinfo = addr(info) + return <- each_annotation_range(pinfo, info.annotation_count) + } +} + +[generic] +def each_annotation(info : FuncInfo) { + unsafe { + let pinfo = addr(info) + return <- each_annotation_range(pinfo, info.annotation_count) + } +} + +[generic] +def each_annotation(info : EnumInfo) { + unsafe { + let pinfo = addr(info) + return <- each_annotation_range(pinfo, info.annotation_count) + } +} + +def private each_annotation_argument_range(pinfo; argument_count : uint) { + return <- generator { + for (index in range(int(argument_count))) { + unsafe { + yield get_annotation_argument(*pinfo, index) + } + } + return false + } +} + +//! Iterates the arguments of one annotation. +[generic] +def each_annotation_argument(info : AnnotationInfo) { + unsafe { + let pinfo = addr(info) + return <- each_annotation_argument_range(pinfo, info.count) + } +} + +//! Iterates field/global annotation arguments (`@name = value` on a field). +[generic] +def each_annotation_argument(info : VarInfo) { + unsafe { + let pinfo = addr(info) + return <- each_annotation_argument_range(pinfo, info.annotation_argument_count) + } +} + [generic] def is_same_type(a, b : TypeInfo; refMatters : RefMatters = RefMatters.yes; diff --git a/doc/reflections/das2rst.das b/doc/reflections/das2rst.das index 710ab7c73..eba876524 100644 --- a/doc/reflections/das2rst.das +++ b/doc/reflections/das2rst.das @@ -314,7 +314,7 @@ def document_module_rtti(_root : string) { group_by_regex("Rtti context access", mod, %regex~(get_total_functions|get_total_variables|get_function_info|get_variable_info|get_variable_value|get_function_by_mnh|get_line_info|this_context|context_for_each_function|context_for_each_variable|class_info|type_info)$%%), group_by_regex("Program access", mod, %regex~(program_for_each_module|program_for_each_registered_module|get_this_module|get_module|has_module)$%%), group_by_regex("Module access", mod, %regex~(module_for_each_structure|module_for_each_enumeration|module_for_each_function|module_for_each_generic|module_for_each_global|module_for_each_annotation|module_for_each_dependency)$%%), - group_by_regex("Annotation access", mod, %regex~(get_annotation_argument_value|add_annotation_argument)$%%), + group_by_regex("Annotation access", mod, %regex~(get_annotation_argument_value|add_annotation_argument|get_annotation|get_annotation_argument|resolve_annotation|each_annotation|each_annotation_argument)$%%), group_by_regex("Compilation and simulation", mod, %regex~(compile|compile_file|for_each_expected_error|for_each_require_declaration|simulate|create_ast_serializer|create_ast_deserializer|delete_ast_serializer|serialize_program|deserialize_program|ast_serializer_get_data)$%%), group_by_regex("File access", mod, %regex~(make_file_access|set_file_source|add_file_access_root|add_extra_module)$%%), group_by_regex("Structure access", mod, %regex~(rtti_builtin_structure_for_each_annotation|basic_struct_for_each_field|structure_for_each_annotation|basic_struct_for_each_parent)$%%), diff --git a/doc/source/reference/embedding/cpp_api.rst b/doc/source/reference/embedding/cpp_api.rst index f6a414cee..0269dc7a7 100644 --- a/doc/source/reference/embedding/cpp_api.rst +++ b/doc/source/reference/embedding/cpp_api.rst @@ -362,7 +362,7 @@ Key capabilities (vs ``addExtern``): * ``vec4f`` argument type = "any" — accepts any daslang type **TypeInfo union warning:** ``TypeInfo`` has a union — -``structType``, ``enumType``, and ``annotation_or_name`` share memory. +``structType``, ``enumType``, and ``annotation_info`` share memory. Which member is valid depends on ``ti->type``: * ``tStructure`` → ``ti->structType`` diff --git a/doc/source/reference/tutorials/integration_cpp_06_interop.rst b/doc/source/reference/tutorials/integration_cpp_06_interop.rst index 0d8597d8d..70c010d15 100644 --- a/doc/source/reference/tutorials/integration_cpp_06_interop.rst +++ b/doc/source/reference/tutorials/integration_cpp_06_interop.rst @@ -96,21 +96,22 @@ The ``TypeInfo`` union .. code-block:: cpp union { - StructInfo * structType; // tStructure - EnumInfo * enumType; // tEnumeration - mutable TypeAnnotation * annotation_or_name; // tHandle + StructInfo * structType; // tStructure + EnumInfo * enumType; // tEnumeration + AnnotationInfo * annotation_info; // tHandle }; .. warning:: Accessing the wrong union member is **undefined behavior**. Always check ``ti->type`` before accessing ``structType``, ``enumType``, or - ``annotation_or_name``. + ``annotation_info``. For handled types (``type == tHandle``), use ``ti->getAnnotation()`` -to safely resolve the annotation — it handles tagged-pointer resolution -automatically. ``das_to_string(Type::tHandle)`` returns an empty string; -use ``ti->getAnnotation()->name`` for the type name. +to safely resolve the live ``TypeAnnotation`` — ``annotation_info`` only +carries the annotation name and module name (resolved lazily against the +registered modules and cached). ``das_to_string(Type::tHandle)`` returns +an empty string; use ``ti->getAnnotation()->name`` for the type name. Registering interop functions diff --git a/doc/source/stdlib/handmade/function-rtti-get_annotation-0xdbc45dc33bc67300.rst b/doc/source/stdlib/handmade/function-rtti-get_annotation-0xdbc45dc33bc67300.rst new file mode 100644 index 000000000..dad3e5777 --- /dev/null +++ b/doc/source/stdlib/handmade/function-rtti-get_annotation-0xdbc45dc33bc67300.rst @@ -0,0 +1 @@ +Returns the AnnotationInfo at the given index for a ``StructInfo``, ``FuncInfo``, or ``EnumInfo``. Panics when the index is out of range; ``annotation_count`` carries the valid range. diff --git a/doc/source/stdlib/handmade/function-rtti-get_annotation_argument-0xcf249595f05c3934.rst b/doc/source/stdlib/handmade/function-rtti-get_annotation_argument-0xcf249595f05c3934.rst new file mode 100644 index 000000000..12ac4f15d --- /dev/null +++ b/doc/source/stdlib/handmade/function-rtti-get_annotation_argument-0xcf249595f05c3934.rst @@ -0,0 +1 @@ +Returns the AnnotationArgumentInfo at the given index of an ``AnnotationInfo`` (or the field/global annotation arguments of a ``VarInfo``). Panics when the index is out of range. diff --git a/doc/source/stdlib/handmade/function-rtti-resolve_annotation-0xdf30cb3959a59b9e.rst b/doc/source/stdlib/handmade/function-rtti-resolve_annotation-0xdf30cb3959a59b9e.rst new file mode 100644 index 000000000..3e0babc17 --- /dev/null +++ b/doc/source/stdlib/handmade/function-rtti-resolve_annotation-0xdf30cb3959a59b9e.rst @@ -0,0 +1 @@ +Resolves an ``AnnotationInfo`` to the live ``Annotation`` object by looking up its module and name among the registered modules. Returns null when the annotation's module is no longer registered. Successful lookups are cached. diff --git a/doc/source/stdlib/handmade/structure_annotation-rtti-AnnotationArgumentInfo.rst b/doc/source/stdlib/handmade/structure_annotation-rtti-AnnotationArgumentInfo.rst new file mode 100644 index 000000000..dab68b7ea --- /dev/null +++ b/doc/source/stdlib/handmade/structure_annotation-rtti-AnnotationArgumentInfo.rst @@ -0,0 +1,7 @@ +One argument of an annotation, deep-copied into the context debug heap (never points into the AST). +argument type - tBool, tInt, tFloat, or tString +argument name +string value (tString arguments only) +boolean value +integer value +floating point value diff --git a/doc/source/stdlib/handmade/structure_annotation-rtti-AnnotationInfo.rst b/doc/source/stdlib/handmade/structure_annotation-rtti-AnnotationInfo.rst new file mode 100644 index 000000000..8d89a8d9e --- /dev/null +++ b/doc/source/stdlib/handmade/structure_annotation-rtti-AnnotationInfo.rst @@ -0,0 +1,4 @@ +One annotation attached to a structure, function, or enumeration - name, declaring module, and arguments, deep-copied into the context debug heap so it stays valid after the Program is released. Use ``resolve_annotation`` for the live Annotation object. +annotation name +name of the module where the annotation is declared +number of arguments diff --git a/doc/source/stdlib/handmade/structure_annotation-rtti-DebugInfoHelper.rst b/doc/source/stdlib/handmade/structure_annotation-rtti-DebugInfoHelper.rst index 782a0537c..edaf2d8a0 100644 --- a/doc/source/stdlib/handmade/structure_annotation-rtti-DebugInfoHelper.rst +++ b/doc/source/stdlib/handmade/structure_annotation-rtti-DebugInfoHelper.rst @@ -1,2 +1 @@ Helper object which holds debug information about the simulated program. -The RTTI context pointer. diff --git a/doc/source/stdlib/handmade/structure_annotation-rtti-EnumInfo.rst b/doc/source/stdlib/handmade/structure_annotation-rtti-EnumInfo.rst index 73a79e5de..1f2830034 100644 --- a/doc/source/stdlib/handmade/structure_annotation-rtti-EnumInfo.rst +++ b/doc/source/stdlib/handmade/structure_annotation-rtti-EnumInfo.rst @@ -5,3 +5,4 @@ fields in the enumeration number of fields in the enumeration hash of the enumeration flags of the enumeration +number of annotations attached to the enumeration diff --git a/doc/source/stdlib/handmade/structure_annotation-rtti-FuncInfo.rst b/doc/source/stdlib/handmade/structure_annotation-rtti-FuncInfo.rst index 5d758ffbd..10e793c39 100644 --- a/doc/source/stdlib/handmade/structure_annotation-rtti-FuncInfo.rst +++ b/doc/source/stdlib/handmade/structure_annotation-rtti-FuncInfo.rst @@ -11,3 +11,4 @@ number of arguments in the function stack size in bytes number of local variables number of accessed global variables +number of annotations attached to the function diff --git a/doc/source/stdlib/handmade/structure_annotation-rtti-StructInfo.rst b/doc/source/stdlib/handmade/structure_annotation-rtti-StructInfo.rst index 7c59d3b56..93b8f0fe9 100644 --- a/doc/source/stdlib/handmade/structure_annotation-rtti-StructInfo.rst +++ b/doc/source/stdlib/handmade/structure_annotation-rtti-StructInfo.rst @@ -8,3 +8,4 @@ flags associated with the structure number of fields in the structure size of the structure in bytes index of the first GC field in the structure, i.e. field which requires garbage collection marking +number of annotations attached to the structure diff --git a/include/daScript/ast/ast.h b/include/daScript/ast/ast.h index 5d4934e42..759a1dbc6 100644 --- a/include/daScript/ast/ast.h +++ b/include/daScript/ast/ast.h @@ -17,10 +17,6 @@ #define DAS_ALLOW_ANNOTATION_LOOKUP 1 #endif -#ifndef DAS_THREAD_SAFE_ANNOTATIONS -#define DAS_THREAD_SAFE_ANNOTATIONS 1 -#endif - namespace das { @@ -1181,6 +1177,7 @@ namespace das static void ClearSharedModules(); static void CollectSharedModules(); static TypeAnnotation * resolveAnnotation ( const TypeInfo * info ); + static Annotation * resolveAnnotation ( const AnnotationInfo * info ); static Type findOption ( const string & name ); static void foreach(const callable & func); virtual uintptr_t rtti_getUserData() {return uintptr_t(0);} @@ -1483,12 +1480,13 @@ namespace das FuncInfo * makeFunctionDebugInfo ( const Function & fn ); EnumInfo * makeEnumDebugInfo ( const Enumeration & en ); FuncInfo * makeInvokeableTypeDebugInfo ( const TypeDeclPtr & blk, const LineInfo & at ); + AnnotationArgumentInfo * makeAnnotationArguments ( const AnnotationArgumentList & list, uint32_t & count ); + AnnotationInfo * makeAnnotationList ( const AnnotationList & list, uint32_t & count ); void appendLocalVariables ( FuncInfo * info, ExpressionPtr body ); void appendGlobalVariables ( FuncInfo * info, const FunctionPtr & body ); void logMemInfo ( TextWriter & tw ); public: shared_ptr debugInfo; - bool rtti = false; public: das_hash_map smn2s; das_hash_map tmn2t; diff --git a/include/daScript/ast/ast_visitor.h b/include/daScript/ast/ast_visitor.h index 56d27cd58..6603050f6 100644 --- a/include/daScript/ast/ast_visitor.h +++ b/include/daScript/ast/ast_visitor.h @@ -346,7 +346,6 @@ namespace das { ctx.heap = make_unique(); ctx.stringHeap = make_unique(); ctx.category = uint32_t(ContextCategory::folding_context); - helper.rtti = true; } protected: Context ctx; diff --git a/include/daScript/simulate/aot.h b/include/daScript/simulate/aot.h index e676fe4d0..88bd3f767 100644 --- a/include/daScript/simulate/aot.h +++ b/include/daScript/simulate/aot.h @@ -33,8 +33,6 @@ namespace das { #define DAS_SETBOOLAND(a,b) (([&]()->bool{ bool & A=((a)); A=A&&((b)); return A; })()) #define DAS_SETBOOLXOR(a,b) (([&]()->bool{ bool & A=((a)); A=A^((b)); return A; })()) - #define DAS_MAKE_ANNOTATION(name) ((TypeAnnotation*)(intptr_t(name)|1)) - DAS_API void das_debug ( Context * context, TypeInfo * typeInfo, const char * FILE, int LINE, vec4f res, const char * message = nullptr ); __forceinline void das_assert ( bool cond, Context * __context__ ) { diff --git a/include/daScript/simulate/aot_builtin_rtti.h b/include/daScript/simulate/aot_builtin_rtti.h index 8bbc6b2a1..5e30873f0 100644 --- a/include/daScript/simulate/aot_builtin_rtti.h +++ b/include/daScript/simulate/aot_builtin_rtti.h @@ -96,6 +96,14 @@ namespace das { struct AnnotationArgument; DAS_API RttiValue rtti_builtin_argument_value(const AnnotationArgument & info, Context * context, LineInfoArg * at); + DAS_API RttiValue rtti_builtin_argument_info_value(const AnnotationArgumentInfo & info, Context * context, LineInfoArg * at); + + DAS_API const AnnotationInfo & rtti_builtin_struct_annotation ( const StructInfo & info, int32_t index, Context * context, LineInfoArg * at ); + DAS_API const AnnotationInfo & rtti_builtin_func_annotation ( const FuncInfo & info, int32_t index, Context * context, LineInfoArg * at ); + DAS_API const AnnotationInfo & rtti_builtin_enum_annotation ( const EnumInfo & info, int32_t index, Context * context, LineInfoArg * at ); + DAS_API const AnnotationArgumentInfo & rtti_builtin_annotation_argument ( const AnnotationInfo & info, int32_t index, Context * context, LineInfoArg * at ); + DAS_API const AnnotationArgumentInfo & rtti_builtin_var_annotation_argument ( const VarInfo & info, int32_t index, Context * context, LineInfoArg * at ); + DAS_API Annotation * rtti_builtin_resolve_annotation ( const AnnotationInfo & info ); DAS_API int32_t rtti_getDimTypeInfo(const TypeInfo & ti, int32_t index, Context * context, LineInfoArg * at); DAS_API int32_t rtti_getDimVarInfo(const VarInfo & ti, int32_t index, Context * context, LineInfoArg * at); diff --git a/include/daScript/simulate/debug_info.h b/include/daScript/simulate/debug_info.h index 380d7b91c..ad37e1c0e 100644 --- a/include/daScript/simulate/debug_info.h +++ b/include/daScript/simulate/debug_info.h @@ -107,9 +107,45 @@ namespace das }; struct StructInfo; + struct Annotation; struct TypeAnnotation; struct EnumInfo; + // POD mirrors of AST annotation data, deep-copied into the DebugInfoAllocator so that + // debug info never outlives its strings (a Context can outlive its Program). + struct AnnotationArgumentInfo { + Type type; // only tBool, tInt, tFloat, tString + const char * name; + const char * sValue; + union { + bool bValue; + int32_t iValue; + float fValue; + }; + AnnotationArgumentInfo() = default; + AnnotationArgumentInfo ( const char * n, bool b ) + : type(Type::tBool), name(n), sValue(nullptr), bValue(b) {} + AnnotationArgumentInfo ( const char * n, int32_t i ) + : type(Type::tInt), name(n), sValue(nullptr), iValue(i) {} + AnnotationArgumentInfo ( const char * n, float f ) + : type(Type::tFloat), name(n), sValue(nullptr), fValue(f) {} + AnnotationArgumentInfo ( const char * n, const char * s ) + : type(Type::tString), name(n), sValue(s), iValue(0) {} + }; + + struct AnnotationInfo { + const char * name; // annotation name + const char * module_name; // module where the annotation is declared + AnnotationArgumentInfo * arguments; // flat array + uint32_t count; + mutable Annotation * resolved; // lazy environment-lookup cache. WARNING: use Module::resolveAnnotation + AnnotationInfo() = default; + AnnotationInfo ( const char * _name, const char * _module_name, + AnnotationArgumentInfo * _arguments, uint32_t _count ) + : name(_name), module_name(_module_name) + , arguments(_arguments), count(_count), resolved(nullptr) {} + }; + struct BasicAnnotation : gc_node { BasicAnnotation ( const string & n, const string & cpn = "" ) : name(n), cppName(cpn) {} virtual ~BasicAnnotation() {} @@ -336,7 +372,7 @@ namespace das union { StructInfo * structType; EnumInfo * enumType; - mutable TypeAnnotation * annotation_or_name; // WARNING: unresolved. use 'getAnnotation' + AnnotationInfo * annotation_info; // WARNING: unresolved. use 'getAnnotation' }; TypeInfo * firstType; // map from, or array TypeInfo * secondType; // map to @@ -350,13 +386,13 @@ namespace das uint32_t argCount; uint32_t dimSize; TypeInfo() = default; - TypeInfo ( Type _type, StructInfo * _structType, EnumInfo * _enumType, TypeAnnotation * _annotation_or_name, + TypeInfo ( Type _type, StructInfo * _structType, EnumInfo * _enumType, AnnotationInfo * _annotation_info, TypeInfo * _firstType, TypeInfo * _secondType, TypeInfo ** _argTypes, const char ** _argNames, uint32_t _argCount, uint32_t _dimSize, uint32_t * _dim, uint32_t _flags, uint32_t _size, uint64_t _hash ) { type = _type; - if ( _structType ) { structType = _structType; DAS_ASSERT(!_enumType && !_annotation_or_name); } - else if ( _enumType ) { enumType = _enumType; DAS_ASSERT(!_structType && !_annotation_or_name); } - else { annotation_or_name = _annotation_or_name; DAS_ASSERT(!_structType && !_enumType); } + if ( _structType ) { structType = _structType; DAS_ASSERT(!_enumType && !_annotation_info); } + else if ( _enumType ) { enumType = _enumType; DAS_ASSERT(!_structType && !_annotation_info); } + else { annotation_info = _annotation_info; DAS_ASSERT(!_structType && !_enumType); } firstType = _firstType; secondType = _secondType; argTypes = _argTypes; @@ -434,20 +470,24 @@ namespace das char * sValue; }; const char * name; - void * annotation_arguments = nullptr; + AnnotationArgumentInfo * annotation_arguments = nullptr; // flat array + uint32_t annotation_argument_count = 0; uint32_t offset; uint32_t nextGcField; VarInfo() = default; - VarInfo(Type _type, StructInfo * _structType, EnumInfo * _enumType, TypeAnnotation * _annotation_or_name, + VarInfo(Type _type, StructInfo * _structType, EnumInfo * _enumType, AnnotationInfo * _annotation_info, TypeInfo * _firstType, TypeInfo * _secondType, TypeInfo ** _argTypes, const char ** _argNames, uint32_t _argCount, uint32_t _dimSize, uint32_t * _dim, uint32_t _flags, uint32_t _size, - uint64_t _hash, const char * _name, uint32_t _offset, uint32_t _nextGcField ) : - TypeInfo(_type,_structType,_enumType,_annotation_or_name, + uint64_t _hash, const char * _name, uint32_t _offset, uint32_t _nextGcField, + AnnotationArgumentInfo * _annotation_arguments = nullptr, uint32_t _annotation_argument_count = 0 ) : + TypeInfo(_type,_structType,_enumType,_annotation_info, _firstType,_secondType,_argTypes,_argNames,_argCount, _dimSize,_dim,_flags,_size,_hash) { name = _name; offset = _offset; nextGcField = _nextGcField; + annotation_arguments = _annotation_arguments; + annotation_argument_count = _annotation_argument_count; value = v_zero(); } }; @@ -462,17 +502,19 @@ namespace das const char* name; const char* module_name; VarInfo ** fields; - void * annotation_list; + AnnotationInfo * annotations; // flat array uint64_t hash; uint64_t init_mnh; uint32_t flags; uint32_t count; uint32_t size; uint32_t firstGcField; + uint32_t annotation_count; StructInfo() = default; StructInfo( const char * _name, const char * _module_name, uint32_t _flags, VarInfo ** _fields, uint32_t _count, - uint32_t _size, uint64_t _init_mnh, void * _annotation_list, uint64_t _hash, uint32_t _firstGcField ) { + uint32_t _size, uint64_t _init_mnh, AnnotationInfo * _annotations, uint32_t _annotation_count, + uint64_t _hash, uint32_t _firstGcField ) { name = _name; module_name = _module_name; flags = _flags; @@ -480,7 +522,8 @@ namespace das count = _count; size = _size; init_mnh = _init_mnh; - annotation_list = _annotation_list; + annotations = _annotations; + annotation_count = _annotation_count; hash = _hash; firstGcField = _firstGcField; } @@ -501,6 +544,8 @@ namespace das uint32_t count; uint64_t hash; uint32_t flags; + AnnotationInfo * annotations; // flat array + uint32_t annotation_count; }; struct LocalVariableInfo : TypeInfo { @@ -530,15 +575,18 @@ namespace das TypeInfo * result; LocalVariableInfo ** locals; VarInfo ** globals; + AnnotationInfo * annotations; // flat array uint64_t hash; uint32_t flags; uint32_t count; uint32_t stackSize; uint32_t localCount; uint32_t globalCount; + uint32_t annotation_count; FuncInfo() = default; FuncInfo( const char * _name, const char * _cppName, VarInfo ** _fields, uint32_t _count, uint32_t _stackSize, - TypeInfo * _result, LocalVariableInfo ** _locals, uint32_t _localCount, uint64_t _hash, uint32_t _flags ) { + TypeInfo * _result, LocalVariableInfo ** _locals, uint32_t _localCount, uint64_t _hash, uint32_t _flags, + AnnotationInfo * _annotations = nullptr, uint32_t _annotation_count = 0 ) { name = _name; cppName = _cppName; fields = _fields; @@ -551,6 +599,8 @@ namespace das flags = _flags; globals = nullptr; globalCount = 0; + annotations = _annotations; + annotation_count = _annotation_count; } }; diff --git a/modules/dasLLVM/daslib/llvm_jit.das b/modules/dasLLVM/daslib/llvm_jit.das index ac16dfc40..6969cfce6 100644 --- a/modules/dasLLVM/daslib/llvm_jit.das +++ b/modules/dasLLVM/daslib/llvm_jit.das @@ -42,21 +42,6 @@ var active_filenames : table // on the first hit and needs no size guard. json_find -35%. Flip + bump CODEGEN_VERSION to re-A/B. let JIT_PACKED_FIND_STR_EARLY_OUT = true -// VarInfo annotation type on c++ side is `vector`. -// They cannot be allocated as global variable and require -// Memory managment - ctors/dtors, same as FileInfo. -struct AnnotationVarInfoEntry { - varinfo : LLVMOpaqueValue? - ann_global : LLVMOpaqueValue? - count : int -} -var annotation_varinfo_list : array - -// (parent_global, resolvent_str_global) pairs for tHandle typeinfo slot 0. -// Static initializer leaves slot 0 null; runtime ctor writes the tagged pointer -// because wasm32 forbids widening ptrtoint inside .data initializers. -var handle_typeinfo_pending : array> - [macro_function] def private get_expr_ptr(expr : ExpressionPtr) { return expr @@ -198,97 +183,6 @@ def generate_fileinfo_ctor_dtor(types : PrimitiveTypes; names : table; j return (constructor, destructor) } -[macro_function] -def generate_annotations_ctor_dtor(types : PrimitiveTypes; var entries : array; jit_mode : bool) { - var ctor_builder = LLVMCreateBuilderInContext(g_ctx) - var dtor_builder = LLVMCreateBuilderInContext(g_ctx) - defer() { - LLVMDisposeBuilder(ctor_builder) - LLVMDisposeBuilder(dtor_builder) - } - let functionType = LLVMFunctionType(types.t_void, null, 0u, 0) - let constructor = LLVMAddFunctionWithType(g_mod, "annotations_constructor", functionType) - let destructor = LLVMAddFunctionWithType(g_mod, "annotations_destructor", functionType) - if (!jit_mode) { - set_private_linkage(constructor) - set_private_linkage(destructor) - } else { - set_public_linkage(constructor) - set_public_linkage(destructor) - } - let entry_ctor = LLVMAppendBasicBlockInContext(g_ctx, constructor, "entry_ctor") - LLVMPositionBuilderAtEnd(ctor_builder, entry_ctor) - let entry_dtor = LLVMAppendBasicBlockInContext(g_ctx, destructor, "entry_dtor") - LLVMPositionBuilderAtEnd(dtor_builder, entry_dtor) - let void_ptr = types.LLVMVoidPtrType() - var initFuncType = LLVMFunctionType(types.t_void, - fixed_array(void_ptr, types.t_int32, void_ptr)) - var freeFuncType = LLVMFunctionType(types.t_void, - fixed_array(void_ptr)) - var initFunc = LLVMAddFunctionWithType(g_mod, "jit_initialize_varinfo_annotations", initFuncType) - LLVMAddGlobalMapping(g_engine, initFunc, get_initialize_varinfo_annotations()) - var freeFunc = LLVMAddFunctionWithType(g_mod, "jit_free_varinfo_annotations", freeFuncType) - LLVMAddGlobalMapping(g_engine, freeFunc, get_free_varinfo_annotations()) - for (entry in entries) { - var ctorArgs = array( - ctor_builder |> LLVMBuildPointerCast(entry.varinfo, void_ptr, ""), - types.ConstI32(entry.count |> uint64()), - ctor_builder |> LLVMBuildPointerCast(entry.ann_global, void_ptr, "")) - LLVMBuildCall2(ctor_builder, initFuncType, initFunc, ctorArgs, "") - var dtorArgs = array( - dtor_builder |> LLVMBuildPointerCast(entry.varinfo, void_ptr, "")) - LLVMBuildCall2(dtor_builder, freeFuncType, freeFunc, dtorArgs, "") - } - LLVMBuildRetVoid(ctor_builder) - LLVMBuildRetVoid(dtor_builder) - return (constructor, destructor) -} - -[macro_function] -def generate_handle_typeinfo_ctor_dtor(types : PrimitiveTypes; entries : array>; jit_mode : bool) { - // Slot 0 (annotation_or_name) of every tHandle TypeInfo holds - // ((TypeAnnotation*)(intptr_t(annName)|1)) - // wasm32 forbids widening ptrtoint inside .data initializers (the cast - // must run post-relocation), so the tagged pointer is materialized at - // runtime by this constructor function. globalopt would otherwise fold - // the ctor-only store back into the static initializer — the parent - // global is marked externally_initialized to block that. - var ctor_builder = LLVMCreateBuilderInContext(g_ctx) - defer() { - LLVMDisposeBuilder(ctor_builder) - } - let functionType = LLVMFunctionType(types.t_void, null, 0u, 0) - let constructor = LLVMAddFunctionWithType(g_mod, "handle_typeinfo_constructor", functionType) - if (!jit_mode) { - set_private_linkage(constructor) - } else { - set_public_linkage(constructor) - } - let entry_ctor = LLVMAppendBasicBlockInContext(g_ctx, constructor, "entry_ctor") - LLVMPositionBuilderAtEnd(ctor_builder, entry_ctor) - - let void_ptr = types.LLVMVoidPtrType() - let void_ptr_ptr = LLVMPointerType(void_ptr, 0u) - let one = LLVMConstInt(types.t_int64, 1 |> uint64(), 0) - for ((parent_global, resolvent_str) in entries) { - // tagged = (i8*)(ptrtoint(resolvent) + 1) - // Mark parent as externally_initialized: globalopt would otherwise - // fold ctor-only stores back into the static initializer, which - // re-introduces the ptrtoint ConstantExpr that wasm32 cannot lower - // in .data. The attribute is LLVM's intended signal for - // "runtime-initialized by external code, don't fold". - LLVMSetExternallyInitialized(parent_global, 1) - let nameInt = LLVMBuildPtrToInt(ctor_builder, resolvent_str, types.t_int64, "nameInt") - let taggedInt = LLVMBuildAdd(ctor_builder, nameInt, one, "taggedInt") - let taggedPtr = LLVMBuildIntToPtr(ctor_builder, taggedInt, void_ptr, "taggedPtr") - // slot 0 is at offset 0 — alias parent_global as i8** and store. - let slot0 = LLVMBuildBitCast(ctor_builder, parent_global, void_ptr_ptr, "slot0") - LLVMBuildStore(ctor_builder, taggedPtr, slot0) - } - LLVMBuildRetVoid(ctor_builder) - return (constructor, unsafe(reinterpret(null))) -} - [macro_function] def get_function_addr(var ctx : Context?; func : Function?) { let mangled_name = get_mangled_name(func) @@ -4437,7 +4331,7 @@ class public LlvmJitVisitor : AstVisitor { def get_llvm_type_for_typeinfo() : LLVMTypeRef { let void_ptr = types.LLVMVoidPtrType() return StructType(types) <| fixed_array( - void_ptr, // union StructInfo *, EnumInfo *, TypeAnnotation * + void_ptr, // union StructInfo *, EnumInfo *, AnnotationInfo * void_ptr, // TypeInfo * firstType void_ptr, // TypeInfo * secondType void_ptr, // TypeInfo ** argTypes @@ -4456,7 +4350,7 @@ class public LlvmJitVisitor : AstVisitor { let void_ptr = types.LLVMVoidPtrType() return StructType(types) <| fixed_array( // Typeinfo fields - void_ptr, // union StructInfo *, EnumInfo *, TypeAnnotation * + void_ptr, // union StructInfo *, EnumInfo *, AnnotationInfo * void_ptr, // TypeInfo * firstType void_ptr, // TypeInfo * secondType void_ptr, // TypeInfo ** argTypes @@ -4472,7 +4366,8 @@ class public LlvmJitVisitor : AstVisitor { // Own fields types.LLVMFloat4Type(), // union { vec4f value; char * sValue } void_ptr, // const char * name - void_ptr, // void * annotation_arguments + void_ptr, // AnnotationArgumentInfo * annotation_arguments + types.t_int32, // uint32_t annotation_argument_count types.t_int32, // uint32_t offset types.t_int32 // uint32_t nextGcField ) @@ -4484,7 +4379,7 @@ class public LlvmJitVisitor : AstVisitor { void_ptr, // const char* name void_ptr, // const char* module_name void_ptr, // VarInfo ** fields - void_ptr, // void * annotation_list + void_ptr, // AnnotationInfo * annotations types.t_int64, // uint64_t hash types.t_int64, // uint64_t init_mnh @@ -4492,7 +4387,8 @@ class public LlvmJitVisitor : AstVisitor { types.t_int32, // uint32_t flags types.t_int32, // uint32_t count types.t_int32, // uint32_t size - types.t_int32) + types.t_int32, // uint32_t firstGcField + types.t_int32) // uint32_t annotation_count } def get_llvm_type_for_enumvalueinfo() : LLVMTypeRef { @@ -4502,15 +4398,25 @@ class public LlvmJitVisitor : AstVisitor { types.t_int64) } - def get_llvm_type_for_annotation_arg_pod() : LLVMTypeRef { + def get_llvm_type_for_annotation_argument_info() : LLVMTypeRef { let void_ptr = types.LLVMVoidPtrType() return StructType(types) <| fixed_array( + types.t_int32, // Type type void_ptr, // const char * name void_ptr, // const char * sValue - types.t_int32, // int32_t type types.t_int32) // int32_t iValue (covers bool/int/float union) } + def get_llvm_type_for_annotation_info() : LLVMTypeRef { + let void_ptr = types.LLVMVoidPtrType() + return StructType(types) <| fixed_array( + void_ptr, // const char * name + void_ptr, // const char * module_name + void_ptr, // AnnotationArgumentInfo * arguments + types.t_int32, // uint32_t count + void_ptr) // Annotation * resolved (lazy cache, starts null) + } + def get_llvm_type_for_enuminfo(len : uint) : LLVMTypeRef { let void_ptr = types.LLVMVoidPtrType() return StructType(types) <| fixed_array( @@ -4519,7 +4425,9 @@ class public LlvmJitVisitor : AstVisitor { void_ptr, // EnumValueInfo ** fields types.t_int32, // uint32_t count types.t_int64, // uint64_t hash - types.t_int32) // uint32_t flags (EnumInfo::flag_unsigned for uint*-backed enums) + types.t_int32, // uint32_t flags (EnumInfo::flag_unsigned for uint*-backed enums) + void_ptr, // AnnotationInfo * annotations + types.t_int32) // uint32_t annotation_count } ///////////////////////////////////////////////////////////////////////////////////////// @@ -4569,22 +4477,86 @@ class public LlvmJitVisitor : AstVisitor { return <- init_values } + def create_annotation_argument_info(arg : AnnotationArgumentInfo) : LLVMValueRef { + let ann_ty = get_llvm_type_for_annotation_argument_info() + var fields = fixed_array( + types.ConstI32(uint64(int(arg.basicType))), + get_string_constant_ptr(g_builder, "{arg.name}"), + arg.basicType == Type.tString ? get_string_constant_ptr(g_builder, "{arg.sValue}") : LLVMConstPointerNull(types.LLVMVoidPtrType()), + types.ConstI32(uint64(uint(arg.iValue)))) + return LLVMConstNamedStruct(ann_ty, array_data_ptr(fields), 4u) + } + + // (arguments_ptr, count) — emits an AnnotationArgumentInfo[] global, or (null, 0) + def create_annotation_arguments_global(vi : VarInfo?) : tuple { + if (vi.annotation_argument_count == 0u) { + return (LLVMConstPointerNull(types.LLVMVoidPtrType()), 0u) + } + let ann_ty = get_llvm_type_for_annotation_argument_info() + var ann_vals : array + ann_vals |> reserve(int(vi.annotation_argument_count)) + for (i in range(int(vi.annotation_argument_count))) { + unsafe { + ann_vals |> push <| create_annotation_argument_info(get_annotation_argument(*vi, i)) + } + } + let n = vi.annotation_argument_count + let ann_global = LLVMAddGlobal(g_mod, LLVMArrayType(ann_ty, n), "varinfo_annotations") + set_globalvar_linkage(ann_global) + ann_global |> LLVMSetInitializer <| LLVMConstArray(ann_ty, array_data_ptr(ann_vals), n) + return (g_builder |> LLVMBuildPointerCast(ann_global, types.LLVMVoidPtrType(), ""), n) + } + + // one AnnotationInfo constant, including its (optionally emitted) argument array global + def create_annotation_info_value(ann : AnnotationInfo) : LLVMValueRef { + let info_ty = get_llvm_type_for_annotation_info() + let arg_ty = get_llvm_type_for_annotation_argument_info() + var args_ptr = LLVMConstPointerNull(types.LLVMVoidPtrType()) + if (ann.count > 0u) { + var arg_vals : array + arg_vals |> reserve(int(ann.count)) + for (gi in range(int(ann.count))) { + arg_vals |> push <| create_annotation_argument_info(get_annotation_argument(ann, gi)) + } + let args_global = LLVMAddGlobal(g_mod, LLVMArrayType(arg_ty, ann.count), "annotation_info_args") + set_globalvar_linkage(args_global) + args_global |> LLVMSetInitializer <| LLVMConstArray(arg_ty, array_data_ptr(arg_vals), ann.count) + args_ptr = g_builder |> LLVMBuildPointerCast(args_global, types.LLVMVoidPtrType(), "") + } + var fields = fixed_array( + get_string_constant_ptr(g_builder, "{ann.name}"), + get_string_constant_ptr(g_builder, "{ann.module_name}"), + args_ptr, + types.ConstI32(ann.count |> uint64()), + LLVMConstPointerNull(types.LLVMVoidPtrType())) + return LLVMConstNamedStruct(info_ty, array_data_ptr(fields), 5u) + } + + // (annotations_ptr, count) — packs AnnotationInfo constants into an array global, or (null, 0) + def pack_annotation_list(var ann_vals : array) : tuple { + if (ann_vals |> empty()) { + return (LLVMConstPointerNull(types.LLVMVoidPtrType()), 0u) + } + let info_ty = get_llvm_type_for_annotation_info() + let n = ann_vals |> length() |> uint() + let list_global = LLVMAddGlobal(g_mod, LLVMArrayType(info_ty, n), "annotation_info_list") + set_globalvar_linkage(list_global) + list_global |> LLVMSetInitializer <| LLVMConstArray(info_ty, array_data_ptr(ann_vals), n) + return (g_builder |> LLVMBuildPointerCast(list_global, types.LLVMVoidPtrType(), ""), n) + } + def get_value_for_varinfo(vi : VarInfo?) : LLVMValueRef { - // get_llvm_type_for_typeinfo(); // VarInfo : TypeInfo - // types.LLVMFloat4Type(); // union { vec4f value; char * sValue } - // void_ptr; // const char * name - // void_ptr; // void * annotation_arguments - // types.t_int32; // uint32_t offset - // types.t_int32 // uint32_t nextGcField var init_values <- get_typeinfo_fields(unsafe(reinterpret vi)) var x = types.ConstReal(0f |> double()) var y = types.ConstReal(0f |> double()) var z = types.ConstReal(0f |> double()) var w = types.ConstReal(0f |> double()) + var (ann_ptr, ann_count) = create_annotation_arguments_global(vi) init_values |> push <| LLVMBuildFloat4_xyzw(g_builder, types, x, y, z, w, "float4") init_values |> push <| get_string_constant_ptr(g_builder, vi.name) - init_values |> push <| LLVMConstPointerNull(types.LLVMVoidPtrType()) + init_values |> push <| ann_ptr + init_values |> push <| types.ConstI32(ann_count |> uint64()) init_values |> push <| types.ConstI32(vi.offset |> uint64()) init_values |> push <| types.ConstI32(vi.nextGcField |> uint64()) let ty = get_llvm_type_for_varinfo() @@ -4601,30 +4573,7 @@ class public LlvmJitVisitor : AstVisitor { set_globalvar_linkage(varinfo) let vi = unsafe(ei[i]) varinfo |> LLVMSetInitializer <| get_value_for_varinfo(vi) - queue_handle_typeinfo_if_needed(unsafe(reinterpret vi), varinfo) array_init |> push <| unsafe(reinterpret(varinfo)) - if (vi.annotation_arguments != null) { - let ann_ty = get_llvm_type_for_annotation_arg_pod() - var ann_vals : array - for (arg in *vi.annotation_arguments) { - var fields = fixed_array( - get_string_constant_ptr(g_builder, string(arg.name)), - get_string_constant_ptr(g_builder, string(arg.sValue)), - types.ConstI32(uint64(int(arg.basicType))), - types.ConstI32(uint64(uint(arg.iValue)))) - ann_vals |> push <| LLVMConstNamedStruct(ann_ty, array_data_ptr(fields), 4u) // nolint:PERF006 annotation count unknown - } - if (!(ann_vals |> empty())) { - let n = ann_vals |> length() |> uint() - let ann_global = LLVMAddGlobal(g_mod, LLVMArrayType(ann_ty, n), "varinfo_annotations") - set_globalvar_linkage(ann_global) - ann_global |> LLVMSetInitializer <| LLVMConstArray(ann_ty, array_data_ptr(ann_vals), n) - annotation_varinfo_list |> push <| AnnotationVarInfoEntry( - varinfo = varinfo, - ann_global = ann_global, - count = int(n)) - } - } } let my_struct_array_type = LLVMArrayType(LLVMPointerType(ty, 0u), len) let varinfo_array = LLVMAddGlobal(g_mod, my_struct_array_type, "varinfo_ptr_array") @@ -4641,13 +4590,22 @@ class public LlvmJitVisitor : AstVisitor { var struct_info_global = LLVMAddGlobal(g_mod, ty, "enuminfo_{ei.module_name}_{ei.name}") set_globalvar_linkage(struct_info_global) enuminfo_cache[ei] = struct_info_global + var ann_vals : array + for (i in range(int(ei.annotation_count))) { + unsafe { + ann_vals |> push <| create_annotation_info_value(get_annotation(*ei, i)) // nolint:PERF006 count known small + } + } + var (ann_ptr, ann_count) = pack_annotation_list(ann_vals) var init_values <- [ get_string_constant_ptr(g_builder, ei.name), get_string_constant_ptr(g_builder, ei.module_name), create_enumvalueinfo_global_array(ei.fields, ei.count), types.ConstI32(ei.count |> uint64()), LLVMConstInt(types.t_int64, ei.hash, 0), - types.ConstI32(ei.flags |> uint64()) + types.ConstI32(ei.flags |> uint64()), + ann_ptr, + types.ConstI32(ann_count |> uint64()) ] struct_info_global |> LLVMSetInitializer() <| make_initializer(ty, init_values) return struct_info_global @@ -4659,12 +4617,18 @@ class public LlvmJitVisitor : AstVisitor { var struct_info_global = LLVMAddGlobal(g_mod, ty, "si_{si.name}") set_globalvar_linkage(struct_info_global) structinfo_cache[si] = struct_info_global - var void_ptr = types.LLVMVoidPtrType() + var ann_vals : array + for (i in range(int(si.annotation_count))) { + unsafe { + ann_vals |> push <| create_annotation_info_value(get_annotation(*si, i)) // nolint:PERF006 count known small + } + } + var (ann_ptr, ann_count) = pack_annotation_list(ann_vals) var init_values <- [ get_string_constant_ptr(g_builder, si.name), get_string_constant_ptr(g_builder, si.module_name), create_struct_fields_global_array(si.fields, si.count), - LLVMConstPointerNull(void_ptr), // NOTE: annotations are not used right now + ann_ptr, LLVMConstInt(types.t_int64, si.hash, 0), LLVMConstInt(types.t_int64, si.init_mnh, 0), @@ -4672,7 +4636,8 @@ class public LlvmJitVisitor : AstVisitor { types.ConstI32(si.flags |> uint64()), types.ConstI32(si.count |> uint64()), types.ConstI32(si.size |> uint64()), - types.ConstI32(si.firstGcField |> uint64())] + types.ConstI32(si.firstGcField |> uint64()), + types.ConstI32(ann_count |> uint64())] struct_info_global |> LLVMSetInitializer() <| make_initializer(ty, init_values) return struct_info_global } @@ -4720,15 +4685,26 @@ class public LlvmJitVisitor : AstVisitor { return global } - // For tHandle: emits resolvent string global + queues (parent, resolvent) - // for runtime slot-0 store. Parent's slot 0 stays null in the static - // initializer — wasm32 forbids widening ptrtoint inside .data initializers. - def queue_handle_typeinfo_if_needed(info : TypeInfo?; parent_global : LLVMValueRef) { - if (info == null || info.basicType != Type.tHandle) return - let resolvent = "~{info.annotation._module.name}::{info.annotation.name}" - var llvm_resolvent = g_builder |> get_string_constant_ptr(resolvent) - LLVMSetAlignment(llvm_resolvent, 2u) - handle_typeinfo_pending |> push((parent_global, llvm_resolvent)) + // For tHandle: dedup'd static AnnotationInfo { name, module_name, null, 0, null }. + // Plain pointer-to-global in slot 0 — no tagged pointer, no runtime ctor + // (the old ptrtoint|1 scheme needed one because wasm32 forbids widening + // ptrtoint inside .data initializers). + def create_handled_annotation_global(info : TypeInfo?) : LLVMValueRef { + let key = "{info.annotation._module.name}::{info.annotation.name}" + if (handled_ann_cache |> key_exists(key)) return handled_ann_cache[key] + let ty = get_llvm_type_for_annotation_info() + var fields = fixed_array( + get_string_constant_ptr(g_builder, string(info.annotation.name)), + get_string_constant_ptr(g_builder, string(info.annotation._module.name)), + LLVMConstPointerNull(types.LLVMVoidPtrType()), + types.ConstI32(0ul), + LLVMConstPointerNull(types.LLVMVoidPtrType())) + let ann_global = LLVMAddGlobal(g_mod, ty, "handled_annotation_info") + set_globalvar_linkage(ann_global) + ann_global |> LLVMSetInitializer <| LLVMConstNamedStruct(ty, array_data_ptr(fields), 5u) + let res = g_builder |> LLVMBuildPointerCast(ann_global, types.LLVMVoidPtrType(), "") + handled_ann_cache[key] = res + return res } def rec_create_type_info_global(info : TypeInfo?) { @@ -4738,8 +4714,7 @@ class public LlvmJitVisitor : AstVisitor { initValues[i] = LLVMConstPointerNull(types.LLVMVoidPtrType()) } if (info.basicType == Type.tHandle) { - // slot 0 stays null — parent-site queue_handle_typeinfo_if_needed - // arranged the runtime store. + initValues[0] = create_handled_annotation_global(info) } elif (info.basicType == Type.tStructure) { var t = create_struct_global_pointer(info.structType) initValues[0] = t @@ -4798,7 +4773,6 @@ class public LlvmJitVisitor : AstVisitor { typeinfo_cache[ti] = type_info_global - queue_handle_typeinfo_if_needed(ti, type_info_global) let init_values <- get_typeinfo_fields(ti) var type_info_init = LLVMGetUndef(type_info_type) for (i in urange(init_values |> length())) { @@ -4816,6 +4790,7 @@ class public LlvmJitVisitor : AstVisitor { typeinfo_cache : table enuminfo_cache : table varinfo_cache : table + handled_ann_cache : table // "module::name" -> AnnotationInfo global // ExprDebug def override visitExprDebug(expr : ExprDebug?) : ExpressionPtr { @@ -7409,8 +7384,7 @@ def public generate_llvm_code(var dll : DLLHandle?; fmna : DllName; dll_enabled } // During JIT compilation we create some complex -// global types (fileInfo, varAnnotations), -// that can't be const initialized. +// global types (fileInfo), that can't be const initialized. // This function creates single global constructor // that will be called at the startup of compiled // object file. @@ -7418,25 +7392,20 @@ def public generate_llvm_code(var dll : DLLHandle?; fmna : DllName; dll_enabled def public generate_global_ctors_dtors(types : PrimitiveTypes?; jit_mode : bool) { let ctor_dtor <- [ // FileInfo - generate_fileinfo_ctor_dtor(*types, active_filenames, jit_mode), - // VarInfo - generate_annotations_ctor_dtor(*types, annotation_varinfo_list, jit_mode), - generate_handle_typeinfo_ctor_dtor(*types, handle_typeinfo_pending, jit_mode) + generate_fileinfo_ctor_dtor(*types, active_filenames, jit_mode) ] ctor_dtor |> add_ctor_dtor_function(types) reset_codegen_accumulators() } -// Clears the per-codegen-run accumulators (filenames / varinfo / handle typeinfo). +// Clears the per-codegen-run accumulators (filenames). // Normally drained at the tail of generate_global_ctors_dtors; also called from // the JIT failure path (reset_jit_globals_after_failure) so a file that panics // mid-codegen doesn't leave stale fileinfo entries that crash the next file's // ctor generation with assert(variable != null). def public reset_codegen_accumulators() { active_filenames = table() - annotation_varinfo_list |> clear() - handle_typeinfo_pending |> clear() } [macro_function] diff --git a/modules/dasLLVM/daslib/llvm_jit_run.das b/modules/dasLLVM/daslib/llvm_jit_run.das index fd589d470..83d48a8aa 100644 --- a/modules/dasLLVM/daslib/llvm_jit_run.das +++ b/modules/dasLLVM/daslib/llvm_jit_run.das @@ -33,7 +33,7 @@ var LINK_WHOLE_LIB = false // when true, standalone exe links against the whole // invalidates cached DLLs (e.g. edits to llvm_jit.das, llvm_macro.das, llvm_jit_common.das, // runtime helper ABI, default target triple). Cache filenames fold this in, so a bump // makes every previously written DLL miss the cache on the next run and get GC'd. -let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x25ul +let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x26ul let JIT_FNV_PRIME : uint64 = 1099511628211ul diff --git a/skills/cpp_integration.md b/skills/cpp_integration.md index 31983bf25..123bc50e9 100644 --- a/skills/cpp_integration.md +++ b/skills/cpp_integration.md @@ -289,10 +289,10 @@ Where `vec4f` as an `ArgType` means "any type" — the argument accepts any dasl - `vec4f` argument type = "any" — accept arguments of any daslang type - Used internally for `sprint`, `hash`, `write`, `binary_save/load`, `invoke_in_context` -**TypeInfo union warning**: `TypeInfo` has a union — `structType`, `enumType`, and `annotation_or_name` share the same memory. Which member is valid depends on `ti->type`: +**TypeInfo union warning**: `TypeInfo` has a union — `structType`, `enumType`, and `annotation_info` share the same memory. Which member is valid depends on `ti->type`: - `tStructure` → `ti->structType` (StructInfo *) - `tEnumeration` / `tEnumeration8` / `tEnumeration16` → `ti->enumType` (EnumInfo *) -- `tHandle` → use `ti->getAnnotation()` (resolves tagged pointer safely) +- `tHandle` → use `ti->getAnnotation()` (resolves the name-based `AnnotationInfo` to the live `TypeAnnotation` and caches) Accessing the wrong union member is **undefined behavior**. `das_to_string(Type::tHandle)` returns an empty string — use `ti->getAnnotation()->name` for handled type names. diff --git a/src/ast/ast_debug_info_helper.cpp b/src/ast/ast_debug_info_helper.cpp index 5def3ca6d..2a30ac21d 100644 --- a/src/ast/ast_debug_info_helper.cpp +++ b/src/ast/ast_debug_info_helper.cpp @@ -66,6 +66,41 @@ namespace das { } } + AnnotationArgumentInfo * DebugInfoHelper::makeAnnotationArguments ( const AnnotationArgumentList & list, uint32_t & count ) { + count = 0; + if ( list.empty() ) return nullptr; + auto args = (AnnotationArgumentInfo *) debugInfo->allocate(uint32_t(sizeof(AnnotationArgumentInfo) * list.size())); + for ( const auto & arg : list ) { + switch ( arg.type ) { + case Type::tBool: case Type::tInt: case Type::tFloat: case Type::tString: + break; + default: + continue; // nested aList args only exist during parsing + } + auto & ai = args[count++]; + ai.type = arg.type; + ai.name = debugInfo->allocateCachedName(arg.name); + ai.sValue = arg.type==Type::tString ? debugInfo->allocateCachedName(arg.sValue) : nullptr; + ai.iValue = arg.iValue; // raw union copy covers bool/int/float + } + return count ? args : nullptr; + } + + AnnotationInfo * DebugInfoHelper::makeAnnotationList ( const AnnotationList & list, uint32_t & count ) { + count = 0; + if ( list.empty() ) return nullptr; + auto anns = (AnnotationInfo *) debugInfo->allocate(uint32_t(sizeof(AnnotationInfo) * list.size())); + for ( const auto & decl : list ) { + if ( !decl->annotation ) continue; + auto & ai = anns[count++]; + ai.name = debugInfo->allocateCachedName(decl->annotation->name); + ai.module_name = debugInfo->allocateCachedName(decl->annotation->module ? decl->annotation->module->name : ""); + ai.arguments = makeAnnotationArguments(decl->arguments, ai.count); + ai.resolved = nullptr; + } + return count ? anns : nullptr; + } + EnumInfo * DebugInfoHelper::makeEnumDebugInfo ( const Enumeration & en ) { auto mangledName = en.getMangledName(); auto it = emn2e.find(mangledName); @@ -87,6 +122,7 @@ namespace das { eni->fields[i]->value = !ev.value ? -1 : getConstExprIntOrUInt(ev.value); i ++; } + eni->annotations = makeAnnotationList(en.annotations, eni->annotation_count); eni->hash = hash_blockz64((uint8_t *)mangledName.c_str()); emn2e[mangledName] = eni; return eni; @@ -98,7 +134,7 @@ namespace das { if ( it!=fmn2f.end() ) return it->second; FuncInfo * fni = debugInfo->makeNode(); fni->name = debugInfo->allocateCachedName(fn.name); - if ( rtti && fn.builtIn ) { + if ( fn.builtIn ) { auto bfn = (BuiltInFunction *) &fn; fni->cppName = debugInfo->allocateCachedName(bfn->cppName); } else { @@ -124,6 +160,7 @@ namespace das { fni->result = makeTypeInfo(nullptr, fn.result); fni->locals = nullptr; fni->localCount = 0; + fni->annotations = makeAnnotationList(fn.annotations, fni->annotation_count); fni->hash = hash_blockz64((uint8_t *)mangledName.c_str()); fmn2f[mangledName] = fni; return fni; @@ -198,11 +235,7 @@ namespace das { sti->init_mnh = fn->getMangledNameHash(); } } - if ( rtti ) { - sti->annotation_list = (void *) &st.annotations; - } else { - sti->annotation_list = nullptr; - } + sti->annotations = makeAnnotationList(st.annotations, sti->annotation_count); sti->hash = hash_blockz64((uint8_t *)mangledName.c_str()); return sti; } @@ -238,14 +271,15 @@ namespace das { } else if ( elemType->isEnumT() ) { info->enumType = elemType->enumType ? makeEnumDebugInfo(*elemType->enumType) : nullptr; } else if ( elemType->annotation ) { -#if DAS_THREAD_SAFE_ANNOTATIONS - auto annName = debugInfo->allocateCachedName("~" + elemType->annotation->module->name + "::" + elemType->annotation->name); - info->annotation_or_name = ((TypeAnnotation*)(intptr_t(annName)|1)); -#else - info->annotation_or_name = elemType->annotation; -#endif + auto ann = (AnnotationInfo *) debugInfo->allocate(sizeof(AnnotationInfo)); + ann->name = debugInfo->allocateCachedName(elemType->annotation->name); + ann->module_name = debugInfo->allocateCachedName(elemType->annotation->module ? elemType->annotation->module->name : ""); + ann->arguments = nullptr; + ann->count = 0; + ann->resolved = nullptr; + info->annotation_info = ann; } else { - info->annotation_or_name = nullptr; + info->annotation_info = nullptr; } info->flags = 0; if (type->ref) { @@ -325,12 +359,8 @@ namespace das { makeTypeInfo(vi, var.type); vi->name = debugInfo->allocateCachedName(var.name); vi->offset = var.offset; - if ( rtti && !var.annotation.empty() ) { - vi->annotation_arguments = (void *) &var.annotation; - } else { - vi->annotation_arguments = nullptr; - } - if ( rtti && var.init && var.init->constexpression ) { + vi->annotation_arguments = makeAnnotationArguments(var.annotation, vi->annotation_argument_count); + if ( var.init && var.init->constexpression ) { if ( var.init->rtti_isStringConstant() ) { auto sval = static_cast(var.init); vi->sValue = debugInfo->allocateCachedName(sval->text); @@ -357,7 +387,8 @@ namespace das { makeTypeInfo(vi, var.type); vi->name = debugInfo->allocateCachedName(var.name); vi->offset = 0; - if ( rtti && var.init && var.init->constexpression ) { + vi->annotation_arguments = makeAnnotationArguments(var.annotation, vi->annotation_argument_count); + if ( var.init && var.init->constexpression ) { if ( var.init->rtti_isStringConstant() ) { auto sval = static_cast(var.init); vi->sValue = debugInfo->allocateCachedName(sval->text); diff --git a/src/ast/ast_module.cpp b/src/ast/ast_module.cpp index aaedb422e..739369a02 100644 --- a/src/ast/ast_module.cpp +++ b/src/ast/ast_module.cpp @@ -119,34 +119,26 @@ namespace das { if ( info->type != Type::tHandle ) { return nullptr; } - intptr_t ann = (intptr_t) (info->annotation_or_name); - if ( ann & 1 ) { - auto bound = daScriptEnvironment::getBound(); - DAS_VERIFYF(bound && bound->modules,"missing bound environment"); - // we add ~ at the beginning of the name for padding - // if name is allocated by the compiler, it does not guarantee that it is aligned - // we check if there is a ~ at the beginning of the name, and if it is - we skip it - // that way we can accept both aligned and unaligned names - auto cvtbuf = (char *) ann; - if ( cvtbuf[0]=='~' ) cvtbuf++; - string moduleName, annName; - splitTypeName(cvtbuf, moduleName, annName); - TypeAnnotation * resolve = nullptr; - for ( auto pm = bound->modules; pm!=nullptr; pm=pm->next ) { - if ( pm->name == moduleName ) { - if ( auto annT = pm->findAnnotation(annName) ) { - resolve = (TypeAnnotation *) annT; - } - break; - } - } - if ( bound->g_resolve_annotations ) { - info->annotation_or_name = resolve; + return (TypeAnnotation *) resolveAnnotation(info->annotation_info); + } + + Annotation * Module::resolveAnnotation ( const AnnotationInfo * info ) { + if ( !info ) return nullptr; + if ( info->resolved ) return info->resolved; + auto bound = daScriptEnvironment::getBound(); + DAS_VERIFYF(bound && bound->modules,"missing bound environment"); + Annotation * resolve = nullptr; + for ( auto pm = bound->modules; pm!=nullptr; pm=pm->next ) { + if ( pm->name == info->module_name ) { + resolve = pm->findAnnotation(info->name); + break; } - return resolve; - } else { - return info->annotation_or_name; } + // only cache successful lookups - a miss may be a not-yet-registered module + if ( resolve && bound->g_resolve_annotations ) { + info->resolved = resolve; + } + return resolve; } atomic g_envTotal(0); diff --git a/src/ast/ast_simulate.cpp b/src/ast/ast_simulate.cpp index 759b2c06f..effde64ea 100644 --- a/src/ast/ast_simulate.cpp +++ b/src/ast/ast_simulate.cpp @@ -3623,7 +3623,6 @@ namespace das context.constStringHeap->setInitialSize(globalStringHeapSize); } DebugInfoHelper helper(context.debugInfo); - helper.rtti = options.getBoolOption("rtti",policies.rtti); context.thisHelper = &helper; context.globalVariables = (GlobalVariable *) context.code->allocate( totalVariables*sizeof(GlobalVariable) ); context.globalsSize = 0; diff --git a/src/builtin/module_builtin_rtti.cpp b/src/builtin/module_builtin_rtti.cpp index 1f315de37..029d19143 100644 --- a/src/builtin/module_builtin_rtti.cpp +++ b/src/builtin/module_builtin_rtti.cpp @@ -28,6 +28,8 @@ IMPLEMENT_EXTERNAL_TYPE_FACTORY(VarInfo,VarInfo) IMPLEMENT_EXTERNAL_TYPE_FACTORY(LocalVariableInfo,LocalVariableInfo) IMPLEMENT_EXTERNAL_TYPE_FACTORY(FuncInfo,FuncInfo) IMPLEMENT_EXTERNAL_TYPE_FACTORY(AnnotationArgument,AnnotationArgument) +IMPLEMENT_EXTERNAL_TYPE_FACTORY(AnnotationArgumentInfo,AnnotationArgumentInfo) +IMPLEMENT_EXTERNAL_TYPE_FACTORY(AnnotationInfo,AnnotationInfo) IMPLEMENT_EXTERNAL_TYPE_FACTORY(AnnotationArguments,AnnotationArguments) IMPLEMENT_EXTERNAL_TYPE_FACTORY(AnnotationArgumentList,AnnotationArgumentList) IMPLEMENT_EXTERNAL_TYPE_FACTORY(AnnotationDeclaration,AnnotationDeclaration) @@ -559,6 +561,25 @@ namespace das { } }; + struct AnnotationArgumentInfoAnnotation : ManagedStructureAnnotation { + AnnotationArgumentInfoAnnotation(ModuleLibrary & ml) : ManagedStructureAnnotation ("AnnotationArgumentInfo", ml) { + addFieldEx ( "basicType", "type", offsetof(AnnotationArgumentInfo, type), makeType(ml) ); + addField("name"); + addField("sValue"); + addField("bValue"); + addField("iValue"); + addField("fValue"); + } + }; + + struct AnnotationInfoAnnotation : ManagedStructureAnnotation { + AnnotationInfoAnnotation(ModuleLibrary & ml) : ManagedStructureAnnotation ("AnnotationInfo", ml) { + addField("name"); + addField("module_name"); + addField("count"); + } + }; + TypeDeclPtr makeAnnotationDeclarationFlags() { auto ft = new TypeDecl(Type::tBitfield); ft->alias = "AnnotationDeclarationFlags"; @@ -737,6 +758,7 @@ namespace das { addField("module_name"); addField("hash"); addField("flags"); + addField("annotation_count"); fieldType = makeType(*mlib); fieldType->ref = true; } @@ -767,6 +789,7 @@ namespace das { addField("size"); addField("init_mnh"); addField("hash"); + addField("annotation_count"); } void init () { fieldType = makeType(*mlib); @@ -841,8 +864,7 @@ namespace das { addField("name"); addField("offset"); addField("nextGcField"); - addFieldEx ( "annotation_arguments", "annotation_arguments", - offsetof(VarInfo, annotation_arguments), makeType(ml) ); + addField("annotation_argument_count"); // default values addField("sValue"); addField("value"); @@ -882,6 +904,7 @@ namespace das { addField("flags"); addField("localCount"); addField("globalCount"); + addField("annotation_count"); fieldType = makeType(*mlib); fieldType->ref = true; } @@ -1025,7 +1048,6 @@ namespace das { struct DebugInfoHelperAnnotation : ManagedStructureAnnotation { DebugInfoHelperAnnotation(ModuleLibrary & ml) : ManagedStructureAnnotation ("DebugInfoHelper", ml) { - addField("rtti"); } }; @@ -1133,7 +1155,6 @@ namespace das { void rtti_builtin_module_for_each_enumeration ( Module * module, const TBlock & block, Context * context, LineInfoArg * at ) { DebugInfoHelper helper; - helper.rtti = true; module->enumerations.foreach([&](auto penum){ EnumInfo * info = helper.makeEnumDebugInfo(*penum); vec4f args[1] = { @@ -1155,6 +1176,21 @@ namespace das { return RttiValue{}; } + RttiValue rtti_builtin_argument_info_value(const AnnotationArgumentInfo & info, Context * context, LineInfoArg * at ) { + const auto align = sizeof(vec4f) - sizeof(int32_t); + switch (info.type) { + case Type::tBool: return RttiValue::create(info.bValue, align); + case Type::tInt: return RttiValue::create(info.iValue, align); + case Type::tFloat: return RttiValue::create(info.fValue, align); + case Type::tString: { + const char * sval = info.sValue ? info.sValue : ""; + return RttiValue::create(context->allocateString(sval, uint64_t(strlen(sval)), at), align); + } + default: DAS_ASSERT(false); // I guess unreachable? + } + return RttiValue{}; + } + RttiValue rtti_builtin_variable_value(const VarInfo & info) { RttiValue def {}; @@ -1206,7 +1242,6 @@ namespace das { void rtti_builtin_module_for_each_structure ( Module * module, const TBlock & block, Context * context, LineInfoArg * at ) { DebugInfoHelper helper; - helper.rtti = true; module->structures.foreach([&](auto structPtr){ if ( structPtr->isTemplate ) return; StructInfo * info = helper.makeStructureDebugInfo(*structPtr); @@ -1217,22 +1252,63 @@ namespace das { }); } + // deprecated shim: reconstructs AST-shaped (Annotation*, AnnotationArgumentList) pairs + // from the POD AnnotationInfo copies for the duration of the block invoke void rtti_builtin_structure_for_each_annotation ( const StructInfo & info, const Block & block, Context * context, LineInfoArg * at ) { - if ( info.annotation_list ) { - auto al = (const AnnotationList *) info.annotation_list; - for ( const auto & adp : *al ) { - vec4f args[2] = { - cast::from(adp->annotation), - cast::from(&adp->arguments) - }; - context->invoke(block, args, nullptr, at); + for ( uint32_t ai=0, ais=info.annotation_count; ai!=ais; ++ai ) { + const auto & adp = info.annotations[ai]; + auto ann = Module::resolveAnnotation(&adp); + if ( !ann ) continue; // annotation module no longer registered + AnnotationArgumentList arguments; + arguments.reserve(adp.count); + for ( uint32_t i=0, is=adp.count; i!=is; ++i ) { + const auto & arg = adp.arguments[i]; + AnnotationArgument a; + a.type = arg.type; + a.name = arg.name; + if ( arg.type==Type::tString ) a.sValue = arg.sValue ? arg.sValue : ""; + a.iValue = arg.iValue; + arguments.push_back(a); } + vec4f args[2] = { + cast::from(ann), + cast::from(&arguments) + }; + context->invoke(block, args, nullptr, at); } } + const AnnotationInfo & rtti_builtin_struct_annotation ( const StructInfo & info, int32_t index, Context * context, LineInfoArg * at ) { + if ( uint32_t(index)>=info.annotation_count ) context->throw_error_at(at, "annotation index out of range, %i of %u", index, info.annotation_count); + return info.annotations[index]; + } + + const AnnotationInfo & rtti_builtin_func_annotation ( const FuncInfo & info, int32_t index, Context * context, LineInfoArg * at ) { + if ( uint32_t(index)>=info.annotation_count ) context->throw_error_at(at, "annotation index out of range, %i of %u", index, info.annotation_count); + return info.annotations[index]; + } + + const AnnotationInfo & rtti_builtin_enum_annotation ( const EnumInfo & info, int32_t index, Context * context, LineInfoArg * at ) { + if ( uint32_t(index)>=info.annotation_count ) context->throw_error_at(at, "annotation index out of range, %i of %u", index, info.annotation_count); + return info.annotations[index]; + } + + const AnnotationArgumentInfo & rtti_builtin_annotation_argument ( const AnnotationInfo & info, int32_t index, Context * context, LineInfoArg * at ) { + if ( uint32_t(index)>=info.count ) context->throw_error_at(at, "annotation argument index out of range, %i of %u", index, info.count); + return info.arguments[index]; + } + + const AnnotationArgumentInfo & rtti_builtin_var_annotation_argument ( const VarInfo & info, int32_t index, Context * context, LineInfoArg * at ) { + if ( uint32_t(index)>=info.annotation_argument_count ) context->throw_error_at(at, "annotation argument index out of range, %i of %u", index, info.annotation_argument_count); + return info.annotation_arguments[index]; + } + + Annotation * rtti_builtin_resolve_annotation ( const AnnotationInfo & info ) { + return Module::resolveAnnotation(&info); + } + void rtti_builtin_module_for_each_function ( Module * module, const TBlock & block, Context * context, LineInfoArg * at ) { DebugInfoHelper helper; - helper.rtti = true; module->functions.foreach([&](auto funcPtr){ if ( funcPtr->isTemplate ) return; FuncInfo * info = helper.makeFunctionDebugInfo(*funcPtr); @@ -1245,7 +1321,6 @@ namespace das { void rtti_builtin_module_for_each_generic ( Module * module, const TBlock & block, Context * context, LineInfoArg * at ) { DebugInfoHelper helper; - helper.rtti = true; module->generics.foreach([&](auto funcPtr){ FuncInfo * info = helper.makeFunctionDebugInfo(*funcPtr); vec4f args[1] = { @@ -1257,7 +1332,6 @@ namespace das { void rtti_builtin_module_for_each_global ( Module * module, const TBlock & block, Context * context, LineInfoArg * at ) { DebugInfoHelper helper; - helper.rtti = true; module->globals.foreach([&](auto var){ VarInfo * info = helper.makeVariableDebugInfo(*var); vec4f args[1] = { @@ -1286,7 +1360,6 @@ namespace das { void rtti_builtin_basic_struct_for_each_field ( const BasicStructureAnnotation & ann, const TBlock & block, Context * context, LineInfoArg * at ) { DebugInfoHelper helper; - helper.rtti = true; for ( auto & it : ann.fields ) { const auto & fld = it.second; TypeInfo * info = helper.makeTypeInfo(nullptr, fld.decl); @@ -1653,6 +1726,8 @@ namespace das { addAnnotation(new AstSerializerAnnotation(lib)); addEnumeration(new EnumerationType()); addAnnotation(new AnnotationArgumentAnnotation(lib)); + addAnnotation(new AnnotationArgumentInfoAnnotation(lib)); + addAnnotation(new AnnotationInfoAnnotation(lib)); addVectorAnnotation(this,lib,"AnnotationArguments"); addVectorAnnotation(this,lib,"AnnotationArgumentList"); addAnnotation(new ProgramAnnotation(lib)); @@ -1771,6 +1846,9 @@ namespace das { addExtern(*this, lib, "get_annotation_argument_value", SideEffects::modifyExternal, "rtti_builtin_argument_value") ->args({"info","context","at"}); + addExtern(*this, lib, "get_annotation_argument_value", + SideEffects::modifyExternal, "rtti_builtin_argument_info_value") + ->args({"info","context","at"}); addExtern(*this, lib, "module_for_each_enumeration", SideEffects::modifyExternal, "rtti_builtin_module_for_each_enumeration") ->args({"module","block","context","line"}); @@ -1788,7 +1866,25 @@ namespace das { ->args({"module","block","context","line"}); addExtern(*this, lib, "rtti_builtin_structure_for_each_annotation", SideEffects::modifyExternal, "rtti_builtin_structure_for_each_annotation") - ->args({"struct","block","context","line"}); + ->args({"struct","block","context","line"})->setDeprecated("use each_annotation() instead"); + addExtern(*this, lib, "get_annotation", + SideEffects::none, "rtti_builtin_struct_annotation") + ->args({"struct","index","context","at"}); + addExtern(*this, lib, "get_annotation", + SideEffects::none, "rtti_builtin_func_annotation") + ->args({"function","index","context","at"}); + addExtern(*this, lib, "get_annotation", + SideEffects::none, "rtti_builtin_enum_annotation") + ->args({"enumeration","index","context","at"}); + addExtern(*this, lib, "get_annotation_argument", + SideEffects::none, "rtti_builtin_annotation_argument") + ->args({"annotation","index","context","at"}); + addExtern(*this, lib, "get_annotation_argument", + SideEffects::none, "rtti_builtin_var_annotation_argument") + ->args({"variable","index","context","at"}); + addExtern(*this, lib, "resolve_annotation", + SideEffects::accessExternal, "rtti_builtin_resolve_annotation") + ->arg("annotation"); addExtern(*this, lib, "basic_struct_for_each_field", SideEffects::invokeAndAccessExternal, "rtti_builtin_basic_struct_for_each_field") ->args({"annotation","block","context","line"}); diff --git a/src/builtin/module_builtin_rtti.h b/src/builtin/module_builtin_rtti.h index ea9bc4d79..b0dd2e755 100644 --- a/src/builtin/module_builtin_rtti.h +++ b/src/builtin/module_builtin_rtti.h @@ -37,6 +37,8 @@ MAKE_EXTERNAL_TYPE_FACTORY(VarInfo, das::VarInfo) MAKE_EXTERNAL_TYPE_FACTORY(LocalVariableInfo, das::LocalVariableInfo) MAKE_EXTERNAL_TYPE_FACTORY(FuncInfo, das::FuncInfo) MAKE_EXTERNAL_TYPE_FACTORY(AnnotationArgument, das::AnnotationArgument) +MAKE_EXTERNAL_TYPE_FACTORY(AnnotationArgumentInfo, das::AnnotationArgumentInfo) +MAKE_EXTERNAL_TYPE_FACTORY(AnnotationInfo, das::AnnotationInfo) MAKE_EXTERNAL_TYPE_FACTORY(AnnotationArguments, das::AnnotationArguments) MAKE_EXTERNAL_TYPE_FACTORY(AnnotationArgumentList, das::AnnotationArgumentList) MAKE_EXTERNAL_TYPE_FACTORY(AnnotationDeclaration, das::AnnotationDeclaration) diff --git a/src/builtin/module_jit.cpp b/src/builtin/module_jit.cpp index 47f02d8e2..1de5915af 100644 --- a/src/builtin/module_jit.cpp +++ b/src/builtin/module_jit.cpp @@ -580,36 +580,6 @@ extern "C" { reinterpret_cast(dummy)->~FileInfo(); } - struct JitAnnotationArgPod { - const char * name; - const char * sValue; - int32_t type; - int32_t iValue; // covers bool/int/float (union bit-cast) - }; - - DAS_API void jit_initialize_varinfo_annotations ( void * varinfo_ptr, int32_t nArgs, JitAnnotationArgPod * args ) { - auto vi = (VarInfo *) varinfo_ptr; - auto aa = new AnnotationArguments(); - aa->reserve(nArgs); - for ( int32_t i = 0; i < nArgs; i++ ) { - AnnotationArgument arg; - arg.type = (Type) args[i].type; - arg.name = args[i].name ? args[i].name : ""; - arg.sValue = args[i].sValue ? args[i].sValue : ""; - arg.iValue = args[i].iValue; - aa->push_back(std::move(arg)); - } - vi->annotation_arguments = aa; - } - - DAS_API void jit_free_varinfo_annotations ( void * varinfo_ptr ) { - auto vi = (VarInfo *) varinfo_ptr; - if ( vi->annotation_arguments ) { - delete (AnnotationArguments *) vi->annotation_arguments; - vi->annotation_arguments = nullptr; - } - } - DAS_API void * jit_ast_typedecl ( uint64_t hash, Context * context, LineInfoArg * at ) { if ( !context->thisProgram ) context->throw_error_at(at, "can't get ast_typeinfo, no program. is 'options rtti' missing?"); auto ti = context->thisProgram->astTypeInfo.find(hash); @@ -654,8 +624,6 @@ extern "C" { void *das_get_jit_debug_line() { return (void *)&jit_debug_line; } void *das_get_jit_initialize_fileinfo () { return (void*)&jit_initialize_fileinfo; } void *das_get_jit_free_fileinfo () { return (void*)&jit_free_fileinfo; } - void *jit_get_initialize_varinfo_annotations () { return (void*)&jit_initialize_varinfo_annotations; } - void *jit_get_free_varinfo_annotations () { return (void*)&jit_free_varinfo_annotations; } void *das_get_jit_ast_typedecl () { return (void*)&jit_ast_typedecl; } template @@ -1235,10 +1203,6 @@ extern "C" { SideEffects::none, "das_get_jit_initialize_fileinfo"); addExtern(*this, lib, "get_jit_free_fileinfo", SideEffects::none, "das_get_jit_free_fileinfo"); - addExtern(*this, lib, "get_initialize_varinfo_annotations", - SideEffects::none, "jit_get_initialize_varinfo_annotations"); - addExtern(*this, lib, "get_free_varinfo_annotations", - SideEffects::none, "jit_get_free_varinfo_annotations"); addExtern(*this, lib, "recreate_fileinfo_name", SideEffects::worstDefault, "das_recreate_fileinfo_name"); addExtern(*this, lib, "load_dynamic_library", diff --git a/src/simulate/json_print.cpp b/src/simulate/json_print.cpp index ca9d5374c..04babc236 100644 --- a/src/simulate/json_print.cpp +++ b/src/simulate/json_print.cpp @@ -36,23 +36,21 @@ namespace das { embed = false; optional = false; string name = vi->name ? vi->name : ""; - if ( vi->annotation_arguments ) { - auto aa = (AnnotationArguments *) vi->annotation_arguments; - for ( auto arg : *aa ) { - if ( arg.name=="enum_as_int" && arg.type==Type::tBool ) { - enumAsInt = arg.bValue; - } else if ( arg.name=="unescape" && arg.type==Type::tBool ) { - unescape = arg.bValue; - } else if ( arg.name=="embed" && arg.type==Type::tBool ) { - embed = arg.bValue; - } else if ( arg.name=="optional" && arg.type==Type::tBool ) { - optional = arg.bValue; - } else if ( arg.name=="rename" ) { - if ( arg.type==Type::tString ) { - name = arg.sValue; - } else if ( arg.type==Type::tBool && !name.empty() && name[0]=='_' ) { - name = name.substr(1); - } + for ( uint32_t ai=0, ais=vi->annotation_argument_count; ai!=ais; ++ai ) { + const auto & arg = vi->annotation_arguments[ai]; + if ( strcmp(arg.name,"enum_as_int")==0 && arg.type==Type::tBool ) { + enumAsInt = arg.bValue; + } else if ( strcmp(arg.name,"unescape")==0 && arg.type==Type::tBool ) { + unescape = arg.bValue; + } else if ( strcmp(arg.name,"embed")==0 && arg.type==Type::tBool ) { + embed = arg.bValue; + } else if ( strcmp(arg.name,"optional")==0 && arg.type==Type::tBool ) { + optional = arg.bValue; + } else if ( strcmp(arg.name,"rename")==0 ) { + if ( arg.type==Type::tString ) { + name = arg.sValue; + } else if ( arg.type==Type::tBool && !name.empty() && name[0]=='_' ) { + name = name.substr(1); } } } diff --git a/src/simulate/json_scan.cpp b/src/simulate/json_scan.cpp index 70b4967a6..eea9d4ee3 100644 --- a/src/simulate/json_scan.cpp +++ b/src/simulate/json_scan.cpp @@ -254,18 +254,16 @@ namespace das { static string getFieldName(VarInfo * vi) { string name = vi->name ? vi->name : ""; - if (vi->annotation_arguments) { - auto aa = (AnnotationArguments *)vi->annotation_arguments; - for (auto & arg : *aa) { - if (arg.name == "rename") { - if (arg.type == Type::tString) { - name = arg.sValue; - } else if (arg.type == Type::tBool && !name.empty() && name[0] == '_') { - // @rename _field — strip leading underscore - name = name.substr(1); - } - break; + for (uint32_t ai = 0, ais = vi->annotation_argument_count; ai != ais; ++ai) { + const auto & arg = vi->annotation_arguments[ai]; + if (strcmp(arg.name, "rename") == 0) { + if (arg.type == Type::tString) { + name = arg.sValue; + } else if (arg.type == Type::tBool && !name.empty() && name[0] == '_') { + // @rename _field — strip leading underscore + name = name.substr(1); } + break; } } return name; diff --git a/tests-cpp/small/test_annotation_lifetime.cpp b/tests-cpp/small/test_annotation_lifetime.cpp new file mode 100644 index 000000000..95485dcfb --- /dev/null +++ b/tests-cpp/small/test_annotation_lifetime.cpp @@ -0,0 +1,41 @@ +// Annotation info is deep-copied into the context debug heap (AnnotationInfo / +// AnnotationArgumentInfo). The Context must be able to reflect struct / field +// annotations after the Program is gone, with no `options rtti` involved — +// the old representation pointed straight into the AST and required keeping +// the ProgramPtr alive for the Context's whole lifetime. + +#include + +#include "daScript/daScript.h" + +using namespace das; + +#define SCRIPT_PATH "/tests-cpp/small/test_annotation_lifetime.das" + +TEST_CASE("annotation reflection survives Program release") { + TextPrinter tout; + ModuleGroup dummyLibGroup; + auto fAccess = make_smart(); + auto program = compileDaScript(getDasRoot() + SCRIPT_PATH, + fAccess, tout, dummyLibGroup); + REQUIRE(program); + REQUIRE_FALSE(program->failed()); + + Context ctx(program->getContextStackSize()); + REQUIRE(program->simulate(ctx, tout)); + // no `options rtti` in the fixture — simulate drops thisProgram + REQUIRE(ctx.thisProgram == nullptr); + + // release the Program before touching reflection — debug info is self-contained + program.reset(); + + auto fnCheck = ctx.findFunction("check"); + REQUIRE(fnCheck); + bool ok = false; + bool ranClean = ctx.runWithCatchAndClear([&](){ + vec4f r = ctx.callOrFastcall(fnCheck, nullptr, nullptr); + ok = cast::to(r); + }); + CHECK_MESSAGE(ranClean, "check() panicked after program release"); + CHECK_MESSAGE(ok, "annotation data missing or wrong after program release"); +} diff --git a/tests-cpp/small/test_annotation_lifetime.das b/tests-cpp/small/test_annotation_lifetime.das new file mode 100644 index 000000000..0f53e83c6 --- /dev/null +++ b/tests-cpp/small/test_annotation_lifetime.das @@ -0,0 +1,50 @@ +// Companion fixture for test_annotation_lifetime.cpp. +// Deliberately NO `options rtti` — annotation reflection must work from the +// context debug heap alone, after the Program is released. + +options gen2 + +require daslib/rtti + +[comment(tag = "alive", level = 3)] +struct Annotated { + @marker = 42 x : int + y : int +} + +[export] +def check : bool { + let s = Annotated() + let sinfo = (typeinfo rtti_typeinfo(s)).structType + if (sinfo == null) { + return false + } + var struct_ok = false + for (ann in each_annotation(*sinfo)) { + if (ann.name == "comment" && ann.count == 2u) { + var tag_ok = false + var level_ok = false + for (arg in each_annotation_argument(ann)) { + if (arg.name == "tag" && arg.sValue == "alive") { + tag_ok = true + } elif (arg.name == "level" && arg.iValue == 3) { + level_ok = true + } + } + struct_ok = tag_ok && level_ok + } + } + var field_ok = false + unsafe { + for (fld in *sinfo) { + if (fld.name == "x" && fld.annotation_argument_count == 1u) { + for (arg in each_annotation_argument(fld)) { + if (arg.name == "marker" && arg.iValue == 42) { + field_ok = true + } + } + } + } + } + return struct_ok && field_ok +} diff --git a/tests/language/annotation_info.das b/tests/language/annotation_info.das new file mode 100644 index 000000000..00b634c94 --- /dev/null +++ b/tests/language/annotation_info.das @@ -0,0 +1,168 @@ +// Annotation reflection through context debug info (AnnotationInfo / AnnotationArgumentInfo). +// Deliberately compiled WITHOUT `options rtti` — annotation data is deep-copied into the +// context debug heap and must not depend on the Program staying alive. + +options gen2 +options no_unused_function_arguments = false +options no_unused_block_arguments = false + +require dastest/testing_boost public +require daslib/rtti +require strings + +[comment(v_int = 13, v_float = 1.5, v_bool = true, v_str = "hello", v_esc = "q\"uote\\back\nline")] +struct AnnotatedStruct { + @v_int = 13 a : int + @v_float = 1.5 b : float + @v_bool = true c : bool + @v_str = "hello" d : string + e : int // no annotations +} + +enum PlainEnum { + one + two +} + +[export] +def annotated_fn() { + feint("annotated\n") +} + +[test] +def test_struct_annotations(t : T?) { + let s = AnnotatedStruct() + let info = typeinfo rtti_typeinfo(s) + let sinfo = info.structType + t |> success(sinfo != null, "AnnotatedStruct should have StructInfo") + t |> equal(int(sinfo.annotation_count), 1) + var seen_args = 0 + for (ann in each_annotation(*sinfo)) { + t |> equal("{ann.name}", "comment") + t |> equal(int(ann.count), 5) + for (arg in each_annotation_argument(ann)) { + let value = get_annotation_argument_value(arg) + if (arg.name == "v_int") { + t |> success(value is tInt, "v_int should be int") + t |> equal(value as tInt, 13) + } elif (arg.name == "v_float") { + t |> success(value is tFloat, "v_float should be float") + t |> equal(value as tFloat, 1.5) + } elif (arg.name == "v_bool") { + t |> success(value is tBool, "v_bool should be bool") + t |> equal(value as tBool, true) + } elif (arg.name == "v_str") { + t |> success(value is tString, "v_str should be string") + t |> equal(value as tString, "hello") + } elif (arg.name == "v_esc") { + t |> success(value is tString, "v_esc should be string") + // annotation string_constant unescapes only \" — backslashes stay + // literal; build the expected value from chars to stay independent + // of expression-string escape rules + let expected = "q" + to_char(34) + "uote" + to_char(92) + to_char(92) + "back" + to_char(92) + "nline" + t |> equal(value as tString, expected) + } + seen_args ++ + } + } + t |> equal(seen_args, 5) +} + +[test] +def test_field_annotations(t : T?) { + let s = AnnotatedStruct() + let sinfo = (typeinfo rtti_typeinfo(s)).structType + var checked = 0 + unsafe { + for (fld in *sinfo) { + if (fld.name == "a") { + t |> equal(int(fld.annotation_argument_count), 1) + for (arg in each_annotation_argument(fld)) { + t |> equal("{arg.name}", "v_int") + t |> equal(get_annotation_argument_value(arg) as tInt, 13) + } + checked ++ + } elif (fld.name == "b") { + for (arg in each_annotation_argument(fld)) { + t |> equal(get_annotation_argument_value(arg) as tFloat, 1.5) + } + checked ++ + } elif (fld.name == "c") { + for (arg in each_annotation_argument(fld)) { + t |> equal(get_annotation_argument_value(arg) as tBool, true) + } + checked ++ + } elif (fld.name == "d") { + for (arg in each_annotation_argument(fld)) { + t |> equal(get_annotation_argument_value(arg) as tString, "hello") + } + checked ++ + } elif (fld.name == "e") { + t |> equal(int(fld.annotation_argument_count), 0) + checked ++ + } + } + } + t |> equal(checked, 5) +} + +[test] +def test_function_annotations(t : T?) { + var found_fn = false + var found_export = false + context_for_each_function() $(finfo) { + if (finfo.name == "annotated_fn") { + found_fn = true + for (ann in each_annotation(finfo)) { + if (ann.name == "export") { + found_export = true + } + } + } + } + t |> success(found_fn, "annotated_fn should be in context debug info") + t |> success(found_export, "annotated_fn should carry the [export] annotation") +} + +[test] +def test_enum_annotations(t : T?) { + let info = typeinfo rtti_typeinfo(PlainEnum.one) + let einfo = info.enumType + t |> success(einfo != null, "PlainEnum should have EnumInfo") + t |> equal(int(einfo.annotation_count), 0) +} + +[test] +def test_resolve_annotation(t : T?) { + let s = AnnotatedStruct() + let sinfo = (typeinfo rtti_typeinfo(s)).structType + for (ann in each_annotation(*sinfo)) { + let live = resolve_annotation(ann) + t |> success(live != null, "comment annotation should resolve to the live object") + if (live != null) { + t |> equal("{live.name}", "comment") + t |> success(live.isStructureAnnotation, "comment is a structure annotation") + } + } +} + +// the deprecated API keeps working: Annotation* is env-resolved, args are reconstructed +[test] +def test_deprecated_structure_for_each_annotation(t : T?) { + let s = AnnotatedStruct() + let sinfo = (typeinfo rtti_typeinfo(s)).structType + var seen = 0 + unsafe { + structure_for_each_annotation(*sinfo) $(ann; args) { + t |> equal("{ann.name}", "comment") + t |> equal(length(args), 5) + for (arg in args) { + if (arg.name == "v_int") { + t |> equal(arg.iValue, 13) + } + } + seen ++ + } + } + t |> equal(seen, 1) +} diff --git a/tutorials/integration/cpp/06_interop.cpp b/tutorials/integration/cpp/06_interop.cpp index b7e26b673..4237d04ab 100644 --- a/tutorials/integration/cpp/06_interop.cpp +++ b/tutorials/integration/cpp/06_interop.cpp @@ -32,11 +32,11 @@ using namespace das; vec4f describe_type(Context & context, SimNode_CallBase * call, vec4f * args) { TypeInfo * ti = call->types[0]; // TypeInfo for the first argument - // NOTE: TypeInfo has a union: structType / enumType / annotation_or_name + // NOTE: TypeInfo has a union: structType / enumType / annotation_info // Which member is valid depends on ti->type: // tStructure → structType (StructInfo *) // tEnumeration → enumType (EnumInfo *) - // tHandle → annotation_or_name (use getAnnotation()) + // tHandle → annotation_info (use getAnnotation()) // Accessing the wrong union member is undefined behavior! TextWriter tw; diff --git a/utils/daslang-live/main.cpp b/utils/daslang-live/main.cpp index 2676c930b..5ee3535e7 100644 --- a/utils/daslang-live/main.cpp +++ b/utils/daslang-live/main.cpp @@ -182,8 +182,7 @@ static void auto_tick_agents() { struct CompileResult { // moduleGroup owns the non-builtin dep modules (deletes them on reset()). - // Program/Context hold raw Module* into it via library.modules and TypeInfo - // (vi->annotation_arguments → FieldDeclaration::annotation), so it must + // Program/Context hold raw Module* into it via library.modules, so it must // outlive program+ctx. Declared first → destroyed last. unique_ptr moduleGroup; ProgramPtr program;