Finish Delayed field evaluation implementation and document feature set as part of "Supported Toolchain"

[MrChocolateMoose]

Is your feature request related to a problem? Please describe.

I looked through the existing issues and couldn't find an issue noting that there is still work needed to be done to complete delayed field evaluation functionality. I'd like to communicate that I think this is an important use case.

I see some mention of dynamic functions being implemented on the back-end in 0.9.0

I would very much like to use delayed field evaluation functionality and also request additional functionality in sphinx-needs, but it's unclear how to phrase the ask because I only want to ask for features that would be feasible to be in the supported toolchain. I am less interested in asking for sphinx-needs features that are not feasible to certify especially where they relate to a critical aspect of my process workflow.

Documentation stating what the supported toolchain is would be very important to me and also dovetail into information I would expect to see in a safety manual.

Describe the solution you'd like
I would like to see at least variant functions and dynamic functions (with some preset of built-ins) implemented in ubcode.

Since I assume there are logistical challenges with custom functionality implemented in python code, I assume that using arbitrary "custom functions" is a no-no for the safe toolchain, but no documentation beyond my own judgement is guiding me.

Describe alternatives you've considered
I am not currently using some kind of documentation process that uses build time events to write out static values into needextend nor is it my plan in the immediate term.

Additional context
N/A

[ubmarco]

Hi, thanks for the write-up. The following functionality belongs to delayed evaluation:

• global options (defaults)
• needextend
• variants
• dynamic functions

From those the dynamic functions are certainly the most problematic ones to reproduce in ubCode. We are currently looking into Starlark as a thread-safe alternative with an implementation in Rust that has bindings for Python. It looks promising.

There are more problems with dynamic functions. First, the definition of custom functions cannot be read out by ubCode. We would have to change the definition to a declarative form, e.g. users configure a python file and a function inside that gets registered. An implementation in Sphinx-Needs would be required first to get the interface right.

Then, dependencies. The function signature of custom functions is

def my_own_function(app, need, needs):
    return "Awesome"

Users cannot specify what the function depends on, which has 2 effects down the road:

1. ubCode does not know when to re-run the function once something changes. Any change in the needs, anywhere in the project, could affect the dynamic function implementation to return something different. That is because the function depends on all needs. That is not so much a problem for Sphinx-Needs, as the build just runs once. The ubCode language server, however, is expected to evaluate information in real-time. Re-executing all functions for each change of a need item is not a solution.
2. The execution order of dynamic function cannot be controlled by the user as there is no interface for this. Users are currently in charge to only depend on information that is statically available, e.g. primary RST field information. Dynamic functions cannot depend on other unresolved fields.

Another problem is, the current form of dynamic functions is running in the context of the user provided Python environment. That means dynamic functions can depend on external libraries that ubCode has no access to. For example, functions are free to contact an Elastic Search instance during function execution and use the return data as field value. With great power comes great responsibility... I guess the main use cases of dynamic functions are field value generation based on other needs, e.g. calculate a ISO26262 TCL value of a use case from linked features. But you never know.

With that said, I feel we need a version 2 of dynamic functions that addresses these points.
Sorry I don't have better news for you right now. We are working on a big release for ubCode that will catch up with all the features of Sphinx-Needs 6 (strong typing, schema validation, new variant syntax, parsing strategy and more). Dynamic function will be detected (the [[ ]] syntax), but not evaluated due to above reasons.

[ubmarco]

As a starting point for the discussion (which actually belongs to Sphinx-Needs), what's your use case for dynamic functions?