1365 evaluation order

[hackowitz-af]

Closes #1365

jsonschema/tests/test_validators.py

Adds a test method for the expected behavior, and reverences the issue being solved. The method is slightly longer than others to avoid TestValidationErrorMessages.message_for, which asserts that there be exactly one error.

jsonschema/_keyword.py

The Keyword class can define additional tags/behavior for the implementation of the keyword. For now, I have only added a "needs" label, which names other keywords that it cannot be executed before. The name "needs" may be inaccurate, and I'd be happy to change it.

The keyword decorator makes it trivial to annotate a function to be used as a validator keyword.

keyword and Keyword are both optional and completely back-compatible. They would only be necessary for new behavior. There is no change to existing validator function behavior.

jsonschema/validators.py

The only changed behavior is that the iter_errors evaluates Keywords after their needs. The logic is not as clean as I'd like, but is reasonably straightforward:

• Keep a memo of todo keywords.
• As keywords are evaluated, they are no longer todo.
• Any Keyword that needs a todo keyword is labeled dependent and skipped
• If all attempted keywords were dependent of each other, there is a circular dependency and an error is raised.
• dependent keywords are re-attempted.

I'd be happy to add comments or a docstring to make the code more self-descriptive if necessary.

---

📚 Documentation preview 📚: https://python-jsonschema--1366.org.readthedocs.build/en/1366/

[Julian]

Thanks sincerely for giving this a shot! Like I said on the issues, I'd like this done in a more systematic way. Specifically the concerns include:

• this code is quite performance sensitive so adding extra indirection here is quite possibly harmful on CPython and needs careful benchmarking
• more importantly, I don't want to bolt on the notion of dependence order to the existing APIs -- doing this right I think needs to involve a rewrite of the validator protocol (which I'm referring to as "dialects" as that's the modern name for the concept in JSON Schema) in order to support annotations (== state + dependency order). Such an API should allow users to specify ordering, to modify ordering when extending validators, and then should likely do things like topologically sort the order at validator-build time in order to ensure the order is well defined and known ahead of time when compiling a schema. Such a thing is a larger project, which is why it hasn't happened yet, but it's the "right" next step to me.

If you'd like to help, which is very much appreciated, you could review the Dialect v2 label, help by speccing out what a new protocol might look like (I have ideas here clearly but haven't tried to write them out), or help investigate our existing benchmarks and thereby help ensure we're as good or better when we introduce this dependency notion.

I'm going to close this as I wouldn't merge it as an incremental change as I said, but again, I do appreciate your work!

[Julian]

(Just because I was giving a list of ways to hopefully constructively focus any help offered -- reviewing unrelated issues is another way to help this move along too, as I have more or less a fixed amount of attention I can spend on the library at this point and so time spent on diagnosing other issues of course takes a decent portion of that attention span up, which delays getting new design work done.)

[hackowitz-af]

All good, thanks for the feedback! I'll see about speccing out a new protocol in the next few months, and getting familiar with the related issues.