Formal Validation with @Validation

This page is mainly meant as a overview or presentation around the introduction of formal input validation using @Validation as introduced by #23754.

Formal Input Validation

• context-less (static context)
• no state
• no data related validation (just the input itself)
• no semantic validation

Typical examples

• property is required, e.g. name
• number must be be positive, e.g. page
• text must match a pattern, e.g. a UID
• set of properties must be defined together, e.g. cc+co => coc

What is @Validation?

@Validation is a declarative Java-API for (a subset of) JSON-schema-validation

🥳 we can extract JSON-schema conform JSON and include it in our OpenAPI documentation to inform clients about expectations that is both human and machine readable

It follows: If input is formally wrong, the caller made an avoidable mistake => BadRequestException

What extend of JSON-schema-validation is supported?

@Validation supports most of the fundamental validations 1:1 as far as they make sense in the context of mapping JSON to Java

The Rules include

• type (JSON node type)
• enum (JSON equals from a list of options)
• string: minLength, maxLength, pattern
• number: minimum, exclusiveMinimum, maximum, exclusiveMaximum, multipleOf
• array: minItems, maxItems, uniqueItems
• object: minProperties, maxProperties, required, dependentRequired

Custom validations can be added using @Validator implementing @Validation.Validator

Where does it apply?

• In #23754 validation from @Validation is first added to URL request parameter objects
• Opt-in: by using records that implement UrlParams
• can easily be extended to request bodies (as validation is JSON based)

Examlple MinMaxDataElementParams

record MinMaxDataElementParams(
    List<String> fields, List<String> filters, @Collapsed PagedParams paged) 
    implements UrlParams {

  public static final MinMaxDataElementParams DEFAULT =
      new MinMaxDataElementParams(List.of(), List.of(), PagedParams.DEFAULT);
}

But: Only supports records to model "objects"

Object-composition with @Collapsed

@Collapsed is like an include for nested records

record Outer(
  String name,
  @Collapsed Inner inner) {}
record Inner(int page, int pageSize) {}

Is eqivalent to

record Outer(
  String name,
  int page, 
  int pageSize) {}

💡 allows equivalent of multiple inheritance

👍 Can be nested as deep as needed

Meta-Annotations

Declaration of meta-annotations:

  @Retention(RUNTIME)
  @Validation(minLength = 1, minItems = 1, minProperties = 1)
  @interface NotEmpty {}

Usage:

record MyParams(@NotEmpty String text) {}

A test example of how to re-create common bean annotations.

💡 meta-annotations do support parametrization (see example)

Custom Validators

Use @Validator refering to a Class implementing @Validation.Validator

record NoKeywordsValidator {
    @Override
    public void validate(JsonMixed value, Consumer<Error> addError) {
      if (!value.isString()) return;
      if (List.of("select", "delete").stream().anyMatch(keyword -> value.text().contains(keyword)))
        addError.accept(Error.of(Rule.CUSTOM, value, "Value contains keyword %s", value.text()));
    }
}

Usage:

record QueryParams(
  @Validator(NoKeywordsValidator.class) String query) {}

💡 validators do support parametrization (see example)

Validation & Mapping to Java

• JSON + Java types can be mixed
• allows "delayed" validation

record ImportParams(
  // header
  UID de,
  UID ou,
  // body/data
  JsonArray elements
) {}

In the service...

record Entry(UID coc, @Validation(minimum=0) Number value) {}

void import(ImportParams params) {
  params.elements.forEach(e -> {
    // validate
    e.validate(Entry.class);
    // map
    Entry entry = e.to(Entry.class));
    // ...
  }
}

💡 allows to validate thought the same mechanism while also stream-processing large inputs.

elements and e exist in the original input context, e.g. we can get the index of each e for error handling etcetera

Why @Validation? (and not bean-validation or alike)

• uniform validation of URL request parameters and request bodies
• uniform API: @Validation + @Validator
• full generic support (e.g. List<@Validation(maxLength=5)String>
• consistent behaviour as validation and type mapping go hand in hand
• validation occurs on input (JSON) not Java (which might have lost information or failed to translate masking errors)
• easy to extend and customize: @Validator + @Validation.Validator + meta-annotations
• low overhead, fail fast/early
• allows mixing JSON and Java types
• full control on this critical feature remains in DHIS2 hands
  • => options (like direct integration of complex URL parameters)

Complex URL parameters

JSON in the URL JURL can be used for URL request parameters to seamlessly and effortlessly work with URL request parameters that are more complex than a simple value.

💡 Instead of inventing use-case specific mini languages like those for filter or fields we can use JURL (see example)

To illustrate the idea:

filter=name,id,dataElement[id,name],orgUnits::size

in JURL

filter=(name,id,(dataElements:(id,name)),(orgUnits:size))

Open Questions

• use of meta-annotations?
• include first error (clarity) or all errors (completeness) in message?
• any particular JSON structure for machine readable errors list?