diff --git a/DESIGN.md b/DESIGN.md index 34ea936..95abe70 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -11,14 +11,16 @@ Thim optimizes for four properties: ## Compilation -1. The Gradle plugin tracks HTML, message bundles and model sources. +1. The Gradle plugin tracks HTML, strict YAML message catalogs and model sources. 2. KSP resolves a page-model class from the template filename and configured model packages. -3. The compiler links fixed layouts and fragments, then validates properties, nullability, localized messages and supported directives. +3. The compiler links fixed layouts and fragments, then validates properties, nullability, locale/key/argument parity, plural and select rules, and supported directives. 4. It emits readable Java renderers and one package-local resource containing static UTF-8 content. 5. Each template jar publishes its generated registry through Java's service loader. Kotlin modules use the KSP Gradle integration. Java modules run KSP2 directly against Java sources, including records and bean accessors. Both paths call the same compiler and generate the same runtime code. +Message catalogs use a failsafe YAML 1.2 subset containing only mappings and string scalars. Locale directories, filenames and nested mappings provide structure without repeating dotted keys. Parsing and validation are build-time work; generated branches contain the translated UTF-8 bytes and locale decisions. + ## Runtime The runtime, generated renderers, Spring adapter and Gradle plugin are Java. A request creates one buffered `HtmlOutput`, copies static byte ranges and encodes escaped dynamic values directly into that buffer. It creates no intermediate escaped strings and performs no character conversion for static content. diff --git a/README.md b/README.md index bb1c29c..042dbb5 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ Apply the plugin after the Kotlin JVM plugin in Kotlin modules: ```kotlin plugins { kotlin("jvm") - id("no.beint.thim") version "0.4.20" + id("no.beint.thim") version "0.5.0" } ``` @@ -54,7 +54,7 @@ Java modules need only the Java and Thim plugins: ```kotlin plugins { java - id("no.beint.thim") version "0.4.20" + id("no.beint.thim") version "0.5.0" } ``` @@ -63,7 +63,9 @@ The plugin supplies the runtime, compiler and Spring adapter and tracks template ```kotlin thim { templates.set(layout.projectDirectory.dir("src/main/resources/templates")) - messages.set(layout.projectDirectory.dir("src/main/resources")) + messages.set(layout.projectDirectory.dir("src/main/resources/i18n")) + defaultLocale.set("en") + supportedLocales.set(listOf("en", "nb")) generatedPackage.set("your.group.your_module.thim.generated") registryName.set("ThimTemplates") modelPackages.set(listOf("no.example.page")) @@ -91,6 +93,51 @@ Use `thimCheck` for fast template validation during development: `thimCheck` runs the same validation as normal compilation. Java modules also write a report to `build/reports/thim/check.json`. +## Message catalogs + +Thim compiles localized YAML catalogs into the generated renderer. SnakeYAML Engine is a compiler dependency only; parsing, key lookup and pattern interpretation never happen at request time. + +The directory name is a canonical BCP 47 language tag. The relative YAML filename and nested mappings form the message namespace: + +```text +src/main/resources/i18n/ +├── en/ +│ └── home.yaml +└── nb/ + └── home.yaml +``` + +```yaml +# en/home.yaml +title: Thim {version} +introduction: |- + Compile templates and translations together. + Ship no runtime template engine. +inbox: + _plural: unreadCount + one: One unread message + other: "{unreadCount} unread messages" +salutation: + _select: audience + MEMBER: Welcome back, {name} + other: Welcome, {name} +``` + +Use named model properties in the template: + +```html +Thim +

Unread messages

+``` + +`_plural` accepts the locale's reachable subset of `zero`, `one`, `two`, `few`, `many` and the required `other` category. Its argument must be a non-null integral property. `_select` requires a non-null string or enum and also requires `other`; enum variants must name real enum constants. Selections can be nested. All interpolated values are HTML-escaped; catalogs cannot produce raw HTML. Write `{{` or `}}` for a literal brace. + +Every configured locale must contain exactly the same message keys and argument contracts. The default locale defines the contract. Missing translations, extra keys, misspelled placeholders, incompatible argument types and unused messages (when enabled) fail compilation. At runtime Thim chooses an exact configured language tag, then a configured language-only tag, then the default locale. + +Catalogs use a deliberately small YAML 1.2 profile: mappings and string scalars only. The failsafe schema means plain `no`, `true`, `12` and `2026-08-08` remain text. Duplicate keys, tags, anchors, aliases, sequences and multiple documents are rejected. Block scalars are supported for multiline copy. Only the `.yaml` extension is accepted. + +Integer cardinal rules derived from [Unicode CLDR 49](https://unicode.org/cldr/charts/49/supplemental/language_plural_rules.html) are embedded for `af`, `bg`, `bs`, `ca`, `cs`, `cy`, `da`, `de`, `el`, `en`, `eo`, `es`, `et`, `eu`, `fi`, `fo`, `fr`, `ga`, `gd`, `gl`, `hr`, `hu`, `is`, `it`, `lt`, `lv`, `nb`, `nl`, `nn`, `no`, `pl`, `pt`, `ro`, `sk`, `sl`, `sq`, `sr`, `sv` and `sw`. A catalog that uses `_plural` with another language fails compilation rather than guessing. + ## Typed controller routes Kotlin applications can opt into controller-side route builders with `generateRoutes.set(true)`. The generated object is placed beside the template registry and replaces a `Templates` suffix with `Routes`: `WebAppTemplates` produces `WebAppRoutes`. Set `routesName` to override it. @@ -135,7 +182,7 @@ fun select(): ThimResult = Existing handlers declared as Kotlin `Any` or Java `Object` remain supported, but `ThimResult` documents mixed page/redirect outcomes more clearly. -Artifacts are published through `https://maven.pkg.github.com/beint-no/thim`. Add that repository to `pluginManagement` and dependency resolution. +Artifacts and the Gradle plugin marker are published to Maven Central. Add `mavenCentral()` to `pluginManagement` and dependency resolution. ## Template syntax @@ -149,7 +196,7 @@ Thim accepts: - property, message, static URL and quoted-literal values on ordinary `th:*` attributes - `no.beint.thim.TrustedUrl` properties on URL attributes such as `th:href`, `th:src` and `th:action` - conditional HTML boolean attributes -- literal `#{message}` expressions with typed arguments +- literal `#{message(argument=${property})}` expressions with typed, named arguments - `${#locale.language}` for a language attribute Every dynamic value is encoded for its output context. Use static `@{...}` expressions or `TrustedUrl` for URLs, and use `SafeHtml` only with `th:utext`. Dynamic JavaScript, CSS and event-handler content is rejected. diff --git a/build.gradle.kts b/build.gradle.kts index b153431..50d911e 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -15,7 +15,7 @@ plugins { allprojects { group = "no.beint.thim" - version = "0.4.20" + version = "0.5.0" } subprojects { diff --git a/compiler/build.gradle.kts b/compiler/build.gradle.kts index d2bc38c..cfade77 100644 --- a/compiler/build.gradle.kts +++ b/compiler/build.gradle.kts @@ -18,4 +18,13 @@ tasks.withType().configureEach { dependencies { implementation("com.google.devtools.ksp:symbol-processing-api:2.3.10") + implementation("org.snakeyaml:snakeyaml-engine:3.1.1") + + testImplementation(platform("org.junit:junit-bom:6.0.3")) + testImplementation(kotlin("test-junit5")) + testRuntimeOnly("org.junit.platform:junit-platform-launcher") +} + +tasks.test { + useJUnitPlatform() } diff --git a/compiler/src/main/kotlin/no/beint/thim/compiler/Expressions.kt b/compiler/src/main/kotlin/no/beint/thim/compiler/Expressions.kt index 8b91691..01085cf 100644 --- a/compiler/src/main/kotlin/no/beint/thim/compiler/Expressions.kt +++ b/compiler/src/main/kotlin/no/beint/thim/compiler/Expressions.kt @@ -4,7 +4,7 @@ internal data class PathSegment(val name: String, val safe: Boolean) internal data class PathExpression(val segments: List) -internal data class MessageExpression(val key: String, val arguments: List) +internal data class MessageExpression(val key: String, val arguments: Map) internal sealed interface UrlArgument @@ -76,13 +76,20 @@ internal object Expressions { val opening = body.indexOf('(') if (opening == -1) { require(body.matches(keyPattern)) { "$context: invalid message key '$body'" } - return MessageExpression(body, emptyList()) + return MessageExpression(body, emptyMap()) } require(body.endsWith(')')) { "$context: invalid message expression '$value'" } val key = body.substring(0, opening).trim() require(key.matches(keyPattern)) { "$context: invalid message key '$key'" } - val arguments = splitArguments(body.substring(opening + 1, body.length - 1), context) - .map { path(it, context) } + val arguments = linkedMapOf() + splitArguments(body.substring(opening + 1, body.length - 1), context).forEach { argument -> + val equals = argument.indexOf('=') + require(equals > 0) { "$context: message arguments must use name=\${property}" } + val name = argument.substring(0, equals).trim() + require(name.matches(argumentNamePattern)) { "$context: invalid message argument name '$name'" } + require(name !in arguments) { "$context: duplicate message argument '$name'" } + arguments[name] = path(argument.substring(equals + 1).trim(), context) + } return MessageExpression(key, arguments) } @@ -156,6 +163,7 @@ internal object Expressions { } private val keyPattern = Regex("[A-Za-z0-9_][A-Za-z0-9_.-]*") + private val argumentNamePattern = Regex("[A-Za-z_][A-Za-z0-9_]*") private val numberPattern = Regex("-?\\d+") private val pathVariablePattern = Regex("\\{([A-Za-z_][A-Za-z0-9_]*)}") } diff --git a/compiler/src/main/kotlin/no/beint/thim/compiler/MessageCatalog.kt b/compiler/src/main/kotlin/no/beint/thim/compiler/MessageCatalog.kt index 5b04d8d..05d0075 100644 --- a/compiler/src/main/kotlin/no/beint/thim/compiler/MessageCatalog.kt +++ b/compiler/src/main/kotlin/no/beint/thim/compiler/MessageCatalog.kt @@ -1,121 +1,411 @@ package no.beint.thim.compiler +import org.snakeyaml.engine.v2.api.LoadSettings +import org.snakeyaml.engine.v2.api.lowlevel.Compose +import org.snakeyaml.engine.v2.api.lowlevel.Parse +import org.snakeyaml.engine.v2.events.CollectionStartEvent +import org.snakeyaml.engine.v2.events.Event +import org.snakeyaml.engine.v2.events.NodeEvent +import org.snakeyaml.engine.v2.events.ScalarEvent +import org.snakeyaml.engine.v2.events.SequenceStartEvent +import org.snakeyaml.engine.v2.nodes.MappingNode +import org.snakeyaml.engine.v2.nodes.Node +import org.snakeyaml.engine.v2.nodes.NodeType +import org.snakeyaml.engine.v2.nodes.ScalarNode +import org.snakeyaml.engine.v2.nodes.Tag +import org.snakeyaml.engine.v2.schema.FailsafeSchema import java.nio.charset.StandardCharsets import java.nio.file.Files import java.nio.file.Path -import java.util.Properties +import java.util.IllformedLocaleException +import java.util.Locale import kotlin.io.path.extension +import kotlin.io.path.isDirectory +import kotlin.io.path.name import kotlin.io.path.nameWithoutExtension +internal enum class MessageArgumentKind { + TEXT, + NUMBER, + SELECT, +} + +internal sealed interface MessagePart + +internal data class MessageText(val value: String) : MessagePart + +internal data class MessageArgument(val name: String) : MessagePart + +internal sealed interface MessageValue { + fun arguments(): Map +} + +internal data class MessagePattern(val parts: List) : MessageValue { + override fun arguments(): Map = parts + .filterIsInstance() + .associate { it.name to MessageArgumentKind.TEXT } +} + +internal data class MessageSelection( + val argument: String, + val kind: MessageArgumentKind, + val variants: Map, +) : MessageValue { + override fun arguments(): Map = buildMap { + put(argument, kind) + variants.values.forEach { value -> + value.arguments().forEach { (name, valueKind) -> + val existing = putIfAbsent(name, valueKind) + if (existing != null) put(name, mergeArgumentKinds(name, existing, valueKind)) + } + } + } +} + internal data class MessageDefinition( - val base: String, - val localized: Map, + val values: Map, + val arguments: Map, ) internal class MessageCatalog private constructor( private val definitions: Map, + val defaultLocale: String, + val supportedLocales: List, ) { private val used = linkedSetOf() - fun use(key: String, argumentCount: Int, context: String): MessageDefinition { + fun use(key: String, arguments: Set, context: String): MessageDefinition { val definition = definitions[key] ?: error( "$context: message '$key' does not exist${suggestion(key, definitions.keys)}", ) used += key - val placeholders = placeholders(definition.base, "$context message '$key'") - definition.localized.forEach { (locale, value) -> - require(placeholders(value, "$context message '$key' locale '$locale'") == placeholders) { - "$context: message '$key' uses different placeholders in locale '$locale'" + val expected = definition.arguments.keys + require(arguments == expected) { + val missing = expected - arguments + val extra = arguments - expected + buildString { + append("$context: message '$key' arguments do not match") + if (missing.isNotEmpty()) append("; missing ${missing.sorted()}") + if (extra.isNotEmpty()) append("; unknown ${extra.sorted()}") } } - val expected = if (placeholders.isEmpty()) 0 else placeholders.max() + 1 - require(argumentCount == expected) { - "$context: message '$key' requires $expected arguments, received $argumentCount" - } return definition } - fun locales(): Set = definitions.values - .flatMapTo(linkedSetOf()) { definition -> - definition.localized.filterValues { it != definition.base }.keys - } - fun requireAllUsed() { val unused = definitions.keys - used require(unused.isEmpty()) { "Unused messages: ${unused.sorted()}" } } companion object { - fun load(directory: Path): MessageCatalog { - if (!Files.exists(directory)) return MessageCatalog(emptyMap()) - val files = Files.walk(directory).use { paths -> - paths.filter { Files.isRegularFile(it) && it.extension == "properties" && bundlePattern.matches(it.fileName.toString()) } + fun load(directory: Path, defaultLocale: String, supportedLocales: List): MessageCatalog { + val canonicalDefault = canonicalLocale(defaultLocale) + val canonicalSupported = supportedLocales.map(::canonicalLocale) + require(canonicalSupported.isNotEmpty()) { "Thim needs at least one supported locale" } + require(canonicalSupported.size == canonicalSupported.distinct().size) { + "Supported locales contain duplicates: $canonicalSupported" + } + require(canonicalDefault in canonicalSupported) { + "Default locale '$canonicalDefault' is not in supported locales $canonicalSupported" + } + if (!Files.exists(directory)) return MessageCatalog(emptyMap(), canonicalDefault, canonicalSupported) + require(Files.isDirectory(directory)) { "Message catalog directory does not exist: $directory" } + val shortExtensions = Files.walk(directory).use { paths -> + paths.filter { Files.isRegularFile(it) && it.extension == "yml" } + .map(Path::toString) .sorted() .toList() } - val grouped = files.groupBy { bundleIdentity(it) } - val definitions = linkedMapOf() + require(shortExtensions.isEmpty()) { + "Message catalogs must use the .yaml extension, found $shortExtensions" + } - grouped.forEach { (identity, bundleFiles) -> - val baseFile = bundleFiles.singleOrNull { localeOf(it) == null } - ?: error("Message bundle '$identity' has localized files but no base file") - val base = readProperties(baseFile) - val localized = bundleFiles.filter { it != baseFile }.associate { file -> - localeOf(file)!! to readProperties(file) + val discovered = Files.list(directory).use { paths -> + paths.filter { Files.isDirectory(it) } + .filter { localeDirectory -> containsYaml(localeDirectory) } + .map { it.name } + .sorted() + .toList() + } + val unexpected = discovered - canonicalSupported.toSet() + require(unexpected.isEmpty()) { "Message catalog has unsupported locale directories $unexpected" } + + val localeDefinitions = linkedMapOf>() + canonicalSupported.forEach { locale -> + val localeDirectory = directory.resolve(locale) + require(localeDirectory.isDirectory()) { "Message catalog is missing locale directory '$locale'" } + localeDefinitions[locale] = readLocale(localeDirectory, locale) + } + + val base = localeDefinitions.getValue(canonicalDefault) + localeDefinitions.forEach { (locale, values) -> + val missing = base.keys - values.keys + val extra = values.keys - base.keys + require(missing.isEmpty()) { "Message catalog locale '$locale' is missing ${missing.sorted()}" } + require(extra.isEmpty()) { "Message catalog locale '$locale' has extra keys ${extra.sorted()}" } + } + + val definitions = base.keys.associateWithTo(linkedMapOf()) { key -> + val values = canonicalSupported.associateWithTo(linkedMapOf()) { locale -> + localeDefinitions.getValue(locale).getValue(key) } - localized.forEach { (locale, values) -> - val missing = base.keys - values.keys - val extra = values.keys - base.keys - require(missing.isEmpty()) { "$fileLabel: locale '$locale' is missing ${missing.sorted()}" } - require(extra.isEmpty()) { "$fileLabel: locale '$locale' has extra keys ${extra.sorted()}" } + val arguments = values.getValue(canonicalDefault).arguments() + values.forEach { (locale, value) -> + validatePluralCategories(key, locale, value) + val localizedArguments = value.arguments() + val missing = arguments.keys - localizedArguments.keys + val extra = localizedArguments.keys - arguments.keys + require(missing.isEmpty() && extra.isEmpty()) { + buildString { + append("Message '$key' in locale '$locale' changes the argument contract") + if (missing.isNotEmpty()) append("; missing ${missing.sorted()}") + if (extra.isNotEmpty()) append("; extra ${extra.sorted()}") + } + } + localizedArguments.forEach { (name, kind) -> + require(arguments.getValue(name) == kind) { + "Message '$key' argument '$name' is ${arguments.getValue(name).name.lowercase()} " + + "in '$canonicalDefault' but ${kind.name.lowercase()} in '$locale'" + } + } + } + MessageDefinition(values, arguments) + } + return MessageCatalog(definitions, canonicalDefault, canonicalSupported) + } + + private fun validatePluralCategories(key: String, locale: String, value: MessageValue) { + if (!value.usesPlural()) return + val language = Locale.forLanguageTag(locale).language + val categories = pluralCategories[language] + require(categories != null) { + "Message '$key' uses _plural in locale '$locale', whose cardinal rules are not supported yet" + } + value.pluralSelections().forEach { selection -> + val unreachable = selection.variants.keys - categories + require(unreachable.isEmpty()) { + "Message '$key' in locale '$locale' uses unreachable plural categories ${unreachable.sorted()}; " + + "supported categories are ${categories.sorted()}" } + } + } - base.forEach { (key, value) -> - require(key !in definitions) { "Duplicate message key '$key'" } - val localizedValues = localized.mapValues { (locale, values) -> - values.getValue(key) + private fun readLocale(directory: Path, locale: String): Map { + val files = Files.walk(directory).use { paths -> + paths.filter { Files.isRegularFile(it) && it.extension == "yaml" } + .sorted() + .toList() + } + require(files.isNotEmpty()) { "Message catalog locale '$locale' contains no .yaml files" } + val definitions = linkedMapOf() + files.forEach { file -> + val relative = directory.relativize(file) + val namespaceParts = relative.map { it.toString() }.toMutableList() + namespaceParts[namespaceParts.lastIndex] = relative.fileName.nameWithoutExtension + namespaceParts.forEach { part -> + require(part.matches(keyPartPattern)) { "$file: invalid namespace component '$part'" } + } + val namespace = namespaceParts.joinToString(".") + parseFile(file, namespace).forEach { (key, value) -> + require(definitions.putIfAbsent(key, value) == null) { "$file: duplicate message '$key'" } + } + } + return definitions + } + + private fun parseFile(path: Path, namespace: String): Map { + val settings = LoadSettings.builder() + .setLabel(path.toString()) + .setSchema(FailsafeSchema()) + .setAllowDuplicateKeys(false) + .setAllowRecursiveKeys(false) + .setAllowNonScalarKeys(false) + .setMaxAliasesForCollections(0) + .setCodePointLimit(MAX_CATALOG_CODE_POINTS) + .build() + val source = Files.readString(path, StandardCharsets.UTF_8) + validateEvents(path, settings, source) + val documents = Compose(settings).composeAllFromString(source).toList() + require(documents.size == 1) { "$path: expected exactly one YAML document" } + val root = documents.single() + requireNode(root, NodeType.MAPPING, path, "catalog root must be a mapping") + val definitions = linkedMapOf() + parseNamespace(path, root as MappingNode, namespace, definitions) + return definitions + } + + private fun validateEvents(path: Path, settings: LoadSettings, source: String) { + Parse(settings).parseString(source).forEach { event -> + val violation = when { + event is SequenceStartEvent -> "YAML sequences are not supported" + event is NodeEvent && event.anchor.isPresent -> "anchors and aliases are not supported" + event is ScalarEvent && event.tag.isPresent -> "YAML tags are not supported" + event is CollectionStartEvent && event.tag.isPresent -> "YAML tags are not supported" + else -> null + } + if (violation != null) throw IllegalArgumentException(problem(path, event, violation)) + } + } + + private fun parseNamespace( + path: Path, + mapping: MappingNode, + prefix: String, + definitions: MutableMap, + ) { + val entries = mappingEntries(path, mapping) + require(entries.keys.none { it.startsWith('_') }) { + problem(path, mapping, "reserved metadata is only valid inside a message definition") + } + entries.forEach { (name, node) -> + require(name.matches(keyPartPattern)) { problem(path, node, "invalid message key '$name'") } + val key = "$prefix.$name" + val value = when (node.nodeType) { + NodeType.SCALAR -> pattern(path, node as ScalarNode) + NodeType.MAPPING -> { + val child = node as MappingNode + val childEntries = mappingEntries(path, child) + if (childEntries.keys.any { it.startsWith('_') }) { + selection(path, child, childEntries) + } else { + parseNamespace(path, child, key, definitions) + null + } } - definitions[key] = MessageDefinition(value, localizedValues) + else -> error(problem(path, node, "message values must be strings or nested mappings")) + } + if (value != null) { + require(definitions.putIfAbsent(key, value) == null) { problem(path, node, "duplicate message '$key'") } } } - return MessageCatalog(definitions) } - private fun readProperties(path: Path): Map { - val keys = Files.readAllLines(path, StandardCharsets.UTF_8) - .asSequence() - .map(String::trim) - .filter { it.isNotEmpty() && !it.startsWith('#') && !it.startsWith('!') } - .map { it.substringBefore('=').substringBefore(':').trim() } - .toList() - val duplicate = keys.groupingBy { it }.eachCount().filterValues { it > 1 }.keys - require(duplicate.isEmpty()) { "$path contains duplicate keys ${duplicate.sorted()}" } + private fun selection(path: Path, mapping: MappingNode, entries: Map): MessageSelection { + val metadata = entries.keys.filter { it.startsWith('_') } + require(metadata.size == 1 && metadata.single() in setOf("_plural", "_select")) { + problem(path, mapping, "message definition needs exactly one of _plural or _select") + } + val metadataName = metadata.single() + val selectorNode = entries.getValue(metadataName) + requireNode(selectorNode, NodeType.SCALAR, path, "$metadataName must name an argument") + val argument = (selectorNode as ScalarNode).value + require(argument.matches(argumentPattern)) { problem(path, selectorNode, "invalid argument '$argument'") } + val variants = entries.filterKeys { !it.startsWith('_') }.mapValuesTo(linkedMapOf()) { (name, node) -> + require(name.matches(variantPattern)) { problem(path, node, "invalid variant '$name'") } + when (node.nodeType) { + NodeType.SCALAR -> pattern(path, node as ScalarNode) + NodeType.MAPPING -> { + val nested = node as MappingNode + val nestedEntries = mappingEntries(path, nested) + require(nestedEntries.keys.any { it.startsWith('_') }) { + problem(path, nested, "variant mappings must define _plural or _select") + } + selection(path, nested, nestedEntries) + } + else -> error(problem(path, node, "variants must be strings or nested selections")) + } + } + require("other" in variants) { problem(path, mapping, "message selection needs an 'other' variant") } + val kind = if (metadataName == "_plural") MessageArgumentKind.NUMBER else MessageArgumentKind.SELECT + if (kind == MessageArgumentKind.NUMBER) { + val invalid = variants.keys - pluralVariants + require(invalid.isEmpty()) { problem(path, mapping, "invalid plural variants ${invalid.sorted()}") } + } + return MessageSelection(argument, kind, variants) + } - val properties = Properties() - Files.newBufferedReader(path, StandardCharsets.UTF_8).use(properties::load) - return properties.stringPropertyNames().associateWith(properties::getProperty) + private fun pattern(path: Path, scalar: ScalarNode): MessagePattern { + requireSafeNode(path, scalar) + val parts = mutableListOf() + val text = StringBuilder() + fun flush() { + if (text.isNotEmpty()) { + parts += MessageText(text.toString()) + text.clear() + } + } + val value = scalar.value + var index = 0 + while (index < value.length) { + when { + value.startsWith("{{", index) -> { + text.append('{') + index += 2 + } + value.startsWith("}}", index) -> { + text.append('}') + index += 2 + } + value[index] == '{' -> { + val end = value.indexOf('}', index + 1) + require(end >= 0) { problem(path, scalar, "unterminated message argument") } + val name = value.substring(index + 1, end) + require(name.matches(argumentPattern)) { problem(path, scalar, "invalid message argument '{$name}'") } + flush() + parts += MessageArgument(name) + index = end + 1 + } + value[index] == '}' -> error(problem(path, scalar, "unmatched '}' in message; write '}}' for a literal brace")) + else -> text.append(value[index++]) + } + } + flush() + return MessagePattern(parts) } - private fun placeholders(value: String, context: String): Set { - val found = placeholderPattern.findAll(value).map { it.groupValues[1].toInt() }.toSet() - val withoutPlaceholders = value.replace(placeholderPattern, "") - require('{' !in withoutPlaceholders && '}' !in withoutPlaceholders) { - "$context: only numeric {0} placeholders are supported" + private fun mappingEntries(path: Path, mapping: MappingNode): Map { + requireSafeNode(path, mapping) + val entries = linkedMapOf() + mapping.value.forEach { tuple -> + val keyNode = tuple.keyNode + requireNode(keyNode, NodeType.SCALAR, path, "mapping keys must be strings") + requireSafeNode(path, keyNode) + val key = (keyNode as ScalarNode).value + require(entries.putIfAbsent(key, tuple.valueNode) == null) { + problem(path, keyNode, "duplicate key '$key'") + } } - if (found.isNotEmpty()) { - require(found == (0..found.max()).toSet()) { "$context: placeholders must be contiguous from {0}" } + return entries + } + + private fun requireNode(node: Node, type: NodeType, path: Path, message: String) { + require(node.nodeType == type) { problem(path, node, message) } + requireSafeNode(path, node) + } + + private fun requireSafeNode(path: Path, node: Node) { + require(node.anchor.isEmpty) { problem(path, node, "anchors and aliases are not supported") } + val expectedTag = when (node.nodeType) { + NodeType.SCALAR -> Tag.STR + NodeType.MAPPING -> Tag.MAP + else -> null } - return found + require(expectedTag == null || node.tag == expectedTag) { problem(path, node, "YAML tags are not supported") } } - private fun bundleIdentity(path: Path): String { - val stem = path.nameWithoutExtension.replace(localeSuffix, "") - return "${path.parent}:$stem" + private fun problem(path: Path, node: Node, message: String): String = + node.startMark.map { mark -> "$path:${mark.line + 1}:${mark.column + 1}: $message" } + .orElse("$path: $message") + + private fun problem(path: Path, event: Event, message: String): String = + event.startMark.map { mark -> "$path:${mark.line + 1}:${mark.column + 1}: $message" } + .orElse("$path: $message") + + private fun canonicalLocale(value: String): String { + val locale = try { + Locale.Builder().setLanguageTag(value).build() + } catch (exception: IllformedLocaleException) { + throw IllegalArgumentException("Invalid locale '$value': ${exception.message}", exception) + } + val canonical = locale.toLanguageTag() + require(canonical != "und" && canonical == value) { + "Locale '$value' is not a canonical BCP 47 tag; use '$canonical'" + } + return canonical } - private fun localeOf(path: Path): String? { - val match = localeSuffix.find(path.nameWithoutExtension) ?: return null - return match.groupValues[1].replace('_', '-') + private fun containsYaml(directory: Path): Boolean = Files.walk(directory).use { paths -> + paths.anyMatch { Files.isRegularFile(it) && it.extension == "yaml" } } private fun suggestion(value: String, candidates: Collection): String { @@ -140,9 +430,54 @@ internal class MessageCatalog private constructor( return previous[right.length] } - private val bundlePattern = Regex("(?:messages|[A-Za-z0-9_-]+-msgs)(?:_[a-z]{2}(?:_[A-Z]{2})?)?\\.properties") - private val localeSuffix = Regex("_([a-z]{2}(?:_[A-Z]{2})?)$") - private val placeholderPattern = Regex("\\{(\\d+)}") - private const val fileLabel = "Message bundle" + private val keyPartPattern = Regex("[A-Za-z][A-Za-z0-9_-]*") + private val argumentPattern = Regex("[A-Za-z_][A-Za-z0-9_]*") + private val variantPattern = Regex("[A-Za-z0-9][A-Za-z0-9_-]*") + private val pluralVariants = setOf("zero", "one", "two", "few", "many", "other") + private val pluralCategories = buildMap { + listOf( + "af", "bg", "da", "de", "el", "en", "eo", "et", "eu", "fi", "fo", "hu", + "is", "nb", "nl", "nn", "no", "sq", "sv", "sw", + ).forEach { put(it, setOf("one", "other")) } + listOf("fr", "pt", "es", "ca", "gl", "it").forEach { put(it, setOf("one", "many", "other")) } + listOf("cs", "sk", "bs", "hr", "sr", "lt", "ro").forEach { + put(it, setOf("one", "few", "other")) + } + put("pl", setOf("one", "few", "many", "other")) + put("sl", setOf("one", "two", "few", "other")) + put("lv", setOf("zero", "one", "other")) + put("ga", setOf("one", "two", "few", "many", "other")) + put("cy", pluralVariants) + put("gd", setOf("one", "two", "few", "other")) + } + private const val MAX_CATALOG_CODE_POINTS = 3 * 1024 * 1024 + } +} + +internal fun MessageValue.usesPlural(): Boolean = when (this) { + is MessagePattern -> false + is MessageSelection -> kind == MessageArgumentKind.NUMBER || variants.values.any(MessageValue::usesPlural) +} + +private fun MessageValue.pluralSelections(): List = when (this) { + is MessagePattern -> emptyList() + is MessageSelection -> (if (kind == MessageArgumentKind.NUMBER) listOf(this) else emptyList()) + + variants.values.flatMap(MessageValue::pluralSelections) +} + +internal fun MessageValue.selectVariants(argument: String): Set = when (this) { + is MessagePattern -> emptySet() + is MessageSelection -> buildSet { + if (kind == MessageArgumentKind.SELECT && this@selectVariants.argument == argument) { + addAll(variants.keys - "other") + } + variants.values.forEach { addAll(it.selectVariants(argument)) } } } + +private fun mergeArgumentKinds(name: String, left: MessageArgumentKind, right: MessageArgumentKind): MessageArgumentKind = when { + left == right -> left + left == MessageArgumentKind.TEXT -> right + right == MessageArgumentKind.TEXT -> left + else -> error("Message argument '$name' is used as both ${left.name.lowercase()} and ${right.name.lowercase()}") +} diff --git a/compiler/src/main/kotlin/no/beint/thim/compiler/RendererGenerator.kt b/compiler/src/main/kotlin/no/beint/thim/compiler/RendererGenerator.kt index 1aa061f..b047427 100644 --- a/compiler/src/main/kotlin/no/beint/thim/compiler/RendererGenerator.kt +++ b/compiler/src/main/kotlin/no/beint/thim/compiler/RendererGenerator.kt @@ -48,9 +48,21 @@ internal class RendererGenerator( val modelName = model.qualifiedName?.asString() ?: error("$templateName: model must have a qualified name") val rendererName = modelName.replace(Regex("[^A-Za-z0-9_]"), "_") + "ThimRenderer" val code = CodeWriter(staticContent, registryName) - val locales = if (usesMessages(nodes)) catalog.locales() else emptySet() - regionalLocales = locales.filter { '-' in it }.withIndex().associate { (index, locale) -> locale to index + 1 } - languages = locales.filter { '-' !in it }.withIndex().associate { (index, locale) -> locale to index + 1 } + val locales = if (usesMessages(nodes)) { + catalog.supportedLocales.filterTo(linkedSetOf()) { it != catalog.defaultLocale } + } else { + emptySet() + } + val localeIds = if (locales.isEmpty()) { + emptyMap() + } else { + buildMap { + put(catalog.defaultLocale, 0) + locales.forEachIndexed { index, locale -> put(locale, index + 1) } + } + } + regionalLocales = localeIds.filterKeys { '-' in it } + languages = localeIds.filterKeys { '-' !in it } code.line("final class $rendererName {") code.indent { @@ -58,17 +70,24 @@ internal class RendererGenerator( code.line() code.line("static void render($modelName model, RenderContext context, HtmlOutput output) throws IOException {") code.indent { - if (regionalLocales.isNotEmpty()) { - val cases = regionalLocales.entries.joinToString(" ") { (locale, index) -> - "case \"${javaString(locale)}\" -> $index;" + if (regionalLocales.isNotEmpty() || languages.isNotEmpty()) { + val languageFallback = if (languages.isNotEmpty()) { + val cases = languages.entries.joinToString(" ") { (locale, index) -> + "case \"${javaString(locale)}\" -> $index;" + } + "switch (context.locale().getLanguage()) { $cases default -> 0; }" + } else { + "0" } - code.statement("var locale = switch (context.locale().toLanguageTag()) { $cases default -> 0; };") - } - if (languages.isNotEmpty()) { - val cases = languages.entries.joinToString(" ") { (locale, index) -> - "case \"${javaString(locale)}\" -> $index;" + val resolution = if (regionalLocales.isNotEmpty()) { + val cases = regionalLocales.entries.joinToString(" ") { (locale, index) -> + "case \"${javaString(locale)}\" -> $index;" + } + "switch (context.locale().toLanguageTag()) { $cases default -> $languageFallback; }" + } else { + languageFallback } - code.statement("var language = switch (context.locale().getLanguage()) { $cases default -> 0; };") + code.statement("var messageLocale = $resolution;") } val scope = Scope(model, recordUse = { property -> usedRootProperties.getOrPut(modelName, ::mutableSetOf).add(property) @@ -276,11 +295,11 @@ internal class RendererGenerator( } else if (safeHtml != null) { val attributeLocation = attributeLocation(element, "th:utext") if (safeHtml.trim().startsWith("#{")) { - val message = Expressions.message(safeHtml, diagnosticContext(attributeLocation, "THIM-MESSAGE-SYNTAX", "th:utext")) - requireDiagnostic(message.arguments.isEmpty(), "THIM-RAW-MESSAGE-ARGUMENTS", attributeLocation) { - "raw messages cannot contain arguments" - } - renderMessage(message, scope, code, diagnosticContext(attributeLocation, "THIM-MESSAGE", "th:utext"), attributeLocation, raw = true) + diagnostic( + "THIM-RAW-MESSAGE-UNSUPPORTED", + attributeLocation, + "localized messages are escaped text; use a non-null SafeHtml model property for th:utext", + ) } else { val value = scope.resolve( Expressions.path(safeHtml, diagnosticContext(attributeLocation, "THIM-EXPRESSION-SYNTAX", "th:utext")), @@ -628,43 +647,71 @@ internal class RendererGenerator( code: CodeWriter, context: String, location: SourceLocation?, - raw: Boolean = false, ) { - val definition = catalog.use(expression.key, expression.arguments.size, context) - val arguments = expression.arguments.map { - scope.resolve(it, diagnosticContext(location, "THIM-PROPERTY-UNKNOWN", "message argument"), location).code - } - val localized = definition.localized.filterValues { it != definition.base } - val regional = localized.filterKeys { '-' in it } - val languageValues = localized.filterKeys { '-' !in it } - - if (regional.isNotEmpty()) { - code.open("switch (locale)") - regional.forEach { (locale, value) -> - code.open("case ${regionalLocales.getValue(locale)} ->") - appendMessage(value, arguments, code, raw) - code.close() + val definition = catalog.use(expression.key, expression.arguments.keys, context) + val arguments = expression.arguments.mapValues { (name, path) -> + val resolved = scope.resolve(path, diagnosticContext(location, "THIM-PROPERTY-UNKNOWN", "message argument '$name'"), location) + requireDiagnostic(!resolved.nullable, "THIM-MESSAGE-ARGUMENT-NULLABLE", location) { + "message argument '$name' cannot be nullable" } - code.open("default ->") - } - if (languageValues.isNotEmpty()) { - code.open("switch (language)") - languageValues.forEach { (locale, value) -> - code.open("case ${languages.getValue(locale)} ->") - appendMessage(value, arguments, code, raw) - code.close() + when (definition.arguments.getValue(name)) { + MessageArgumentKind.NUMBER -> requireDiagnostic( + resolved.type.declaration.qualifiedName?.asString() in integralMessageTypes, + "THIM-MESSAGE-ARGUMENT-TYPE", + location, + ) { + "message argument '$name' is used for plural selection and requires an integral numeric property" + } + MessageArgumentKind.SELECT -> requireDiagnostic( + resolved.type.declaration.qualifiedName?.asString() in stringTypes || resolved.type.isEnum(), + "THIM-MESSAGE-ARGUMENT-TYPE", + location, + ) { + "message argument '$name' is used for selection and requires a String or enum property" + } + MessageArgumentKind.TEXT -> requireDiagnostic( + resolved.type.declaration.qualifiedName?.asString() in messageTextTypes || resolved.type.isEnum(), + "THIM-MESSAGE-ARGUMENT-TYPE", + location, + ) { + "message argument '$name' requires a String, number, Boolean, or enum property; prepare other display values in the page model" + } } - code.open("default ->") - appendMessage(definition.base, arguments, code, raw) - code.close() - code.close() - } else { - appendMessage(definition.base, arguments, code, raw) + if (definition.arguments.getValue(name) == MessageArgumentKind.SELECT && resolved.type.isEnum()) { + val declaration = resolved.type.declaration as KSClassDeclaration + val constants = declaration.declarations + .filterIsInstance() + .filter { it.classKind == ClassKind.ENUM_ENTRY } + .map { it.simpleName.asString() } + .toSet() + val variants = definition.values.values.flatMapTo(linkedSetOf()) { it.selectVariants(name) } + val unknown = variants - constants + requireDiagnostic(unknown.isEmpty(), "THIM-MESSAGE-SELECT-VARIANT", location) { + "message argument '$name' has variants ${unknown.sorted()} that are not constants of " + + (declaration.qualifiedName?.asString() ?: declaration.simpleName.asString()) + } + } + resolved + } + val defaultValue = definition.values.getValue(catalog.defaultLocale) + val localized = definition.values + .filterKeys { it != catalog.defaultLocale } + .filterValues { it != defaultValue || it.usesPlural() } + if (localized.isEmpty()) { + renderMessageValue(defaultValue, arguments, code, catalog.defaultLocale) + return } - if (regional.isNotEmpty()) { - code.close() + code.open("switch (messageLocale)") + localized.forEach { (locale, value) -> + val localeId = regionalLocales[locale] ?: languages.getValue(locale) + code.open("case $localeId ->") + renderMessageValue(value, arguments, code, locale) code.close() } + code.open("default ->") + renderMessageValue(defaultValue, arguments, code, catalog.defaultLocale) + code.close() + code.close() } private fun usesMessages(nodes: List): Boolean = nodes.any { node -> @@ -673,14 +720,46 @@ internal class RendererGenerator( ) } - private fun appendMessage(pattern: String, arguments: List, code: CodeWriter, raw: Boolean = false) { - var start = 0 - placeholderPattern.findAll(pattern).forEach { match -> - code.static(if (raw) pattern.substring(start, match.range.first) else escapeHtml(pattern.substring(start, match.range.first))) - code.statement("output.text(${arguments[match.groupValues[1].toInt()]});") - start = match.range.last + 1 + private fun renderMessageValue( + value: MessageValue, + arguments: Map, + code: CodeWriter, + locale: String, + ) { + when (value) { + is MessagePattern -> value.parts.forEach { part -> + when (part) { + is MessageText -> code.static(escapeHtml(part.value)) + is MessageArgument -> code.statement("output.text(${arguments.getValue(part.name).code});") + } + } + is MessageSelection -> { + val argument = arguments.getValue(value.argument) + when (value.kind) { + MessageArgumentKind.NUMBER -> { + val configured = java.util.Locale.forLanguageTag(locale) + code.open( + "switch (no.beint.thim.PluralRules.cardinal(" + + "\"${javaString(configured.language)}\", \"${javaString(configured.country)}\", ${argument.code}))", + ) + } + MessageArgumentKind.SELECT -> { + val selector = if (argument.type.isEnum()) "${argument.code}.name()" else argument.code + code.open("switch ($selector)") + } + MessageArgumentKind.TEXT -> error("Text arguments cannot select message variants") + } + value.variants.filterKeys { it != "other" }.forEach { (variant, selected) -> + code.open("case \"${javaString(variant)}\" ->") + renderMessageValue(selected, arguments, code, locale) + code.close() + } + code.open("default ->") + renderMessageValue(value.variants.getValue("other"), arguments, code, locale) + code.close() + code.close() + } } - code.static(if (raw) pattern.substring(start) else escapeHtml(pattern.substring(start))) } private fun urlCode( @@ -1090,7 +1169,6 @@ internal class RendererGenerator( private companion object { val eachPattern = Regex("([A-Za-z_][A-Za-z0-9_]*)\\s*:\\s*(\\$\\{.+})") - val placeholderPattern = Regex("\\{(\\d+)}") val voidElements = setOf("area", "base", "br", "col", "embed", "hr", "img", "input", "link", "meta", "source", "track", "wbr") val booleanAttributes = setOf( "allowfullscreen", "async", "autofocus", "autoplay", "checked", "controls", "default", "defer", @@ -1145,6 +1223,12 @@ internal class RendererGenerator( "java.lang.Float", "int", "long", "short", "byte", "double", "float", "java.math.BigDecimal", "java.math.BigInteger", ) + val integralMessageTypes = setOf( + "kotlin.Int", "kotlin.Long", "kotlin.Short", "kotlin.Byte", + "java.lang.Integer", "java.lang.Long", "java.lang.Short", "java.lang.Byte", + "int", "long", "short", "byte", + ) + val messageTextTypes = stringTypes + numericTypes + booleanFieldTypes fun javaString(value: String): String = buildString(value.length) { value.forEach { character -> diff --git a/compiler/src/main/kotlin/no/beint/thim/compiler/ThimProcessor.kt b/compiler/src/main/kotlin/no/beint/thim/compiler/ThimProcessor.kt index 7aa76cf..a9ded57 100644 --- a/compiler/src/main/kotlin/no/beint/thim/compiler/ThimProcessor.kt +++ b/compiler/src/main/kotlin/no/beint/thim/compiler/ThimProcessor.kt @@ -30,6 +30,11 @@ private class ThimProcessor( private val logger: KSPLogger = environment.logger private val templatesDirectory = requiredPath(environment, "thim.templates") private val messagesDirectory = requiredPath(environment, "thim.messages") + private val defaultLocale = environment.options["thim.defaultLocale"] ?: "en" + private val supportedLocales = environment.options["thim.supportedLocales"] + ?.split(',') + ?.map(String::trim) + ?: listOf(defaultLocale) private val generatedPackage = environment.options["thim.package"] ?: "thim.generated" private val registryName = environment.options["thim.registry"] ?: "ThimTemplates" private val modelPackages = environment.options["thim.modelPackages"] @@ -82,7 +87,7 @@ private class ThimProcessor( problems += checker.problems } - val catalog = MessageCatalog.load(messagesDirectory) + val catalog = MessageCatalog.load(messagesDirectory, defaultLocale, supportedLocales) val extractedRoutes = if (validateRoutes || generateRoutes) { RouteCatalog.load(resolver, trustedPaths) } else { diff --git a/compiler/src/test/kotlin/no/beint/thim/compiler/ExpressionsTest.kt b/compiler/src/test/kotlin/no/beint/thim/compiler/ExpressionsTest.kt new file mode 100644 index 0000000..dbe19fc --- /dev/null +++ b/compiler/src/test/kotlin/no/beint/thim/compiler/ExpressionsTest.kt @@ -0,0 +1,38 @@ +package no.beint.thim.compiler + +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertTrue + +class ExpressionsTest { + @Test + fun `message arguments are named`() { + val expression = Expressions.message( + "#{home.title(version=\${version}, userName=\${user.name})}", + "test", + ) + + assertEquals("home.title", expression.key) + assertEquals(listOf("version", "userName"), expression.arguments.keys.toList()) + assertEquals(listOf("user", "name"), expression.arguments.getValue("userName").segments.map { it.name }) + } + + @Test + fun `positional message arguments are rejected`() { + val problem = assertFailsWith { + Expressions.message("#{home.title(\${version})}", "test") + } + + assertTrue(problem.message.orEmpty().contains("must use name=\${property}")) + } + + @Test + fun `duplicate message arguments are rejected`() { + val problem = assertFailsWith { + Expressions.message("#{home.title(version=\${first}, version=\${second})}", "test") + } + + assertTrue(problem.message.orEmpty().contains("duplicate message argument 'version'")) + } +} diff --git a/compiler/src/test/kotlin/no/beint/thim/compiler/MessageCatalogTest.kt b/compiler/src/test/kotlin/no/beint/thim/compiler/MessageCatalogTest.kt new file mode 100644 index 0000000..22c74a6 --- /dev/null +++ b/compiler/src/test/kotlin/no/beint/thim/compiler/MessageCatalogTest.kt @@ -0,0 +1,206 @@ +package no.beint.thim.compiler + +import org.junit.jupiter.api.io.TempDir +import java.nio.file.Files +import java.nio.file.Path +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertIs +import kotlin.test.assertTrue + +class MessageCatalogTest { + @TempDir + lateinit var directory: Path + + @Test + fun `loads nested namespaces multiline strings and typed plural arguments`() { + write("en/home.yaml", """ + title: Thim {version} + introduction: |- + First line. + Second line. + inbox: + _plural: unreadCount + one: One unread message + other: "{unreadCount} unread messages" + """) + write("nb/home.yaml", """ + title: Thim {version} + introduction: |- + Første linje. + Andre linje. + inbox: + _plural: unreadCount + one: Én ulest melding + other: "{unreadCount} uleste meldinger" + """) + + val catalog = MessageCatalog.load(directory, "en", listOf("en", "nb")) + + assertEquals(MessageArgumentKind.TEXT, catalog.use("home.title", setOf("version"), "test").arguments["version"]) + assertEquals( + MessageArgumentKind.NUMBER, + catalog.use("home.inbox", setOf("unreadCount"), "test").arguments["unreadCount"], + ) + val introduction = catalog.use("home.introduction", emptySet(), "test") + .values.getValue("en") + assertEquals("First line.\nSecond line.", assertIs(assertIs(introduction).parts.single()).value) + } + + @Test + fun `failsafe schema keeps ambiguous scalars as text`() { + write("en/values.yaml", """ + negative: no + enabled: true + amount: 12 + date: 2026-08-08 + """) + + val catalog = MessageCatalog.load(directory, "en", listOf("en")) + + assertText(catalog, "values.negative", "no") + assertText(catalog, "values.enabled", "true") + assertText(catalog, "values.amount", "12") + assertText(catalog, "values.date", "2026-08-08") + } + + @Test + fun `loads select messages and nested selections`() { + write("en/account.yaml", """ + greeting: + _select: audience + MEMBER: Welcome back, {name} + other: Welcome, {name} + inbox: + _select: audience + MEMBER: + _plural: unreadCount + one: "{name}, you have one unread message" + other: "{name}, you have {unreadCount} unread messages" + other: "Welcome, {name}" + """) + + val catalog = MessageCatalog.load(directory, "en", listOf("en")) + + assertEquals( + mapOf("audience" to MessageArgumentKind.SELECT, "name" to MessageArgumentKind.TEXT), + catalog.use("account.greeting", setOf("audience", "name"), "test").arguments, + ) + assertEquals( + mapOf( + "audience" to MessageArgumentKind.SELECT, + "unreadCount" to MessageArgumentKind.NUMBER, + "name" to MessageArgumentKind.TEXT, + ), + catalog.use("account.inbox", setOf("audience", "unreadCount", "name"), "test").arguments, + ) + } + + @Test + fun `rejects duplicate mapping keys`() { + write("en/home.yaml", """ + title: First + title: Second + """) + + assertProblem("duplicate key 'title'") { MessageCatalog.load(directory, "en", listOf("en")) } + } + + @Test + fun `rejects YAML anchors tags and sequences`() { + write("en/home.yaml", "title: &shared Hello\ncopy: *shared") + assertProblem("anchors and aliases are not supported") { + MessageCatalog.load(directory, "en", listOf("en")) + } + + write("en/home.yaml", "title: !!str Hello") + assertProblem("YAML tags are not supported") { + MessageCatalog.load(directory, "en", listOf("en")) + } + + write("en/home.yaml", "title:\n - Hello") + assertProblem("YAML sequences are not supported") { + MessageCatalog.load(directory, "en", listOf("en")) + } + } + + @Test + fun `rejects short extensions and multiple documents`() { + write("en/home.yml", "title: Hello") + assertProblem("must use the .yaml extension") { + MessageCatalog.load(directory, "en", listOf("en")) + } + + Files.delete(directory.resolve("en/home.yml")) + write("en/home.yaml", "title: Hello\n---\ntitle: Again") + assertProblem("expected exactly one YAML document") { + MessageCatalog.load(directory, "en", listOf("en")) + } + } + + @Test + fun `requires the same keys and argument contract in every locale`() { + write("en/home.yaml", "title: Hello {name}\nsubtitle: Welcome") + write("nb/home.yaml", "title: Hei\nextra: Ekstra") + + assertProblem("locale 'nb' is missing [home.subtitle]") { + MessageCatalog.load(directory, "en", listOf("en", "nb")) + } + + write("nb/home.yaml", "title: Hei\nsubtitle: Velkommen") + assertProblem("changes the argument contract; missing [name]") { + MessageCatalog.load(directory, "en", listOf("en", "nb")) + } + } + + @Test + fun `requires canonical configured locale tags`() { + write("en/home.yaml", "title: Hello") + + assertProblem("not a canonical BCP 47 tag; use 'en-US'") { + MessageCatalog.load(directory, "en-us", listOf("en-us")) + } + } + + @Test + fun `rejects plural categories unreachable in the locale`() { + write("en/home.yaml", """ + inbox: + _plural: count + few: A few messages + other: "{count} messages" + """) + + assertProblem("unreachable plural categories [few]") { + MessageCatalog.load(directory, "en", listOf("en")) + } + } + + @Test + fun `allows projects with no message catalog`() { + val missing = directory.resolve("missing") + val catalog = MessageCatalog.load(missing, "en", listOf("en")) + + assertProblem("message 'home.title' does not exist") { + catalog.use("home.title", emptySet(), "test") + } + } + + private fun assertText(catalog: MessageCatalog, key: String, expected: String) { + val value = catalog.use(key, emptySet(), "test").values.getValue("en") + val pattern = assertIs(value) + assertEquals(expected, assertIs(pattern.parts.single()).value) + } + + private fun assertProblem(expected: String, block: () -> Unit) { + val problem = assertFailsWith(block = block) + assertTrue(problem.message.orEmpty().contains(expected), problem.message) + } + + private fun write(relative: String, contents: String) { + val path = directory.resolve(relative) + Files.createDirectories(path.parent) + Files.writeString(path, contents.trimIndent() + "\n") + } +} diff --git a/example/build.gradle.kts b/example/build.gradle.kts index 0379e5e..2e73f09 100644 --- a/example/build.gradle.kts +++ b/example/build.gradle.kts @@ -28,7 +28,9 @@ dependencies { ksp { arg("thim.templates", layout.projectDirectory.dir("src/main/resources/templates").asFile.absolutePath) - arg("thim.messages", layout.projectDirectory.dir("src/main/resources").asFile.absolutePath) + arg("thim.messages", layout.projectDirectory.dir("src/main/resources/i18n").asFile.absolutePath) + arg("thim.defaultLocale", "en") + arg("thim.supportedLocales", "en,nb") arg("thim.package", "no.beint.thim.example.generated") arg("thim.registry", "ExampleTemplates") arg("thim.generateRoutes", "true") @@ -45,8 +47,8 @@ tasks.withType().configureEach { }) .withPropertyName("thimTemplates") .withPathSensitivity(PathSensitivity.RELATIVE) - inputs.files(fileTree(layout.projectDirectory.dir("src/main/resources")) { - include("**/*.properties") + inputs.files(fileTree(layout.projectDirectory.dir("src/main/resources/i18n")) { + include("**/*.yaml") }) .withPropertyName("thimMessages") .withPathSensitivity(PathSensitivity.RELATIVE) diff --git a/example/src/main/kotlin/no/beint/thim/example/App.kt b/example/src/main/kotlin/no/beint/thim/example/App.kt index 0707138..9f27413 100644 --- a/example/src/main/kotlin/no/beint/thim/example/App.kt +++ b/example/src/main/kotlin/no/beint/thim/example/App.kt @@ -56,10 +56,11 @@ class HomeCtrl { greeting = "Typed models, compiled HTML, no runtime engine.", features = listOf( Feature("Safe", "Properties and messages are checked while the application compiles."), - Feature("Small", "The runtime contains three dependency-free Java types."), + Feature("Small", "The runtime is dependency-free Java."), Feature("Fast", "Generated code writes directly to the HTTP response."), ), showFooter = true, + unreadCount = 3, feedbackForm = FeedbackForm.empty(), ) diff --git a/example/src/main/kotlin/no/beint/thim/example/page/HomePage.kt b/example/src/main/kotlin/no/beint/thim/example/page/HomePage.kt index a3632df..4424d5f 100644 --- a/example/src/main/kotlin/no/beint/thim/example/page/HomePage.kt +++ b/example/src/main/kotlin/no/beint/thim/example/page/HomePage.kt @@ -9,6 +9,7 @@ data class HomePage( val greeting: String, val features: List, val showFooter: Boolean, + val unreadCount: Int, val feedbackForm: FeedbackForm, val errors: FormErrors = FormErrors.NONE, ) diff --git a/example/src/main/resources/i18n/en/home.yaml b/example/src/main/resources/i18n/en/home.yaml new file mode 100644 index 0000000..85a19ed --- /dev/null +++ b/example/src/main/resources/i18n/en/home.yaml @@ -0,0 +1,10 @@ +title: Thim {version} +health: Health check +details: Details +feedback: Send feedback +send: Send +author: Author +inbox: + _plural: unreadCount + one: One unread message + other: "{unreadCount} unread messages" diff --git a/example/src/main/resources/i18n/nb/home.yaml b/example/src/main/resources/i18n/nb/home.yaml new file mode 100644 index 0000000..84c2559 --- /dev/null +++ b/example/src/main/resources/i18n/nb/home.yaml @@ -0,0 +1,10 @@ +title: Thim {version} +health: Helsesjekk +details: Detaljer +feedback: Send tilbakemelding +send: Send inn +author: Forfatter +inbox: + _plural: unreadCount + one: Én ulest melding + other: "{unreadCount} uleste meldinger" diff --git a/example/src/main/resources/messages.properties b/example/src/main/resources/messages.properties deleted file mode 100644 index 275eb60..0000000 --- a/example/src/main/resources/messages.properties +++ /dev/null @@ -1,6 +0,0 @@ -home.title=Thim {0} -home.health=Health check -home.details=Details -home.feedback=Send feedback -home.send=Send -home.author=Author diff --git a/example/src/main/resources/messages_no.properties b/example/src/main/resources/messages_no.properties deleted file mode 100644 index 41eefee..0000000 --- a/example/src/main/resources/messages_no.properties +++ /dev/null @@ -1,6 +0,0 @@ -home.title=Thim {0} -home.health=Helsesjekk -home.details=Detaljer -home.feedback=Send tilbakemelding -home.send=Send inn -home.author=Forfatter diff --git a/example/src/main/resources/templates/home.html b/example/src/main/resources/templates/home.html index 602383b..a97f98f 100644 --- a/example/src/main/resources/templates/home.html +++ b/example/src/main/resources/templates/home.html @@ -3,12 +3,13 @@ - Thim + Thim
-

Thim

+

Thim

Greeting

+

Unread messages

  • diff --git a/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimCompile.java b/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimCompile.java index cb77b0f..9694bcd 100644 --- a/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimCompile.java +++ b/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimCompile.java @@ -17,6 +17,7 @@ import org.gradle.api.tasks.Internal; import org.gradle.api.tasks.OutputDirectory; import org.gradle.api.tasks.OutputFile; +import org.gradle.api.tasks.Optional; import org.gradle.api.tasks.PathSensitive; import org.gradle.api.tasks.PathSensitivity; import org.gradle.api.tasks.TaskAction; @@ -49,9 +50,16 @@ public ThimCompile(ExecOperations execOperations, FileSystemOperations fileSyste public abstract DirectoryProperty getTemplates(); @InputDirectory + @Optional @PathSensitive(PathSensitivity.RELATIVE) public abstract DirectoryProperty getMessages(); + @Input + public abstract Property getDefaultLocale(); + + @Input + public abstract ListProperty getSupportedLocales(); + @Classpath public abstract ConfigurableFileCollection getRunnerClasspath(); @@ -149,6 +157,8 @@ public void compile() throws IOException { var processorOptions = String.join(separator, "thim.templates=" + getTemplates().get().getAsFile().getAbsolutePath(), "thim.messages=" + getMessages().get().getAsFile().getAbsolutePath(), + "thim.defaultLocale=" + getDefaultLocale().get(), + "thim.supportedLocales=" + String.join(",", getSupportedLocales().get()), "thim.package=" + getGeneratedPackage().get(), "thim.registry=" + getRegistryName().get(), "thim.modelPackages=" + String.join(",", getModelPackages().get()), diff --git a/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimExtension.java b/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimExtension.java index 481087b..09e396f 100644 --- a/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimExtension.java +++ b/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimExtension.java @@ -9,6 +9,10 @@ public abstract class ThimExtension { public abstract DirectoryProperty getMessages(); + public abstract Property getDefaultLocale(); + + public abstract ListProperty getSupportedLocales(); + public abstract Property getGeneratedPackage(); public abstract Property getRegistryName(); diff --git a/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimPlugin.java b/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimPlugin.java index f7553ab..493db0a 100644 --- a/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimPlugin.java +++ b/gradle-plugin/src/main/java/no/beint/thim/gradle/ThimPlugin.java @@ -29,7 +29,9 @@ public final class ThimPlugin implements Plugin { public void apply(Project project) { var extension = project.getExtensions().create("thim", ThimExtension.class); extension.getTemplates().convention(project.getLayout().getProjectDirectory().dir("src/main/resources/templates")); - extension.getMessages().convention(project.getLayout().getProjectDirectory().dir("src/main/resources")); + extension.getMessages().convention(project.getLayout().getProjectDirectory().dir("src/main/resources/i18n")); + extension.getDefaultLocale().convention("en"); + extension.getSupportedLocales().convention(extension.getDefaultLocale().map(locale -> java.util.List.of(locale))); extension.getGeneratedPackage().convention(project.provider(() -> generatedPackage(project))); extension.getRegistryName().convention("ThimTemplates"); extension.getModelPackages().convention(project.provider(() -> java.util.List.of(defaultModelPackage(project)))); @@ -66,6 +68,8 @@ private void configureKotlinProject(Project project, ThimExtension extension) { var ksp = project.getExtensions().getByType(KspExtension.class); ksp.arg("thim.templates", extension.getTemplates().map(directory -> directory.getAsFile().getAbsolutePath())); ksp.arg("thim.messages", extension.getMessages().map(directory -> directory.getAsFile().getAbsolutePath())); + ksp.arg("thim.defaultLocale", extension.getDefaultLocale()); + ksp.arg("thim.supportedLocales", extension.getSupportedLocales().map(locales -> String.join(",", locales))); ksp.arg("thim.package", extension.getGeneratedPackage()); ksp.arg("thim.registry", extension.getRegistryName()); ksp.arg("thim.modelPackages", extension.getModelPackages().map(packages -> String.join(",", packages))); @@ -83,7 +87,7 @@ private void configureKotlinProject(Project project, ThimExtension extension) { task.getInputs().files(extension.getTemplates().map(directory -> htmlFiles(project, directory.getAsFile()))) .withPropertyName("thimTemplates") .withPathSensitivity(PathSensitivity.RELATIVE); - task.getInputs().files(extension.getMessages().map(directory -> propertyFiles(project, directory.getAsFile()))) + task.getInputs().files(extension.getMessages().map(directory -> yamlFiles(project, directory.getAsFile()))) .withPropertyName("thimMessages") .withPathSensitivity(PathSensitivity.RELATIVE); }); @@ -159,6 +163,8 @@ private void configureThimTask( task.getModelSources().from(modelSourceDirectories); task.getTemplates().set(extension.getTemplates()); task.getMessages().set(extension.getMessages()); + task.getDefaultLocale().set(extension.getDefaultLocale()); + task.getSupportedLocales().set(extension.getSupportedLocales()); task.getRunnerClasspath().from(runner); task.getProcessorClasspath().from(processor); task.getLibraries().from(main.getCompileClasspath()); @@ -201,8 +207,8 @@ private FileTree htmlFiles(Project project, java.io.File directory) { return project.fileTree(directory, files -> files.include("**/*.html")); } - private FileTree propertyFiles(Project project, java.io.File directory) { - return project.fileTree(directory, files -> files.include("**/*.properties")); + private FileTree yamlFiles(Project project, java.io.File directory) { + return project.fileTree(directory, files -> files.include("**/*.yaml")); } private String generatedPackage(Project project) { diff --git a/runtime/build.gradle.kts b/runtime/build.gradle.kts index 12e5b1c..fc87b03 100644 --- a/runtime/build.gradle.kts +++ b/runtime/build.gradle.kts @@ -2,3 +2,13 @@ plugins { `java-library` id("com.vanniktech.maven.publish") } + +dependencies { + testImplementation(platform("org.junit:junit-bom:6.0.3")) + testImplementation("org.junit.jupiter:junit-jupiter") + testRuntimeOnly("org.junit.platform:junit-platform-launcher") +} + +tasks.test { + useJUnitPlatform() +} diff --git a/runtime/src/main/java/no/beint/thim/PluralRules.java b/runtime/src/main/java/no/beint/thim/PluralRules.java new file mode 100644 index 0000000..daea9d4 --- /dev/null +++ b/runtime/src/main/java/no/beint/thim/PluralRules.java @@ -0,0 +1,104 @@ +package no.beint.thim; + +import java.util.Locale; +import java.util.Objects; + +/** Integer cardinal categories derived from Unicode CLDR 49 plural rules. */ +public final class PluralRules { + private PluralRules() {} + + public static String cardinal(Locale locale, long value) { + Objects.requireNonNull(locale); + return cardinal(locale.getLanguage(), locale.getCountry(), value); + } + + /** Used by generated renderers, which have already resolved the effective catalog locale. */ + public static String cardinal(String language, String country, long value) { + Objects.requireNonNull(language); + Objects.requireNonNull(country); + if (value < 0 && value != Long.MIN_VALUE) value = -value; + var mod10 = Math.abs(value % 10); + var mod100 = Math.abs(value % 100); + return switch (language) { + case "fr" -> { + if (value == 0 || value == 1) yield "one"; + if (value != 0 && value % 1_000_000 == 0) yield "many"; + yield "other"; + } + case "pt" -> { + var portugal = country.equals("PT"); + if (portugal ? value == 1 : value == 0 || value == 1) yield "one"; + if (value != 0 && value % 1_000_000 == 0) yield "many"; + yield "other"; + } + case "es", "ca", "gl", "it" -> { + if (value == 1) yield "one"; + if (value != 0 && value % 1_000_000 == 0) yield "many"; + yield "other"; + } + case "is" -> mod10 == 1 && mod100 != 11 ? "one" : "other"; + case "cs", "sk" -> { + if (value == 1) yield "one"; + if (value >= 2 && value <= 4) yield "few"; + yield "other"; + } + case "pl" -> { + if (value == 1) yield "one"; + if (mod10 >= 2 && mod10 <= 4 && !(mod100 >= 12 && mod100 <= 14)) yield "few"; + if (value != 1 && (mod10 <= 1 || mod10 >= 5 || mod100 >= 12 && mod100 <= 14)) yield "many"; + yield "other"; + } + case "ro" -> { + if (value == 1) yield "one"; + if (value == 0 || value != 1 && mod100 >= 1 && mod100 <= 19) yield "few"; + yield "other"; + } + case "sl" -> switch ((int) mod100) { + case 1 -> "one"; + case 2 -> "two"; + case 3, 4 -> "few"; + default -> "other"; + }; + case "bs", "hr", "sr" -> { + if (mod10 == 1 && mod100 != 11) yield "one"; + if (mod10 >= 2 && mod10 <= 4 && !(mod100 >= 12 && mod100 <= 14)) yield "few"; + yield "other"; + } + case "lv" -> { + if (mod10 == 0 || mod100 >= 11 && mod100 <= 19) yield "zero"; + if (mod10 == 1 && mod100 != 11) yield "one"; + yield "other"; + } + case "lt" -> { + if (mod10 == 1 && !(mod100 >= 11 && mod100 <= 19)) yield "one"; + if (mod10 >= 2 && mod10 <= 9 && !(mod100 >= 11 && mod100 <= 19)) yield "few"; + yield "other"; + } + case "ga" -> { + if (value == 1) yield "one"; + if (value == 2) yield "two"; + if (value >= 3 && value <= 6) yield "few"; + if (value >= 7 && value <= 10) yield "many"; + yield "other"; + } + case "cy" -> { + if (value == 0) yield "zero"; + if (value == 1) yield "one"; + if (value == 2) yield "two"; + if (value == 3) yield "few"; + if (value == 6) yield "many"; + yield "other"; + } + case "gd" -> { + if (value == 1 || value == 11) yield "one"; + if (value == 2 || value == 12) yield "two"; + if (value >= 3 && value <= 10 || value >= 13 && value <= 19) yield "few"; + yield "other"; + } + case "af", "bg", "da", "de", "el", "en", "eo", "et", "eu", "fi", "fo", + "hu", "nb", "nl", "nn", "no", "sq", "sv", "sw" -> value == 1 ? "one" : "other"; + default -> throw new IllegalArgumentException( + "Unsupported plural locale " + language + (country.isEmpty() ? "" : "-" + country)); + }; + } +} diff --git a/runtime/src/test/java/no/beint/thim/PluralRulesTest.java b/runtime/src/test/java/no/beint/thim/PluralRulesTest.java new file mode 100644 index 0000000..469626a --- /dev/null +++ b/runtime/src/test/java/no/beint/thim/PluralRulesTest.java @@ -0,0 +1,44 @@ +package no.beint.thim; + +import org.junit.jupiter.api.Test; + +import java.util.Locale; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class PluralRulesTest { + @Test + void selectsEnglishAndNorwegianCardinals() { + assertEquals("one", PluralRules.cardinal(Locale.ENGLISH, 1)); + assertEquals("one", PluralRules.cardinal(Locale.ENGLISH, -1)); + assertEquals("other", PluralRules.cardinal(Locale.ENGLISH, 0)); + assertEquals("other", PluralRules.cardinal(Locale.forLanguageTag("nb"), 2)); + } + + @Test + void selectsPolishCardinals() { + var locale = Locale.forLanguageTag("pl"); + assertEquals("one", PluralRules.cardinal(locale, 1)); + assertEquals("few", PluralRules.cardinal(locale, 2)); + assertEquals("many", PluralRules.cardinal(locale, 5)); + assertEquals("many", PluralRules.cardinal(locale, 12)); + assertEquals("few", PluralRules.cardinal(locale, 22)); + } + + @Test + void selectsLessCommonCardinalCategories() { + assertEquals("many", PluralRules.cardinal(Locale.FRENCH, 1_000_000)); + assertEquals("many", PluralRules.cardinal(Locale.forLanguageTag("gl"), 1_000_000)); + assertEquals("other", PluralRules.cardinal(Locale.forLanguageTag("pt-PT"), 0)); + assertEquals("other", PluralRules.cardinal("pt", "PT", 0)); + assertEquals("few", PluralRules.cardinal(Locale.forLanguageTag("ro"), 101)); + assertEquals("two", PluralRules.cardinal(Locale.forLanguageTag("sl"), 102)); + assertEquals("zero", PluralRules.cardinal(Locale.forLanguageTag("lv"), 10)); + } + + @Test + void rejectsLocalesWithoutEmbeddedRules() { + assertThrows(IllegalArgumentException.class, () -> PluralRules.cardinal(Locale.JAPANESE, 1)); + } +}