Encoder path, quantized router index, and a catalog SPI

CHANGELOG.md

@@ -17,6 +17,49 @@ All notable changes to models are documented here.
   separately at 1e-3: cosine is scale-invariant, so a runtime that skips L2
   normalization agrees with a normalized reference at exactly 1.0.
 
+## [0.3.1] - 2026-08-08
+
+### Added
+
+- Added the `gemma-embedding` encoder architecture, which makes EmbeddingGemma-300M
+  runnable on the pure-Java backend. Bidirectional attention inverts the loop
+  nesting rather than changing a mask: every position needs every other
+  position's key at the same layer, so the sequence is the unit of work and
+  there is no KV cache. Verified against llama.cpp at 0.99956 minimum cosine,
+  where forcing causal masking measures 0.57266.
+
+- Added a pretrained task classifier for `models-router`, shipped as a
+  quantized index inside the jar. 1929 prompts over ten tasks, 0.9019 accuracy
+  on a held-out split, 0.65 MB. Training prompts live in
+  integrallis/model-router-corpus; what ships here is the derived index.
+
+- Added `ModelCatalogProvider`, a ServiceLoader SPI in `models-api` that lets
+  installed models describe themselves so callers need not hand-write price,
+  latency and per-task quality. `ModelRouter.discoverLocal()` consumes it.
+  Models without a performance profile for the current hardware are estimated
+  from measured peers rather than dropped, because local generation is
+  memory-bandwidth bound and so `tokensPerSecond * sizeBytes` is roughly fixed
+  on one machine.
+
+- Added `AppleFoundationModelsCatalog`, reporting Apple's on-device model when
+  the machine has one. Opt-in via `discoverLocal(true)`: it is present because
+  of the hardware, so discovering it by default would make identical code route
+  differently on a Mac than in production.
+
+- Added `matryoshkaDimensions` to `GgufEmbeddingBackend`, for models trained
+  with Matryoshka Representation Learning. Named for the technique rather than
+  called `dimensions` so that truncating a model never trained that way is not
+  something a caller reaches for by accident.
+
+### Changed
+
+- Encoder positions within a layer now run concurrently, taking a full router
+  index build from 1522 s to 300 s. Bit-exact: the parallel build produces a
+  `quantized.bin` with the same SHA-256 as the sequential one.
+
+- Raised the vectors dependency to 0.1.7 for quantized-only collections, which
+  store the classifier index as 4-bit codes with no full-precision copy.
+
 ## [0.3.0] - 2026-08-05
 
 ### Added

README.md

@@ -139,16 +139,16 @@ directly:
 
 ```kotlin
 dependencies {
-    implementation("com.integrallis:models:0.3.0")
-    implementation("com.integrallis:backend-java:0.3.0") // or backend-native
+    implementation("com.integrallis:models:0.3.1")
+    implementation("com.integrallis:backend-java:0.3.1") // or backend-native
 }
 ```
 
 Use Apple's on-device system model on a supported Apple Silicon Mac:
 
 ```kotlin
 dependencies {
-    implementation("com.integrallis:backend-apple:0.3.0")
+    implementation("com.integrallis:backend-apple:0.3.1")
 }
 ```
 

backend-java/gradle.lockfile

@@ -7,7 +7,7 @@ com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs
 com.google.code.findbugs:jsr305:3.0.2=spotbugs
 com.google.code.gson:gson:2.13.2=spotbugs
 com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
-com.integrallis:vectors-core:0.1.5=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
+com.integrallis:vectors-core:0.1.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
 commons-io:commons-io:2.20.0=spotbugs
 jaxen:jaxen:2.0.0=spotbugs
 net.bytebuddy:byte-buddy-agent:1.17.7=testCompileClasspath,testRuntimeClasspath

backend-java/src/main/java/com/integrallis/models/backend/purejava/EncoderDecoderAdapter.java

@@ -0,0 +1,141 @@
+/*
+ * Copyright 2025-2026 Integrallis Software, LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.integrallis.models.backend.purejava;
+
+import com.integrallis.models.api.LogitBatch;
+import com.integrallis.models.backend.purejava.llama.EncoderForwardPass;
+import java.util.Objects;
+
+/**
+ * Presents an encoder through the backend's decoder contract.
+ *
+ * <p>Almost every method here refuses. That is the honest adaptation: an encoder has no language
+ * model head and no notion of a next token, so there is no meaningful value to return from {@code
+ * forward} or {@code verify}. Refusing loudly beats returning a zero vector or the pooled embedding
+ * under a generative name, either of which would let a caller build something that appears to
+ * generate and does not.
+ */
+final class EncoderDecoderAdapter implements PureJavaDecoder {
+
+  private final EncoderForwardPass encoder;
+
+  EncoderDecoderAdapter(EncoderForwardPass encoder) {
+    this.encoder = Objects.requireNonNull(encoder, "encoder");
+  }
+
+  @Override
+  public float[] embedSequence(int[] tokens) {
+    return encoder.encode(tokens);
+  }
+
+  @Override
+  public boolean supportsSequenceEmbedding() {
+    return true;
+  }
+
+  @Override
+  public int maxBatchSize() {
+    return 1;
+  }
+
+  @Override
+  public void rewind(int checkpoint) {
+    // An encoder holds no state between sequences, so there is nothing to rewind to.
+  }
+
+  @Override
+  public void reset() {
+    // Likewise nothing to reset: every sequence is encoded from scratch.
+  }
+
+  @Override
+  public int checkpoint() {
+    return 0;
+  }
+
+  @Override
+  public void close() {}
+
+  @Override
+  public float[] forward(int token, int position) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public float[] forwardTransient(int token, int position) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public float[] prefill(int[] tokens, int startPosition) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public Session openSession() {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public float[] forward(Session session, int token, int position) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public float[] forwardTransient(Session session, int token, int position) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public float[] prefill(Session session, int[] tokens, int startPosition) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public LogitBatch forwardBatch(Session[] sessions, int[] tokens) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public LogitBatch forwardBatchTransient(Session[] sessions, int[] tokens) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public void rewind(Session session, int checkpoint) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public void reset(Session session) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public LogitBatch verify(int[] tokens, int startPosition) {
+    throw generationUnsupported();
+  }
+
+  @Override
+  public LogitBatch verifyTransient(int[] tokens, int startPosition) {
+    throw generationUnsupported();
+  }
+
+  private static UnsupportedOperationException generationUnsupported() {
+    return new UnsupportedOperationException(
+        "this model is an encoder: it has no language model head and cannot generate tokens");
+  }
+}

backend-java/src/main/java/com/integrallis/models/backend/purejava/GgufEmbeddingBackend.java

@@ -32,6 +32,11 @@
  * embedding model. Getting them wrong does not fail; it quietly degrades retrieval, which is a far
  * worse failure mode than an exception.
  *
+ * <p>Some models leave no such choice. An encoder declares its pooling in metadata and may carry a
+ * dense projection head, so the vector it was trained to produce is a function of the whole
+ * sequence and of weights the caller cannot see. Those models are handed the sequence whole and
+ * pool themselves; only normalization stays a caller decision.
+ *
  * <p>Not thread-safe: each call drives one sequence through shared backend state and resets it
  * between texts. Use one instance per thread, or guard it.
  */
@@ -40,20 +45,40 @@ public final class GgufEmbeddingBackend implements EmbeddingBackend {
   private final PureJavaBackend backend;
   private final Pooling pooling;
   private final boolean normalize;
+  private final boolean modelOwnsPooling;
   private final int dimension;
   private boolean closed;
 
   private GgufEmbeddingBackend(Builder builder) {
     this.backend = builder.backend;
     this.pooling = builder.pooling;
     this.normalize = builder.normalize;
-    this.dimension = backend.metadata().embeddingDim();
-    if (!backend.supportsHiddenState()) {
+    this.modelOwnsPooling = backend.supportsSequenceEmbedding();
+    int full = backend.metadata().embeddingDim();
+    if (builder.matryoshkaDimensions > full) {
+      throw new IllegalArgumentException(
+          "cannot truncate to "
+              + builder.matryoshkaDimensions
+              + " dimensions: the model produces "
+              + full);
+    }
+    this.dimension = builder.matryoshkaDimensions > 0 ? builder.matryoshkaDimensions : full;
+    if (!modelOwnsPooling && !backend.supportsHiddenState()) {
       throw new IllegalArgumentException(
           "model architecture "
               + backend.metadata().modelFamily()
               + " does not expose hidden states for embedding");
     }
+    if (modelOwnsPooling && builder.poolingRequested) {
+      // Honouring the request would produce a vector of the right width computed the wrong way,
+      // which retrieval absorbs as slightly worse results rather than reporting.
+      throw new IllegalArgumentException(
+          "model architecture "
+              + backend.metadata().modelFamily()
+              + " pools internally and cannot be told to use "
+              + builder.pooling
+              + "; drop the pooling() call and let the model decide");
+    }
   }
 
   /** Starts configuring an embedding backend over an already-loaded model. */
@@ -74,7 +99,18 @@ public float[] embed(String text) {
     // Every text is an independent sequence; without this the previous one stays in the KV cache
     // and results depend on call order.
     backend.reset();
-    float[] pooled = pooling == Pooling.MEAN ? meanPooled(tokens) : lastTokenPooled(tokens);
+    float[] pooled;
+    if (modelOwnsPooling) {
+      pooled = backend.embedSequence(tokens);
+    } else {
+      pooled = pooling == Pooling.MEAN ? meanPooled(tokens) : lastTokenPooled(tokens);
+    }
+    // Truncate before normalizing: a Matryoshka prefix is only a unit vector once rescaled to its
+    // own length, and cosine over an unrescaled prefix is not the similarity the model was
+    // trained to produce.
+    if (dimension != pooled.length) {
+      pooled = java.util.Arrays.copyOf(pooled, dimension);
+    }
     if (normalize) {
       l2Normalize(pooled);
     }
@@ -157,15 +193,48 @@ public static final class Builder {
 
     private final PureJavaBackend backend;
     private Pooling pooling = Pooling.LAST_TOKEN;
+    private boolean poolingRequested;
     private boolean normalize = true;
+    private int matryoshkaDimensions;
+
+    /**
+     * Keeps only the first {@code dimensions} components, rescaling to unit length afterwards.
+     *
+     * <p>Valid <em>only</em> for models trained with Matryoshka Representation Learning, which
+     * arranges the vector so that each prefix is itself a usable embedding. Truncating any other
+     * model discards trained dimensions and degrades retrieval without failing.
+     *
+     * <p>No GGUF metadata records whether a model was trained this way, so nothing here can check
+     * it. Hence the name: a caller who does not already know what Matryoshka means will not reach
+     * for this method by accident, which a plain {@code dimensions()} would invite.
+     *
+     * <p>EmbeddingGemma publishes 768, 512, 256 and 128. Sizes between those are not wrong so much
+     * as unmeasured — the published quality figures only cover the listed widths.
+     *
+     * @param dimensions the prefix length to keep, at most the model's full width
+     */
+    public Builder matryoshkaDimensions(int dimensions) {
+      if (dimensions <= 0) {
+        throw new IllegalArgumentException("dimensions must be > 0: " + dimensions);
+      }
+      this.matryoshkaDimensions = dimensions;
+      return this;
+    }
 
     private Builder(PureJavaBackend backend) {
       this.backend = Objects.requireNonNull(backend, "backend");
     }
 
-    /** Defaults to {@link Pooling#LAST_TOKEN}, correct for causal decoder-only embedders. */
+    /**
+     * Defaults to {@link Pooling#LAST_TOKEN}, correct for causal decoder-only embedders.
+     *
+     * <p>Rejected by models that pool internally. Tracked separately from the value so that the
+     * default never counts as a request — a model that owns its pooling should not have to reject a
+     * choice the caller never made.
+     */
     public Builder pooling(Pooling pooling) {
       this.pooling = Objects.requireNonNull(pooling, "pooling");
+      this.poolingRequested = true;
       return this;
     }