Skip to content

Redesign CosineEmbeddingLoss to fit the unified loss compute path #3

Description

@MoonFlowww

Motivation

The unified loss dispatch path in Model calls compute(descriptor, prediction, target, weight) with a single prediction tensor. CosineEmbeddingLoss fundamentally needs two separate embedding inputs (x1, x2). Right now callers must pack both into a stacked tensor shaped [B, 2, D…] or [2, B, D…] — an undocumented, fragile convention that nothing in the public API communicates.

This makes it impossible to use CosineEmbeddingLoss through the standard Model::train() path without prior knowledge of the packing trick, breaking the "safe API" promise.

Current State

src/loss/details/cosine_embedding.hpp:9 carries:

/// TODO: rework
/// - using single path : std::visit([&](const auto& d){ return Loss::Details::compute(d, prediction, target, weight); }, *loss_descriptor_);
///      --> Cos Embedd expect 2 inputs

The current workaround silently accepts prediction.size(1) == 2 or prediction.size(0) == 2 and splits — no compile-time or runtime API contract enforces this.

Proposed Change

Two viable approaches:

Option A — optional second input slot in the unified compute path:

torch::Tensor compute(const CosineEmbeddingDescriptor& d,
                      const torch::Tensor& prediction,
                      const torch::Tensor& target,
                      const std::optional<torch::Tensor>& weight = std::nullopt,
                      const std::optional<torch::Tensor>& input2  = std::nullopt);

input2 is only consulted by CosineEmbedding; all other losses ignore it.

Option B — PairTensor wrapper:

struct PairTensor { torch::Tensor x1, x2; };

Callers construct a PairTensor explicitly. The loss dispatcher checks via if constexpr whether the descriptor requires a pair and unpacks accordingly.

Option B is cleaner long-term (no proliferating optional args) and makes the requirement visible at the call site.

Acceptance Criteria

  • CosineEmbeddingLoss usable through the standard Model::train() path without manual tensor packing
  • The packing convention (if kept) is enforced and documented at the API boundary
  • /// TODO: rework comment removed
  • Existing behaviour (margin, reduction modes) preserved
  • No other loss functions affected

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions