Conversation
Breaking, but current fallback to Iterators.flatten caused more trouble than it was worth.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #60 +/- ##
==========================================
- Coverage 96.25% 92.22% -4.03%
==========================================
Files 8 13 +5
Lines 507 875 +368
==========================================
+ Hits 488 807 +319
- Misses 19 68 +49 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Breaking, but likely wasn't used a lot.
== compared internal fields directly, so vectors of arrays with equal elements but different data layout (offsets, unused data regions) compared unequal, contradicting both elementwise semantics and isequal. Compare element sizes and covered data instead, and specialize isequal consistently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
similar(A, Array{U,M}, dims) allocated data with element type U but then
wrapped it as ArrayOfSimilarArrays{T,...}, converting back to the old
element type. It also hard-coded the old number of outer dimensions
instead of using length(dims).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Third-party subtypes of AbstractArrayOfSimilarArrays previously fell
through to UnknownSplitMode, breaking flatview, stacked, innersize, etc.
Now subtypes only need to implement fused (plus the standard array API),
all split-mode operations are derived from it at the abstract type
level. Also tighten the element type bound of the abstract type to
AbstractArray{T,M}, which resolves the resulting method ambiguities.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
getsplitmode, unstackmode, fused, splitup, innermap and deepmap had generic Any fallbacks that gave non-array objects an UnknownSplitMode and confusing split-mode errors. Non-arrays now get a plain MethodError. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The split mode shared elem_ptr and kernel_size with the vector of arrays it came from, so resizing the vector silently mutated the mode. getsplitmode now makes a defensive copy (package extensions may override this for vector types that cannot be resized). SplitParts gains structural == and hash, since its instances no longer share field identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sum, mean, var and std specializations had no keyword arguments, so calls like mean(X; dims = 1) threw a MethodError instead of reaching the generic implementations. Dispatch on the value of dims instead, keeping the fast flat-data path for dims = Colon. Also add fill!(::ArrayOfSimilarArrays, x::AbstractArray), which the generic reduction machinery requires (and which is useful on its own). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a short guide on fused vs flatview vs stacked vs vecflattened and their memory-sharing behavior, and note the SVector variant of sliced in its docstring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SplitParts carries elem_ptr and kernel_size vectors, so split modes must be adaptable to other array storage (e.g. GPU) like the vectors of arrays they describe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pullbacks reconstruct nested tangents via splitup and give no tangent contribution to data not covered by partial partitions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- vcat and reduce(vcat, ...) of VectorOfSimilarArrays now concatenate the underlying data along its last dimension with a single allocation and return a VectorOfSimilarArrays (previously a generic Vector of views) - reduce(vcat, ...) of VectorOfArrays uses a single-pass implementation instead of pairwise reduction - vcat and append! of VectorOfArrays now only use the data covered by the element arrays, both silently corrupted the result for inputs with unused data regions (e.g. from partial partitions). append! to a target with unused trailing data now throws. - vcat(V::VectorOfArrays) returned V itself, it now returns an independent copy like Base.vcat This (mostly) upstreams LegendDataTypes.fast_flatten: it now amounts to reduce(vcat, xs) for vectors of ArraysOfArrays types, and to flatview(reduce(vcat, map(x -> sliced(x, Val(ndims(x) - 1)), xs))) for generic arrays concatenated along their last dimension. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FixedSizeVectors cannot be resized, so a VectorOfArrays that uses them for elem_ptr and kernel_size has an immutable shape and getsplitmode can safely share the shape information instead of copying it. partitioned(A, lengths::FixedSizeVector) automatically results in fixed-size shape information via similar. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The element-type and split-mode rework is breaking. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mooncake differentiates the ArraysOfArrays types and operations without custom rules (verified end-to-end against Mooncake's rule test machinery), so the extension only adds zero-derivative rules for the functions that query shape/structure information and are not differentiable with respect to array contents. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The consistency checks performed scalar indexing on elem_ptr, so VectorOfArrays with GPU-resident shape information could only be constructed with checks disabled. The checks now route through two helper functions that a new GPUArraysCore extension specializes: the two O(1) boundary reads run under @allowscalar and the O(n) partition check uses a vectorized formulation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Kernels over an ArrayOfSimilarArrays or VectorOfArrays run on the backend of the underlying data. The shape information of a VectorOfArrays may deliberately live on the host while its data lives on a device, so no cross-field consistency is required. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Test both nested array types over JLArrays (the reference AbstractGPUArray implementation) with scalar indexing disallowed, and document the host-side and device-side VectorOfArrays layouts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Mooncake tests use Random, which was only available transitively. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mapat(f, Val(depth), As...) generalizes map/innermap/deepmap to arbitrary nesting depth and multiple (structure-aligned) inputs, similar to depth/axis-targeted operations in Python's AwkwardArrays, but with array-of-arrays nesting semantics. For split arrays it descends the split-mode stack and operates on the underlying flat data, in a single (GPU-compatible) operation per nesting level. innersizes and innerlengths return the per-element sizes/lengths of nested arrays without requiring equal element sizes, with vectorized (GPU-compatible) implementations for VectorOfArrays. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bcastat(f, Val(depth), args...) broadcasts f over the contents of nested arrays at a given nesting depth, with AwkwardArrays-like alignment but array-of-arrays nesting semantics: aligned nested arguments, one value per element of shallower levels (broadcast over everything below), and scalars. For split arrays each nesting level compiles to a single flat (GPU-compatible) broadcast, using a segmented gather to expand outer-aligned values over ragged element contents. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per-element reductions over the contents of nested arrays. For ArrayOfSimilarArrays this reduces over the inner dimensions of the flat data, for GPU-resident VectorOfArrays a new GPUArraysCore-plus- KernelAbstractions extension runs a single-pass segmented reduction kernel (moving host shape information to the device as required). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
VectorOfArrays and AbstractArrayOfSimilarArrays get a broadcast style: outer-level broadcasts over a single outer dimension whose results are arrays now return a VectorOfArrays (with contiguous storage) instead of a Vector of individual arrays. The semantics of broadcasting are unchanged, f still receives whole element arrays, and all other cases fall back to the default broadcast machinery. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Convenience methods that wrap the depth in a Val immediately, relying on constant propagation for type stability (verified via inference tests through a function barrier), like sliced already does for the number of inner dimensions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.