Skip to content

Feature/new tiered int#11

Open
NikolayChirkov wants to merge 50 commits into
mainfrom
feature/new-tiered-int
Open

Feature/new tiered int#11
NikolayChirkov wants to merge 50 commits into
mainfrom
feature/new-tiered-int

Conversation

@NikolayChirkov

Copy link
Copy Markdown
Collaborator

No description provided.

NikolayChirkov and others added 27 commits June 27, 2026 16:20
Switch to TieredInt<StartSize, tierMax...> with buffer and stream serialization, restore comparison operators and std traits, and update tests for the new encoding.

Co-authored-by: Cursor <cursoragent@cursor.com>
Partial deserialization is no longer needed now that DeserializeFrom reads the full encoded value in one pass.

Co-authored-by: Cursor <cursoragent@cursor.com>
Base cell width (1, 2, or 4 bytes) now drives minimum encoded size and tier multipliers, with compile-time validation and tests for StartSize 2 and 4.

Co-authored-by: Cursor <cursoragent@cursor.com>
Select runtime storage from kMaxEncodable, enforce compile-time assignment bounds with if consteval, and fix test includes for libc++ Clang 20.

Co-authored-by: Cursor <cursoragent@cursor.com>
Use StartSize * 2^(NumTiers-1) instead of a global StartSize*8 cap, enforce
the 8-byte wire limit at compile time, and add size static_asserts for all
test TieredInt aliases.

Co-authored-by: Cursor <cursoragent@cursor.com>
Cover every tier boundary where serialized size increases, plus kUpper
at kMaxWireBytes for T1–T6 via buffer and stream round-trips.

Co-authored-by: Cursor <cursoragent@cursor.com>
Use TieredInt<std::uint16_t, ...> / std::int16_t instead of int StartSize,
keep TieredIntFromStartSize aliases, and implement native signed LE encoding
with tier-1 direct values and sign-bit negative overflow.

Co-authored-by: Cursor <cursoragent@cursor.com>
Compare signed and unsigned inputs in separate branches using int64_t
and uint64_t respectively, and add a compile-time T6 kUpper assignment test.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign was incorrectly stored in the header MSB, colliding with large positive overflow headers; encode signed values via ZigZag and reuse the unsigned tier path.

Co-authored-by: Cursor <cursoragent@cursor.com>
Split check_value by signedness before casting, require buffer length in Deserialize, and add signed 3/4-tier tests including extremes above 2^31.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add saturated single-cell mode when the first wire tier fills the header, use safe mixed signed/unsigned compare, propagate cxx_std_23 on the numeric target, and return FixedPoint::Cast by value.

Co-authored-by: Cursor <cursoragent@cursor.com>
Provide a lazy input-iterator view for variable-length TieredInt wire data and throw on truncated deserialize instead of asserting at runtime.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the old Q-format API and AE_FIXED macros in favor of compile-time
range mapping, integer-only runtime arithmetic, and numeric_traits support.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add BoundRatio for compile-time bound conversion, cross-type Cast, multiply/divide,
and min/max/clamp helpers; default tests to MacPorts Clang 20.

Co-authored-by: Cursor <cursoragent@cursor.com>
Max is now a required logical bound with compile-time kScaleExp selection; add/sub, cast, and div_to use shift-based integer arithmetic only.

Co-authored-by: Cursor <cursoragent@cursor.com>
Exponential stores wire codes with compile-time magnitude tables; text_io adds to_chars/FromString for TieredInt, FixedPoint, and Exponential without runtime float.

Co-authored-by: Cursor <cursoragent@cursor.com>
wire_traits and Serialize/Deserialize cover built-in integers, TieredInt, FixedPoint, and Exponential with short-buffer rejection via std::out_of_range.

Co-authored-by: Cursor <cursoragent@cursor.com>
Introduce runtime_numeric_traits with opt-in float/double backend, default BoundaryCode to Wire kRawMax, split exponential wire IO from core headers, and extend wire/type-size tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
Require extension header space and compile-time boundary checks, remove saturated single-cell mode and start-size aliases, and add stack-buffer ostream printing for TieredInt and FixedPoint.

Co-authored-by: Cursor <cursoragent@cursor.com>
Make ordinary numeric construction logical-valued and compile-time
checked for constants, preventing silent runtime clamping through the
normal constructor:

- FixedPoint: consteval integral/floating constructors with compile-time
  range checks; explicit runtime APIs FromRuntimeInteger/Saturating/
  TryFromRuntimeInteger for clamping or checked conversion.
- Exponential: logical-value consteval constructors (integer range
  checked, float consteval-only), explicit FromCode/Code raw-code
  construction, value() accessor, and runtime conversion APIs. Wire IO
  deserializes via FromCode.
- Add compile-fail tests verifying out-of-range constants and runtime
  float construction are rejected at compile time.

Co-authored-by: Cursor <cursoragent@cursor.com>
PackedRing<T, StorageBytes> is a byte ring that stores serialized T
values back to back, with no per-record length headers. It relies on T
being self-delimiting on the wire: the bytes_read reported by
Deserialize is enough to skip from one record to the next.

- Index types (head/tail/used/count) use SmallestUIntForSize<StorageBytes>
  rather than std::size_t.
- push evicts the oldest records until a value fits, or fails if the
  value alone exceeds capacity. Records may straddle the physical buffer
  end; reads reassemble them through a stack buffer before decoding.
- Iterator yields decoded T values by value, never by reference.
- FixedPoint serializes/skips its raw value and Exponential its code(),
  reusing the existing wire_traits, so the ring needs no per-type code.
- Add wire_io SerializedSizeAt<T> to report a record's byte length.
- Tests cover TieredInt/FixedPoint/Exponential payloads, eviction,
  wraparound, and the index type selection.

Co-authored-by: Cursor <cursoragent@cursor.com>
Verify FixedPoint<TieredInt>, Exponential variants, and PackedRing
composition through wire round-trips, ring behavior, and trait checks.
Split Exponential-heavy tests into separate files to keep compile times
practical; FutureCode Exponential tests use an explicit BoundaryCode
because the default kRawMax would build an enormous magnitude table.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add fixed_math primitives and rewrite encode/decode to use arithmetic mapping
with safe DefaultBoundaryCode, avoiding compile-time tables proportional to
BoundaryCode. Update Exponential and composed tests for two-byte wire ranges.

Co-authored-by: Cursor <cursoragent@cursor.com>
Replace hardcoded log/exp iteration counts and work types with ExponentialMathPolicy, fix encode rounding, and add policy and boundary tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
Clamp raw values in int64 space before rep cast, divide wide multiply products in the intermediate type, and strengthen policy boundary tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
- make integral FixedPoint operator+ consteval
- remove GCC -Wtype-limits test warnings
- verify GCC, Clang and MSVC builds
Comment thread CMakeLists.txt Outdated
option(AE_NUMERIC_INSTALL "Install numeric library" ${IS_ROOT_PROJECT} )

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD 23)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be no more than c++20

Comment thread numeric/tiered_int.h Outdated
static constexpr std::uint64_t kHeaderMax =
WireCellTraits<WireCell>::kHeaderMax;

static constexpr std::uint64_t value = []() constexpr {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to use std::invoke to implement immediate invoke pattern

NikolayChirkov and others added 4 commits June 29, 2026 19:35
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@NikolayChirkov

Copy link
Copy Markdown
Collaborator Author

Addressed the review comments in the latest commits:

  • switched the project target to C++20
  • removed/avoided C++23-only usage in tiered_int
  • updated immediate-invoke code to use std::invoke where appropriate
  • renamed static constexpr members to kCamelCase except STL-like value/type members
  • changed byte-count parameters to std::size_t
  • removed the always-true uint64_t max assertion
  • replaced SFINAE-style constraints with requires where appropriate
  • added noexcept to pure integer/constexpr helpers where safe
  • updated function naming to CamelCase
  • added/adjusted asserts for actual invariants
  • applied east const in the reviewed area
  • constrained stream support with requires/operator<< checks

CI is green now, including Windows MSVC, Windows MinGW, Ubuntu GCC, macOS Apple-Clang, and cpplint.

Please re-review when you have a chance.

Comment thread .github/workflows/ci-cd-tests.yml Outdated
Comment thread .github/workflows/ci-cd-tests.yml Outdated
Comment thread numeric/tiered_int.h Outdated
static constexpr std::uint64_t kHeaderMax =
WireCellTraits<WireCell>::kHeaderMax;

static constexpr std::uint64_t kValue = std::invoke([]() constexpr {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You said

renamed static constexpr members to kCamelCase except STL-like value/type members

Comment thread numeric/tiered_int.h Outdated

static constexpr bool kIsSigned =
tiered_int_internal::WireCellTraits<WireCell>::kIsSigned;
static constexpr int NumTiers = sizeof...(TierMaxVals) + 1;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You said

renamed static constexpr members to kCamelCase except STL-like value/type members

Comment thread numeric/tiered_int.h Outdated
if constexpr (std::is_signed_v<U>) {
if (v < 0) {
if (std::is_constant_evaluated()) {
throw std::invalid_argument("TieredInt value must be non-negative");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do this compile with -fno-exceptions?

Comment thread numeric/tiered_int_view.h Outdated
@@ -0,0 +1,155 @@
/*
* Copyright 2025 Aethernet Inc.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IT'S 2026 already!

Comment thread numeric/tiered_int_view.h Outdated
}

if (current_ == end_) {
throw std::out_of_range("TieredIntView iterator dereferenced at end");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do this compile with -fno-exceptions ?

NikolayChirkov and others added 9 commits June 30, 2026 10:34
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Use windows-latest, Visual Studio 18 2026 with -A x64, unified configure/build/test steps, and the same MSVC/MinGW environment setup as aether-client-cpp.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Merge remote README rewrite and document kMaxBoundaryCode default.

Co-authored-by: Cursor <cursoragent@cursor.com>
@NikolayChirkov

Copy link
Copy Markdown
Collaborator Author

Anton, please re-review when you have a chance.

I addressed the previous review comments and did an additional pass across the changed code, not only on the exact commented lines:

  • kept the project at C++20;
  • removed C++23-only usage;
  • kept numeric core exception-free: no throw / stdexcept in numeric core;
  • fixed constexpr validation so it works on GCC/MinGW without diagnostic pragma tricks;
  • kept -fno-exceptions compatibility;
  • restored required CI check names;
  • kept MSVC on Visual Studio generator;
  • removed duplicated workflow build/test steps where possible;
  • applied kCamelCase for ordinary static constexpr members, while keeping STL-like value/type in traits;
  • changed byte counts to std::size_t;
  • replaced SFINAE-style stream checks with requires/concepts;
  • added noexcept where safe;
  • applied CamelCase function naming and east const in touched numeric code;
  • updated assertions/preconditions where invalid use is a contract violation.

CI is green now.

I also rewrote the README manually after the API/documentation changes:

  • clarified TieredInt serialization tiers and chunk sizes;
  • fixed the wrong “first template parameter is the wire cell type” explanation;
  • documented FixedPoint scale placement, including cases where the binary point is far outside the raw storage type;
  • removed old local compiler paths from build docs;
  • moved Exponential-over-TieredInt examples into Combined Types;
  • added practical packed-size examples for payload sizes, latency exponential codes, and linear ping FixedPoint over TieredInt.

Please take another look.

Comment thread numeric/exponential.h Outdated
}

template <typename RuntimeT>
constexpr RuntimeT RuntimeSub(const RuntimeT& lhs, const RuntimeT& rhs) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

east const is better

Comment thread numeric/exponential.h Outdated
return WorkT::FromInteger(0);
}

const int exponent = static_cast<int>((bits >> 23U) & 0xFFU) - 127;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why int and not int32_t ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you are manipulating bits using int and other types without fixes size are not allowed.

Comment thread numeric/exponential.h Outdated
}

template <typename WorkT>
constexpr WorkT FloatToWork(float value) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this functions should be
FloatingToWork with to template parameters TFloating and TWork.
And there must be specializations for TFloating like general float and double; and float16_t float32_t may be added in future.

And the same for WorkToFloating.

Call it like:

 FloatingToWork<WorkT>(float_value);
 WorkTFloating<FloatT>(workt_value);

Comment thread numeric/exponential.h Outdated
int bit = -1;
for (int i = 63; i >= 0; --i) {
if ((value >> static_cast<unsigned>(i)) & 1U) {
bit = i;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return i;

and return -1; in the end.

bit variable is redundant.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe binary search make sense here?

Comment thread ae-numeric/exponential.h
class Exponential {
public:
using runtime_type = RuntimeT;
using wire_type = WireT;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code-style

Comment thread numeric/fixed_point.h Outdated

template <>
struct PromoteRep<std::uint8_t, std::uint8_t> {
using type = std::uint8_t;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's max size Rep, isn't it?

maibe:

using type = std::conditional_t<(sizeof(Rep1) > sizeof(Rep2)), Rep1, Rep2>;

?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And additionally we should properly handle different signess

Comment thread numeric/wire_io.h Outdated
template <typename T>
struct DeserializeResult {
T value;
std::size_t BytesRead;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code-style

Comment thread numeric/packed_ring.h Outdated
const std::size_t avail = AvailFrom(pos);
const std::size_t to_copy =
avail < kMaxRecordBytes ? avail : kMaxRecordBytes;
for (std::size_t i = 0; i < to_copy; ++i) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not copy by one byte.
There is a room for optimizations

  • pass pointer to storage_[pos] if pos + to_copy < kCapacity
  • copy from pos to kCapacity and from kCapacity to to_copy if it wraps. To memcpy instead of copy bytes one by one.

Comment thread numeric/packed_ring.h Outdated
for (std::size_t i = 0; i < to_copy; ++i) {
temp[i] = storage_[Wrap(static_cast<std::size_t>(pos) + i)];
}
return Deserialize<T>(temp, to_copy);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

temp + to_copy, isn't it a std::span?

Comment thread numeric/packed_ring.h Outdated
return true;
}

iterator begin() const {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this also could be constexpr

Comment thread numeric/packed_ring.h Outdated
}

bool operator==(const iterator& other) const {
return ordinal_ == other.ordinal_;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we compare pos_?

Any change to PackedRing leads to iterator invalidation. Because we don't know if element on pos_ still valid. The same true for Nth element we don't know if it's still the same element.
But ordinal_ is additional field used for compare only, it's redundant.

@NikolayChirkov

Copy link
Copy Markdown
Collaborator Author

@BartolomeyKant, please re-review when you have a chance.

I addressed the latest comments and applied the same cleanup across analogous code:

  • renamed the CMake target and public include root from numeric to ae-numeric, so public includes are now #include <ae-numeric/...>;
  • updated install rules, README examples, tests, and the cpplint CI path accordingly;
  • replaced DeserializeResult::BytesRead with bytes_read;
  • rewrote the Exponential floating conversion through FloatingWorkCodec, FloatingToWork, and WorkToFloating, using fixed-width integer bit arithmetic;
  • replaced the FixedPoint PromoteRep specialization table with generic integral promotion, including mixed signedness handling, while preserving same-rep support for packed reps;
  • optimized PackedRing wrap copies with std::span plus memcpy and removed ordinal-only iterator comparison;
  • kept the code C++20 and exception-free.

Local checks passed with MacPorts Clang 20.1.8 and CMAKE_OSX_DEPLOYMENT_TARGET=13.3:

  • cmake --build build-mp-clang20-macos13.3-ae --config Debug --parallel;
  • ctest --test-dir build-mp-clang20-macos13.3-ae --output-on-failure - 5/5 passed;
  • cpplint --recursive ./ae-numeric.

GitHub Actions are green for fc29c6d: cpplint, Ubuntu GCC, macOS Apple-Clang, Windows MinGW, and Windows MSVC.

@BartolomeyKant BartolomeyKant left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reread previous comments. Do not rewrite code just to pretend something is changed.
@NikolayChirkov your AI start hallucinating, validate what you're pushing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants