-
Notifications
You must be signed in to change notification settings - Fork 0
Memory System
This page explains XAI OS's memory model for CPU-only AI workloads.
CPU-only AI is often memory-bound. Effective performance depends on placement, sharing, page-fault behavior, and avoiding duplicate model copies.
Raw DRAM or LPDDR bandwidth is the physical ceiling. Effective CPU-only AI bandwidth is the portion that remains usable after scheduler movement, cache pollution, duplicate weights, page faults, interrupts, and unrelated build/test pressure are removed.
XAI OS should use explicit arenas for:
- shared read-only model weights;
- private KV/cache state;
- source-code indexes;
- tokenizer state;
- network buffers;
- build output;
- logs and telemetry.
No swap should be used for AI memory.
Model arenas should use hugepages or large pages where supported. Hot arenas should be prefaulted before service READY.
Target invariant:
post-warmup page faults on hot AI paths = 0
On Intel Xeon, a latency-sensitive AI Cell should remain inside one NUMA node by default. Model weights should be NUMA-local or deliberately replicated.
On ARM/NVIDIA N1X-compatible systems, memory policy must account for LPDDR or unified memory behavior and cluster topology.
XAI OS should refuse to start or scale an AI Cell when memory bandwidth pressure would violate existing service contracts.
Intel Desktop:
- limited memory channels make duplicate model copies expensive;
- reduce jitter and background memory pressure before claiming bandwidth gains;
- keep build/test output away from model and KV/cache arenas.
Intel Xeon:
- NUMA node locality is mandatory by default;
- model weights should be node-local or intentionally replicated;
- memory-channel utilization is a benchmark gate.
ARM/NVIDIA N1X-compatible:
- LPDDR or unified-memory behavior must be measured directly;
- cluster locality and memory-controller firmware behavior are risks;
- SMMU policy must protect DMA into model and KV/cache arenas.
- Shared model weights are physically shared and mapped read-only.
- KV/cache arenas are private per embedded app agent.
- No swap is used for AI memory.
- Post-READY hot-path page faults are counted and fail policy unless explicitly allowed.
- Benchmark output separates raw bandwidth from effective CPU-only AI bandwidth.
This page defines the GitHub Wiki navigation sidebar.
- Architecture
- AI Cells
- CPU AI Runtime
- App Agents
- Memory System
- Networking
- Scheduler and Core Isolation
- Filesystem and Storage
- Driver Model
- Security Model
- Build System
- Build System
- Project Tracker
- Implementation Plan
- QEMU Full OS Core Workdown
- QEMU 100 Completion Plan
- Example Apps
- Codex Work Packages
- Testing and Benchmarking