Testing and Benchmarking

Testing and Benchmarking

Purpose

This page defines how XAI OS should be tested and benchmarked.

Contents

• Principles
• Baselines
• Metrics
• Platform Gates
• JSON Result Format

Principles

No performance claim is valid without a tuned Linux/BSD baseline, reproducible commands, hardware description, workload description, and raw results.

QEMU on macOS is for correctness only and does not count for performance claims.

Baselines

Baselines should document:

• OS version and kernel version;
• CPU model and firmware settings;
• memory configuration;
• NIC and storage configuration;
• scheduler and IRQ tuning;
• hugepage settings;
• command lines;
• raw logs.

Metrics

Required benchmark categories:

• raw RAM latency;
• effective CPU-AI memory bandwidth;
• TCP/UDP latency;
• CPU sustained performance;
• scheduler jitter;
• core migration count;
• context switch count;
• page fault count;
• interrupt count on hot cores;
• timer tick count on hot cores;
• unexpected IPI count;
• p50/p95/p99/p999 network latency.

Platform Gates

Intel Desktop gates focus on P-core/E-core placement, DDR bandwidth limits, low-latency TCP/UDP, NVMe, and PCIe NIC behavior.

Intel Xeon gates focus on NUMA locality, memory channels, server NICs, many AI Cells, and multi-queue storage.

ARM/NVIDIA gates focus on AArch64 boot, GIC, SMMU, LPDDR/unified memory behavior, and cluster placement.

JSON Result Format

Benchmark tools should emit JSON that follows Benchmark Result Format. Minimal example:

{
  "benchmark": "tcp_latency",
  "platform": "intel_desktop",
  "os": "xaios",
  "baseline": "linux-tuned",
  "hardware": {
    "cpu": "example",
    "memory": "example",
    "nic": "example"
  },
  "metrics": {
    "p50_us": 0,
    "p95_us": 0,
    "p99_us": 0,
    "p999_us": 0,
    "migrations": 0,
    "context_switches": 0,
    "page_faults": 0
  }
}

Related Pages

• Performance Targets
• Benchmark Result Format
• Intel Desktop Port
• Intel Xeon Port
• ARM NVIDIA Port