adipo - Architecture-Aware Fat Binaries

adipo creates and runs fat binaries containing multiple versions of the same executable, each optimized for different CPU micro-architectures. Unlike Apple's lipo which targets different architectures (x86-64 vs ARM64), adipo targets micro-architecture versions within the same architecture family.

Why adipo?

Modern CPUs within the same architecture family have significantly different capabilities:

• x86-64: v1 (baseline) → v2 (SSE4.2) → v3 (AVX2) → v4 (AVX-512)
• ARM64: v8.0 (baseline) → v8.1 (atomics) → v8.2 (SVE) → v9.0 (SVE2)

A binary compiled for x86-64-v3 with AVX2 can be 20-40% faster than v1, but won't run on older CPUs. adipo solves this by packaging all versions together and selecting the best one at runtime.

Use Cases

Multi-Cloud & Multi-Instance Type Deployments

When deploying across multiple cloud providers or instance types, you face a dilemma:

Problem: Different instance types have different CPU capabilities

• AWS c5 instances: x86-64-v2 (Skylake)
• AWS c7a instances: x86-64-v3 (Zen 3)
• AWS c7i instances: x86-64-v4 (Sapphire Rapids with AVX-512)
• ARM Graviton 2: ARM64 v8.2
• ARM Graviton 3: ARM64 v8.4 with SVE

Traditional solutions:

1. Single binary (x86-64-v1): Works everywhere but leaves performance on the table
2. Multiple Docker images: myapp:amd64-v2, myapp:amd64-v3, myapp:arm64-v8
   • Complex deployment logic to choose the right image
   • More images to build, store, and manage
   • Need orchestration layer to select correct image per node

adipo solution:

# Build once with all optimizations
adipo create -o myapp.fat \
  --binary myapp-v1:x86-64-v1 \
  --binary myapp-v2:x86-64-v2 \
  --binary myapp-v3:x86-64-v3 \
  --binary myapp-v4:x86-64-v4 \
  --binary myapp-arm64:aarch64-v8.0

# Deploy the same binary everywhere
./myapp.fat  # Automatically selects the best version

Benefits:

• ✅ Single artifact: One binary for all instance types
• ✅ Automatic selection: No orchestration logic needed
• ✅ Maximum performance: Each instance runs the best available version
• ✅ Simple deployments: No need to match image tags to instance capabilities

When to Use Docker Images vs adipo

Use separate Docker images (one for amd64, one for arm64) when:

• You need different architectures (x86-64 vs ARM64)
• You want container registry's multi-arch manifest support
• You're running on very old kernels (pre-3.17) without memfd_create

Use adipo fat binaries when:

• You want different micro-architecture versions (v1 vs v2 vs v3 vs v4)
• You're deploying across heterogeneous instance types
• You want to simplify deployment without orchestration logic
• You want a single artifact for simplified CI/CD

Best of both worlds: Use both!

FROM alpine
COPY myapp.fat /usr/local/bin/myapp
ENTRYPOINT ["/usr/local/bin/myapp"]

Build one Docker image for amd64 and one for arm64, but each contains a fat binary with all micro-architecture versions. This gives you:

• Docker's multi-arch manifest for architecture selection
• adipo's runtime selection for micro-architecture optimization

Installation

Prerequisites

• Go 1.23 or later (required for ARM64 v8.1+ support via GOARM64)
• Linux or macOS

Install from Pre-built Releases (Recommended)

Download the appropriate archive for your platform from GitHub releases. Each archive contains both adipo and the corresponding adipo-stub binary:

Linux AMD64:

curl -LO https://github.com/DataDog/adipo/releases/download/v0.3.0/adipo-v0.3.0-linux-amd64.tar.gz
tar xzf adipo-v0.3.0-linux-amd64.tar.gz
# Archive contains: adipo, adipo-stub-linux-amd64
sudo mv adipo /usr/local/bin/
sudo mv adipo-stub-linux-amd64 /usr/local/bin/

macOS ARM64 (Apple Silicon):

curl -LO https://github.com/DataDog/adipo/releases/download/v0.3.0/adipo-v0.3.0-darwin-arm64.tar.gz
tar xzf adipo-v0.3.0-darwin-arm64.tar.gz
sudo mv adipo /usr/local/bin/
sudo mv adipo-stub-darwin-arm64 /usr/local/bin/

The stub binary enables creating self-extracting fat binaries. Place it in the same directory as adipo for automatic discovery.

Install with Go

# Install both adipo and adipo-stub
go install github.com/DataDog/adipo/cmd/adipo@latest
go install github.com/DataDog/adipo/cmd/adipo-stub@latest

Both will be installed to $GOPATH/bin (usually ~/go/bin). Make sure this directory is in your PATH.

Note: Both binaries should be in the same directory for automatic stub discovery. When adipo creates a fat binary, it looks for adipo-stub-{os}-{arch} (e.g., adipo-stub-linux-amd64) or a generic adipo-stub next to the adipo binary.

If the stub is in a different location, use --stub-path to specify it explicitly:

adipo create --stub-path /path/to/adipo-stub -o app.fat app1 app2

Build from Source

# Clone the repository
git clone https://github.com/DataDog/adipo
cd adipo

# Build adipo
make build

# Build stub for current platform
make stub

# Optionally install to /usr/local/bin
sudo mv adipo /usr/local/bin/
sudo mv internal/stub/stub.bin /usr/local/bin/adipo-stub-$(uname -s | tr '[:upper:]' '[:lower:]')-$(uname -m | sed 's/x86_64/amd64/')

Alternative Build Systems

For Bazel users, see BAZEL.md (experimental) for instructions on building with Bazel.

Homebrew (Coming Soon)

# Not yet available
brew install adipo

Quick Start

Create a Fat Binary

# Basic usage with automatic architecture detection
adipo create -o app.fat app-v1 app-v2 app-v3 app-v4

# With explicit architecture specifications
adipo create -o app.fat \
  --binary app-v1:x86-64-v1 \
  --binary app-v2:x86-64-v2 \
  --binary app-v3:x86-64-v3 \
  --binary app-v4:x86-64-v4

# ARM64 example
adipo create -o app.fat \
  --binary app-base:aarch64-v8.0 \
  --binary app-sve:aarch64-v8.2,sve \
  --binary app-sve2:aarch64-v9.0,sve2

Run a Fat Binary

Fat binaries are self-executing:

./app.fat [args...]

Or use the CLI for debugging:

# Run with verbose output
adipo run --verbose app.fat

# Force specific version
adipo run --force x86-64-v2 app.fat

Inspect a Fat Binary

# Table format (default)
adipo inspect app.fat

# JSON format
adipo inspect --format json app.fat

Example output:

Fat Binary: app.fat
Format Version: 1
Stub Size: 2.16 MB
Stub Architecture: aarch64-v8.0
Number of Binaries: 2

┌───────┬──────────────┬─────────┬────────────┬──────────┬────────────┬───────┐
│ Index │ Architecture │ Version │ Features   │ Original │ Compressed │ Ratio │
│ 0 *   │ aarch64      │ v8.0    │ (baseline) │ 2.28 MB  │ 1.36 MB    │ 59.6% │
│ 1     │ x86-64       │ v2      │ (baseline) │ 2.31 MB  │ 1.38 MB    │ 59.8% │
└───────┴──────────────┴─────────┴────────────┴──────────┴────────────┴───────┘

* Preferred binary for current CPU (aarch64 v8.0)

Advanced Features

Library Path Support

Configure library paths for binaries that require specific library versions. See LIBRARY_PATHS.md for details.

Quick example:

# Automatic glibc-hwcaps compatible paths
adipo create -o app.fat --enable-lib-path \
  --binary app-v1:x86-64-v1 \
  --binary app-v3:x86-64-v3

# Custom template paths
adipo create -o app.fat \
  --lib-path-template "/opt/glibc-{{.Version}}/lib" \
  --binary app-v1:x86-64-v1 \
  --binary app-v3:x86-64-v3

CPU-Specific Optimization

Optimize binary selection and library paths for specific CPU microarchitectures using CPU hints. This allows you to ship binaries tuned for different CPUs (e.g., AMD Zen 3 vs Intel Skylake) even at the same feature level.

Detect your CPU:

# See what CPU alias your system has
adipo detect-cpu

# Example output on AMD Zen 3:
# Architecture: x86-64
# Version: v3
# CPU Model: AMD Ryzen 9 7950X
# CPU Alias: zen4

Build with CPU hints:

# Create fat binary with CPU-optimized variants
adipo create -o app.fat \
  --binary app-baseline:x86-64-v2 \
  --binary app-zen:x86-64-v3:zen3 \
  --binary app-intel:x86-64-v3:skylake \
  --binary app-zen4:x86-64-v4:zen4

# With library paths using {{.CPUAlias}} template
adipo create -o app.fat \
  --enable-lib-path \
  --lib-path-template "/opt/{{.CPUAlias}}/lib" \
  --lib-path-template "/opt/{{.ArchVersion}}/lib" \
  --binary app-zen:x86-64-v3:zen3 \
  --binary app-intel:x86-64-v3:skylake

How it works:

1. Build time: Specify CPU hints with --binary FILE:ARCH:CPU-HINT
2. Runtime: adipo detects the current CPU and matches against hints
3. Selection: When hint matches detected CPU, that binary is preferred
4. Library paths: Paths with {{.CPUAlias}} are prioritized when hint matches

Example on Zen 3 CPU:

• Runtime detects: CPU alias = "zen3"
• Binary selection: app-zen binary selected (hint="zen3" matches)
• Library paths: /opt/zen3/lib checked before /opt/x86-64-v3/lib

Supported CPU aliases:

• x86-64: haswell, broadwell, skylake, skylake-avx512, icelake, zen, zen2, zen3, zen4
• ARM64 Linux: neoverse-n1, neoverse-n2, neoverse-v1, neoverse-v2, graviton2, graviton3, cortex-a76
• ARM64 macOS: apple-m1, apple-m2, apple-m3, apple-m4

Use adipo detect-cpu to find valid aliases for your architecture.

Hardware Capabilities Execution

Execute programs with automatic library path selection based on CPU capabilities using hwcaps-exec. This is useful for platforms without native glibc hwcaps support or for custom library layouts.

# Auto-detect CPU and select compatible library paths
adipo hwcaps-exec myprogram

# Or use the standalone binary
hwcaps-exec myprogram

# Preview what would be executed (dry run)
hwcaps-exec --dry-run myprogram

# See detailed scanning process
hwcaps-exec --verbose myprogram

The tool automatically scans standard glibc-hwcaps directories (/usr/lib64/glibc-hwcaps/), opt paths (/opt/<arch>/lib), and custom templates. See HWCAPS_EXEC.md for detailed documentation.

Stub Discovery

When creating a fat binary, adipo looks for stub binaries in this order:

1. Explicit path (if --stub-path is provided): Uses the specified stub
2. Platform-specific stub: Looks for adipo-stub-{os}-{arch} next to adipo binary
3. Generic stub: Looks for adipo-stub next to adipo binary
4. Error: If no stub is found and --no-stub is not specified

Architecture Specification Format

ARCH-VERSION[,FEATURE1,FEATURE2,...]

Aliases:

• amd64 = x86-64 (both are accepted)
• arm64 = aarch64 (both are accepted)

Examples:

x86-64-v1           # Baseline x86-64
amd64-v2            # Same as x86-64-v2
x86-64-v3,avx2      # v3 with AVX2 emphasized
x86-64-v4,avx512f   # v4 with specific AVX-512 features
aarch64-v8.0        # Baseline ARM64
arm64-v8.1,crc      # ARM v8.1 with CRC32
aarch64-v9.0,sve2   # ARM v9 with SVE2

Deterministic Extraction Paths

By default, fat binaries extract to random temporary directories. You can configure predictable extraction paths using templates:

# Create fat binary with deterministic extraction path
adipo create -o app.fat \
  --default-extract-dir ~/my-apps \
  --default-extract-file "app-{{.ArchVersion}}" \
  --binary app-v1:x86-64-v1 \
  --binary app-v3:x86-64-v3

# Execution extracts to: ~/my-apps/app-x86-64-v3
./app.fat

Template Variables:

• {{.Arch}} - Architecture name (x86-64, aarch64)
• {{.ArchTriple}} - Architecture triple (x86_64, aarch64)
• {{.Version}} - Version (v1, v2, v3, v8.0, v9.0)
• {{.ArchVersion}} - Combined (x86-64-v3, aarch64-v8.0)

Execution Methods:

# Force disk extraction (skip memfd)
adipo create --default-exec-method disk -o app.fat app-v1 app-v3

# Try memfd, fallback to disk (default)
adipo create --default-exec-method auto -o app.fat app-v1 app-v3

# Memory-only (fails on macOS or old kernels)
adipo create --default-exec-method memfd -o app.fat app-v1 app-v3

Environment Variables

ADIPO_VERBOSE=1        # Enable verbose output
ADIPO_DEBUG=1          # Enable debug output
ADIPO_FORCE=x86-64-v2  # Force specific version
ADIPO_PREFER_DISK=1    # Use disk instead of memory extraction
ADIPO_EXTRACT_DIR=~/my-apps  # Override extraction directory
ADIPO_CLEANUP_ON_EXIT=1      # Clean up extracted file after execution

Performance

Typical overhead:

• Startup time: ~10ms (CPU detection + decompression)
• Memory: ~2-3MB stub + decompressed binary size
• Disk I/O: None (memory extraction) or one temp file (fallback)

Space efficiency:

• Stub: ~2-3MB
• Compressed binaries: ~65% of original size (zstd)
• Example: 4 versions of a 10MB binary = 2MB stub + 26MB compressed = 28MB total vs 40MB for separate files

Limitations

• Self-extracting stub is architecture-specific: The embedded stub can only run on its target architecture

  • x86-64 stub works on x86-64 systems only
  • ARM64 stub works on ARM64 systems only
  • Solution 1: Use --no-stub and extract with adipo run or adipo extract
  • Solution 2: Build separate fat binaries for each main architecture (one with x86-64 stub, one with ARM64 stub)
  • Solution 3: Build cross-compiled stubs with make stub-all-arch (requires cross-compilation toolchain)

• Memory extraction is Linux-only: In-memory extraction uses memfd_create (Linux 3.17+)

  • Fallback to disk extraction works on macOS and older Linux kernels
  • No performance impact, just uses a temporary file

• Supported binary formats: ELF (Linux) and Mach-O (macOS)

  • PE (Windows) support planned

• Micro-architecture mixing only: Can mix x86-64 micro-architectures (v1/v2/v3/v4) and ARM64 versions (v8.x/v9.x)

  • But a single fat binary with both x86-64 AND ARM64 binaries needs architecture-specific stubs
  • Use Docker multi-arch manifests or separate fat binaries for different main architectures

Documentation

• LIBRARY_PATHS.md - Per-binary library path configuration
• TECHNICAL.md - Technical implementation details
• HWCAPS.md - Alternative approach using hardware capabilities for libraries
• BAZEL.md - Building with Bazel (experimental)
• TODO.md - Planned features and improvements

Contributing

Contributions are welcome! Please open an issue or pull request.

License

Apache License Version 2.0 - see LICENSE and NOTICE for details.

Credits

Created by Corentin Chary

Inspired by Apple's lipo but focused on micro-architecture levels rather than different architectures.

Related Projects

• lipo - Apple's tool for creating fat binaries (different architectures)
• upx - Executable packer (compression only, no multi-version support)
• go-build - Go's -tags build system (compile-time selection)