[RFC] Audio and Spectral Processors

[kunitoki]

Plan: ProcessorBase / DomainProcessor / SpectralProcessor Hierarchy

Context

The current AudioProcessor class is a monolithic base that mixes domain-agnostic concerns (name, parameters, state, playhead, lock, presets) with audio-domain specifics (bus layout, sample rate, processBlock). Introducing SpectralProcessor requires extracting these concerns cleanly so both domains can share infrastructure without coupling. AudioGraphProcessor requires no changes — the graph sees all nodes polymorphically through ProcessorBase; mixed-domain support is achieved via a FFTBridgeNode that is a plain AudioProcessor containing a spectral subgraph.

---

Architecture Overview

ProcessorBase                              — name, params, lock, preset/state, playhead
    ↑
DomainProcessor<TProcessContext,           — prepare(ctx), reset(), process(ctx), bypass(ctx)
                TPrepareContext>
    ↑
AudioProcessor                             — bus layout, sampleRate, processBlock() (backward compat)
SpectralProcessor                          — fftSize, hopSize, prepareSpectral()
SpectralGraphProcessor (: SpectralProcessor) — manages a DAG of SpectralProcessors
FFTBridgeNode         (: AudioProcessor)   — FFT → SpectralGraph → IFFT inside one audio node

---

Key Design Decisions

Backward compatibility on AudioProcessor

processBlock(AudioBuffer<float>&, MidiBuffer&) stays pure-virtual. process(AudioProcessContext&) is added as a final non-overridable that builds the context and delegates to processBlock. Existing subclasses compile unchanged. New code continues to override processBlock.

// AudioProcessContext — new, in yup_audio_processors
struct AudioProcessContext
{
    AudioBuffer<float>& audio;
    MidiBuffer& midi;
    uint64_t samplePosition = 0;
};

// In AudioProcessor
void process (AudioProcessContext& ctx) override final  // new entry point from DomainProcessor
{
    processBlock (ctx.audio, ctx.midi);
}

AudioPrepareContext wraps (sampleRate, maxBlockSize) for symmetry; prepare() on AudioProcessor stores both then calls prepareToPlay().

AudioGraphProcessor unchanged

AudioGraphProcessor currently stores shared_ptr<AudioProcessor> and calls processBlock. This remains. The spectral domain enters the audio graph only via FFTBridgeNode, which is a regular AudioProcessor node. No changes to graph scheduling, routing, or compilation.

SpectralProcessor prepare vs constructor

FFT size and hop size live in SpectralPrepareContext (not the constructor) so they can change between prepareSpectral() calls without recreating nodes. This matches the pattern of AudioProcessor::prepareToPlay.

---

Phase 1 — ProcessorBase + DomainProcessor (in yup_audio_processors)

New files

modules/yup_audio_processors/processors/yup_ProcessorBase.h

Extracts from AudioProcessor:

• processorName, getName()
• parameters, parameterMap, addParameter(), getParameters()
• playHead, setPlayHead(), getPlayHead()
• processLock, processIsSuspended, getProcessLock(), isSuspended(), suspendProcessing()
• Pure virtuals: getCurrentPreset, setCurrentPreset, getNumPresets, getPresetName, setPresetName, loadStateFromMemory, saveStateIntoMemory, hasEditor
• Non-pure virtual: createEditor()

modules/yup_audio_processors/processors/yup_ProcessorBase.cpp

Implementations currently in yup_AudioProcessor.cpp for the above members.

modules/yup_audio_processors/processors/yup_DomainProcessor.h (header-only template)

template <typename TProcessContext, typename TPrepareContext>
class DomainProcessor : public ProcessorBase
{
public:
    explicit DomainProcessor (StringRef name) : ProcessorBase (name) {}

    virtual void prepare (const TPrepareContext& context) = 0;
    virtual void reset() = 0;
    virtual void process (TProcessContext& context) = 0;
    virtual void bypass (TProcessContext& context) { ignoreUnused (context); }
    virtual void flush() {}
};

modules/yup_audio_processors/processors/yup_AudioProcessContext.h

struct AudioPrepareContext { float sampleRate; int maxBlockSize; };
struct AudioProcessContext  { AudioBuffer<float>& audio; MidiBuffer& midi; uint64_t samplePosition = 0; };

Modified files

modules/yup_audio_processors/processors/yup_AudioProcessor.h

• Change: class AudioProcessor → class AudioProcessor : public DomainProcessor<AudioProcessContext, AudioPrepareContext>
• Remove: processorName, parameters, parameterMap, playHead, processLock, processIsSuspended (now in ProcessorBase)
• Keep: busLayout, sampleRate, samplesPerBlock, processingPrecision
• Add: prepare() final → stores sampleRate/samplesPerBlock, calls setPlaybackConfiguration then prepareToPlay
• Add: reset() final → calls releaseResources()
• Add: process(AudioProcessContext&) final → calls processBlock(ctx.audio, ctx.midi)
• Keep: all existing pure/non-pure virtuals unchanged (prepareToPlay, releaseResources, processBlock, etc.)

modules/yup_audio_processors/processors/yup_AudioProcessor.cpp

• Move common implementations to yup_ProcessorBase.cpp
• Keep audio-specific implementations

modules/yup_audio_processors/yup_audio_processors.h — add includes before yup_AudioBus.h:

#include "processors/yup_ProcessorBase.h"
#include "processors/yup_DomainProcessor.h"
#include "processors/yup_AudioProcessContext.h"

modules/yup_audio_processors/yup_audio_processors.cpp — add:

#include "processors/yup_ProcessorBase.cpp"

Phase 1 tests

tests/yup_audio_processors/ProcessorBase.cpp

• Name stored and retrieved correctly
• addParameter / getParameters round-trips
• suspendProcessing / isSuspended round-trips
• setPlayHead / getPlayHead round-trips
• DomainProcessor<AudioProcessContext, AudioPrepareContext> concrete subclass dispatches to process()

---

Phase 2 — SpectralBuffer + SpectralProcessor (new yup_spectral_processors module)

The new module depends on yup_audio_processors (for DomainProcessor, ProcessorBase) and yup_dsp (for FFTProcessor). It lives separately rather than in yup_dsp to avoid a circular dependency.

New module directory: modules/yup_spectral_processors/

yup_spectral_processors.h     — module header (BEGIN_YUP_MODULE_DECLARATION, includes)
yup_spectral_processors.cpp   — implementation aggregator
yup_spectral_processors.mm    — Apple ObjC++ aggregator
processors/
    yup_SpectralBuffer.h/.cpp
    yup_SpectralBufferView.h
    yup_SpectralPrepareContext.h
    yup_SpectralProcessContext.h
    yup_SpectralProcessor.h/.cpp

yup_SpectralBuffer.h

Owning multi-channel complex float buffer.
Layout: per-channel interleaved [re0, im0, re1, im1, ...] matching FFTProcessor::performRealFFTForward output convention.

class SpectralBuffer
{
public:
    SpectralBuffer (int numChannels, int numBins);
    void setSize (int numChannels, int numBins, bool keepExistingContent = false);
    void clear() noexcept;
    int getNumChannels() const noexcept;
    int getNumBins() const noexcept;
    float* getWritePointer (int channel) noexcept;
    const float* getReadPointer (int channel) const noexcept;
    std::complex<float>* getComplexWritePointer (int channel) noexcept;
    const std::complex<float>* getComplexReadPointer (int channel) const noexcept;
    void addFrom (int destChannel, const SpectralBuffer& source, int srcChannel, float gain = 1.0f) noexcept;
    void copyFrom (int destChannel, const SpectralBuffer& source, int srcChannel) noexcept;
private:
    std::vector<float> data_;  // flat: [ch0 bins | ch1 bins | ...]
    int numChannels_ = 0, numBins_ = 0;
};

yup_SpectralBufferView.h — non-owning view, wraps SpectralBuffer or raw float** pointers.

yup_SpectralPrepareContext.h

struct SpectralPrepareContext
{
    float sampleRate  = 44100.0f;
    int   fftSize     = 2048;    // must be power of 2; numBins = fftSize/2+1 (real FFT)
    int   hopSize     = 512;     // how often process() fires in samples
    int   numChannels = 2;
};

yup_SpectralProcessContext.h

struct SpectralProcessContext
{
    SpectralBufferView spectrum;      // bins to read/modify in-place
    uint64_t           frameIndex    = 0;   // monotonically increasing since last reset()
    uint64_t           samplePosition = 0;  // audio-sample position of this hop's first sample
};

yup_SpectralProcessor.h

class SpectralProcessor : public DomainProcessor<SpectralProcessContext, SpectralPrepareContext>
{
public:
    explicit SpectralProcessor (StringRef name);
    int   getFftSize()     const noexcept { return fftSize_; }
    int   getHopSize()     const noexcept { return hopSize_; }
    float getSampleRate()  const noexcept { return sampleRate_; }
    int   getNumChannels() const noexcept { return numChannels_; }
    virtual int getLatencyFrames() const noexcept { return 0; }

    void prepare (const SpectralPrepareContext&) override final; // stores members, calls prepareSpectral()
    void reset()                                 override final; // calls releaseSpectralResources()

    virtual void prepareSpectral (const SpectralPrepareContext&) = 0;
    virtual void releaseSpectralResources() = 0;

protected:
    float sampleRate_ = 44100.0f;
    int   fftSize_    = 2048;
    int   hopSize_    = 512;
    int   numChannels_ = 2;
};

Phase 2 tests

tests/yup_spectral_processors/SpectralBuffer.cpp

• setSize allocates; getNumChannels/Bins correct
• clear() zeros all data
• addFrom / copyFrom produce correct values
• SpectralBufferView over SpectralBuffer yields correct pointers

tests/yup_spectral_processors/SpectralProcessor.cpp

• Concrete subclass receives monotonically increasing frameIndex
• prepare() stores sampleRate_, fftSize_, hopSize_, numChannels_
• reset() calls releaseSpectralResources()
• getLatencyFrames() defaults to 0

---

Phase 3 — SpectralGraphProcessor + FFTBridgeNode

New files in modules/yup_spectral_processors/

graph/
    yup_SpectralGraphNodeID.h
    yup_SpectralGraphConnection.h
    yup_SpectralGraphProcessor.h/.cpp
bridges/
    yup_FFTBridgeNode.h/.cpp

yup_SpectralGraphProcessor — mirrors AudioGraphProcessor architecture:

• Stores SpectralProcessor nodes
• Control-thread commitChanges() → compiled immutable plan
• Lock-free plan swap at frame boundaries
• Topological/parallel execution order
• Is itself a SpectralProcessor (can be nested inside another SpectralGraphProcessor)
• No AudioBusLayout — I/O topology is (fftSize, numChannels) from SpectralPrepareContext

yup_FFTBridgeNode — is an AudioProcessor, enabling mixed-domain graphs:

AudioGraphProcessor
    └── FFTBridgeNode (AudioProcessor)
            ├── forward FFT per channel (FFTProcessor)
            ├── SpectralGraphProcessor::process() per hop
            └── inverse FFT + overlap-add per channel

class FFTBridgeNode final : public AudioProcessor
{
public:
    FFTBridgeNode (int fftSize = 2048, float overlapFactor = 0.75f, int numChannels = 2);
    void setSpectralGraph (std::shared_ptr<SpectralGraphProcessor> graph);

    // AudioProcessor
    void prepareToPlay (float sampleRate, int maxBlockSize) override; // allocs all buffers
    void releaseResources() override;
    void processBlock (AudioBuffer<float>& audio, MidiBuffer& midi) override; // no alloc
    int  getLatencySamples() override { return fftSize_; }

private:
    int fftSize_, hopSize_, numChannels_;
    float overlapFactor_;
    FFTProcessor fft_;
    std::shared_ptr<SpectralGraphProcessor> spectralGraph_;
    SpectralBuffer spectralBuffer_;                       // preallocated
    std::vector<std::vector<float>> inputFifo_;           // per-channel
    std::vector<std::vector<float>> outputFifo_;          // per-channel overlap-add
    std::vector<float> analysisWindow_, synthesisWindow_;
    int hopCounter_ = 0;
    uint64_t frameIndex_ = 0, samplePosition_ = 0;
};

processBlock algorithm (no allocation):

1. Copy samples into per-channel input FIFOs, increment hopCounter_
2. While hopCounter_ >= hopSize_: extract fftSize_ samples with analysis window → FFTProcessor::performRealFFTForward per channel → build SpectralProcessContext → spectralGraph_->process() → performRealFFTInverse per channel → overlap-add into output FIFO → decrement hopCounter_ by hopSize_, increment frameIndex_
3. Copy numSamples from output FIFO into audioBuffer

Phase 3 tests

tests/yup_spectral_processors/SpectralGraphProcessor.cpp

• Empty graph processes without crash
• Single-node chain: node receives frames in topological order
• Two-node chain: nodes receive consecutive frames

tests/yup_spectral_processors/FFTBridgeNode.cpp

• DC passthrough (identity spectral graph): output matches input within tolerance
• Sine wave: phase continuity across processBlock call boundaries
• getLatencySamples() == fftSize
• No heap allocation inside processBlock (verified via custom allocator instrumentation)

tests/yup_spectral_processors/SpectralIntegration.cpp

• AudioGraphProcessor with FFTBridgeNode (identity spectral graph) produces output matching input within float tolerance
• AudioGraphProcessor::getAllocationStats().delayLines == 1 (latency compensation for FFTBridgeNode)

---

Critical Files

File	Action
modules/yup_audio_processors/processors/yup_AudioProcessor.h	Change base class, remove extracted members
modules/yup_audio_processors/processors/yup_AudioProcessor.cpp	Move extracted impls to ProcessorBase.cpp
modules/yup_audio_processors/yup_audio_processors.h	Add 3 new includes
modules/yup_audio_processors/yup_audio_processors.cpp	Add ProcessorBase.cpp include
modules/yup_audio_graph/graph/yup_AudioGraphProcessor.cpp	No changes needed
modules/yup_dsp/frequency/yup_FFTProcessor.h	Reference only — used by FFTBridgeNode

---

Verification

Phase 1: All existing AudioProcessor subclasses (including AudioGraphProcessor) compile without changes. Existing tests pass. New ProcessorBase tests pass.

Phase 2: New SpectralBuffer and SpectralProcessor tests pass. Module builds standalone.

Phase 3: Integration test confirms AudioGraphProcessor + FFTBridgeNode round-trip produces identity output. FFTBridgeNode tests confirm real-time safety (no allocation in processBlock).