Problem
_require_single_pulse in src/ess/livedata/handlers/to_nxevent_data.py raises NotImplementedError when an ev44 message contains events from multiple pulses. The ev44 schema supports packing events from several pulses into a single message via reference_time (array of pulse timestamps) and reference_time_index (array of starting indices for each pulse's events within the flat event arrays).
This appears to be a leftover simplification from early development, when all producers sent single-pulse messages. It will break if an upstream source starts batching pulses.
Inconsistency with fast-path adapter
KafkaToMonitorEventsAdapter (the optimized direct-from-flatbuffer path) already silently ignores multi-pulse structure: it takes the flat TimeOfFlightAsNumpy() array and uses reference_time[-1] as the message timestamp, discarding per-pulse boundaries entirely. So the slow path (via MonitorEvents.from_ev44 / DetectorEvents.from_ev44) raises an error, while the fast path silently drops pulse information. Both behaviors are wrong.
What needs to change
The ev44 multi-pulse layout is:
reference_time[i] — timestamp for pulse i
reference_time_index[i] — starting index in the event arrays for pulse i
- Events for pulse
i span indices reference_time_index[i] to reference_time_index[i+1] (or end-of-array for the last pulse)
Each pulse should produce a separate events object with its own reference_time used as the event_time_zero timestamp in the downstream accumulator.
Implementation sketch
from_ev44 returns a list of events objects (one per pulse), slicing the flat arrays using reference_time_index as boundaries
- The
MessageAdapter protocol and AdaptingMessageSource need to support 1:N message expansion (one ev44 → N messages), or the expansion can be handled at the adapter level
KafkaToMonitorEventsAdapter needs the same treatment
- The change is mechanically straightforward (NumPy index slicing), but touches several layers: dataclasses, adapters, and the adapting source
Alternatives
1->N message translation might not be what we want. Explore changing the internal detector/monitor message schema, adding reference_time (etc.) fields to them.
Problem
_require_single_pulseinsrc/ess/livedata/handlers/to_nxevent_data.pyraisesNotImplementedErrorwhen an ev44 message contains events from multiple pulses. The ev44 schema supports packing events from several pulses into a single message viareference_time(array of pulse timestamps) andreference_time_index(array of starting indices for each pulse's events within the flat event arrays).This appears to be a leftover simplification from early development, when all producers sent single-pulse messages. It will break if an upstream source starts batching pulses.
Inconsistency with fast-path adapter
KafkaToMonitorEventsAdapter(the optimized direct-from-flatbuffer path) already silently ignores multi-pulse structure: it takes the flatTimeOfFlightAsNumpy()array and usesreference_time[-1]as the message timestamp, discarding per-pulse boundaries entirely. So the slow path (viaMonitorEvents.from_ev44/DetectorEvents.from_ev44) raises an error, while the fast path silently drops pulse information. Both behaviors are wrong.What needs to change
The ev44 multi-pulse layout is:
reference_time[i]— timestamp for pulseireference_time_index[i]— starting index in the event arrays for pulseiispan indicesreference_time_index[i]toreference_time_index[i+1](or end-of-array for the last pulse)Each pulse should produce a separate events object with its own
reference_timeused as theevent_time_zerotimestamp in the downstream accumulator.Implementation sketch
from_ev44returns a list of events objects (one per pulse), slicing the flat arrays usingreference_time_indexas boundariesMessageAdapterprotocol andAdaptingMessageSourceneed to support 1:N message expansion (one ev44 → N messages), or the expansion can be handled at the adapter levelKafkaToMonitorEventsAdapterneeds the same treatmentAlternatives
1->N message translation might not be what we want. Explore changing the internal detector/monitor message schema, adding
reference_time(etc.) fields to them.