Review of ophyd-async

[TeresiaOlsson]

ophyd-async https://blueskyproject.io/ophyd-async/main/index.html is a new version of ophyd which is compatible with both EPICS and TANGO. It can do asynchronous calls but it is really ophyd 2.0 because it has been refactored with a new architecture. This new architecture seems more fitting for accelerator purpose.

According to the documentation, ophyd-async has most of the features for an hardware abstraction layer that we have in the pyAML specification. We can't use it directly because the steering committee has decided that there shouldn't be any mandatory dependencies of any package in the Bluesky ecosystem but it is still worth to review it and see if there are ideas we can borrow when implementing our own hardware abstraction layer. The developers of it are colleagues at several of the labs in our community so I also think it would be beneficial to learn from their experience.

Here is an explanation of the basics: https://blueskyproject.io/ophyd-async/main/explanations/devices-signals-backends.html

[TeresiaOlsson]

I added everyone who sounded interested in looking at the pros/cons of ophyd-async.

My experience of it so far in my exploration is similar to @yhidaka. I haven't really managed to get the device level to work well because it doesn't seem to be sufficiently configurable. I'm stuck at the point where I for a device have different PV suffix for different machines but just want to have a single device class. I also don't entirely like the way you need to add children. But the signal level with a signal backend looks promising. So could be that the device level is just not suitable for us.

@Sulimankhail is going to implement the TANGO version of the EPICS devices in the accml repository. Let's see what he figures out for how to configure it.

@marlibgin2 and I have plans to test it at MAX IV the next time they have machine week in R1 (it's in six weeks). @JeanLucPons @gupichon if you have something related to the performance of the TANGO bindings you think would be useful to test we could probably do it then.

The biggest pro so far I would say is that they are very helpful and supportive. When I ask a question on their Mattermost channel I usually get a reply within an hour. Sometimes they even write the code for me. So I'm not worried about having dependencies that they maintain. Actually, it rather feels good since our maintainers don't have so many FTEs.

If we conclude that there are parts of it that we like and want to use, then I think it will be possible to convince the steering committee that we should be allowed to add the dependency.

[JeanLucPons]

I was not aware that ohyd-asynch was dependent of Bluesky.
I though it was possible to use only the hardware abstraction in a simple manner without a Bluesky RE.

[TeresiaOlsson]

@JeanLucPons It's not dependent on Bluesky. It is a completely separate package which you can use either on it's own or together with the Bluesky runengine depending on what you want. To use together with the runengine, the devices need to implement the Bluesky protocols, but that's an optional add-on. You can also use it without implementing those protocols since it's implemented in a very modular approach.

But it is the hardware abstraction layer developed as part of the Bluesky ecosystem and so far the steering committee has not approved dependencies of any of those packages. So if we like it we need to ask permission in the same way as we needed to ask for pyAT. But if we use ophyd-async (or parts of it) it would much simplify the question of Bluesky compatibility which the steering committee is also discussing since some labs want it. It would make it straight-forward to add that compatibility as an extension for the labs that want it without adding Bluesky dependence for the labs that don't. So I would say, exactly the solution we need.

[JeanLucPons]

The dependency of Bluesky ["bluesky>=1.13.1rc2",] appears in the toml of ophyd-asynch. bluesky.protocols package is used in both Tango and Epics bindings.
If we manage to get ophyd-aynch working outside the bluesky Run Engine, I think this could be accepted.

[TeresiaOlsson]

So far, I have only seen the protocols being used at the device level but I'm still exploring. So not on the signal level but @yhidaka might know better since he said he had made use of only the signal level in PAMILA.

I also think if it turns out to be a problem for us we can talk to the developers. They seem very interested in getting it to work for us. I will meet them at the ROCK-IT workshop in a couple of weeks and then I'm going to give a short presentation describing our project and our requirements. One of the requirements I will put then is that we don't want any mandatory dependency of the bluesky package but we still have some labs that want bluesky compatibility so we need a solution which allows both.

[TeresiaOlsson]

I have been playing with the derived_signals in ophyd-async to understand how they do conversions there. In their case they don't have hardware to physics but raw to derived but the functionality I think is similar to what we need.

It was very easy to use. Below is the short test case I wrote to write a single plane tune device for MLS.

from pydantic import BaseModel
from typing import Optional, TypedDict
from ophyd_async.core import StandardReadable, derived_signal_r, Transform, DerivedSignalFactory
from ophyd_async.epics.core import epics_signal_r

# Config
class Tune1DConfig(BaseModel):
    name: Optional[str] = 'tune'
    read_pv: str
    settling_time: Optional[float] = 0

# Conversions    
class TuneRaw(TypedDict):
    frequency: float

class TuneDerived(TypedDict):
    tune: float
              
class TuneTransform(Transform):
    
    revolution_frequency: float
    
    def derived_to_raw(self, *, tune: float) -> TypedDict:
        return TuneRaw(
            frequency = tune*self.revolution_frequency*1e-3
            )
    
    def raw_to_derived(self, *, frequency: float) -> TypedDict:
        return TuneDerived(
            tune = frequency*1e3/self.revolution_frequency
            )    

# Tune device
class Tune1D(StandardReadable):
    
    def __init__(self, config: Tune1DConfig, prefix: Optional[str] = None) -> None:
        
        self._settling_time = config.settling_time
        
        self._read_pv = f"{prefix}:{config.read_pv}" if prefix else config.read_pv
        
        revolution_frequency = 6.25e6 # This should not be hardcoded but read from somewhere else.
        
        frequency = epics_signal_r(float, self._read_pv, name='frequency')
                      
        self._factory = DerivedSignalFactory(
                TuneTransform,
                frequency=frequency,
                revolution_frequency=revolution_frequency,
            )
              
        tune = self._factory.derived_signal_r(float,name='tune')
        
        with self.add_children_as_readables():
            self.frequency = frequency
            self.tune = tune
            
        super().__init__(name=config.name)    

[TeresiaOlsson]

I have also been playing with the Status functionality they have. It was also very easy to use.

Here is my test for a masterclock where you can control the execution using a ramp rate and it checks the tolerance between the readback and setpoint after it is done. The set returns a Status object which has attributes success and done. If it's done but the tolerance check gives an error, the success will be false.

import asyncio
from pydantic import BaseModel
from typing import Optional
from ophyd_async.core import (
    StandardReadable,
    AsyncStatus,
    CALCULATE_TIMEOUT,
    DEFAULT_TIMEOUT,
    CalculatableTimeout,
)

from ophyd_async.epics.core import epics_signal_rw

import logging

for handler in logging.root.handlers[:]:
    logging.root.removeHandler(handler)
logging.basicConfig(level=logging.DEBUG)


class MasterClockConfig(BaseModel):

    name: Optional[str] = 'masterclock'
    setpoint_pv: str
    readback_pv: str
    ramp_rate: float # Unit should be in kHz/s
    tolerance: float # Unit should be in Hz
    # TODO: Units needs to be generalised and handled better


class MasterClock(StandardReadable):

    def __init__(self, config: MasterClockConfig, prefix: Optional[str] = None) -> None:

        # TODO: this needs to be generalised so can also incude ca and pva.
        if prefix:
            self._setpoint_pv = prefix + ':' + config.setpoint_pv
            self._readback_pv = prefix + ':' + config.readback_pv
        else:
            self._setpoint_pv = config.setpoint_pv
            self._readback_pv = config.readback_pv

        self._ramp_rate = config.ramp_rate
        self._tolerance = config.tolerance

        with self.add_children_as_readables():
            self.frequency = epics_signal_rw(float, read_pv= self._readback_pv,
                                             write_pv= self._setpoint_pv, name='frequency')
        super().__init__(name=config.name)

    @property
    def setpoint_pv(self):
        return self._setpoint_pv

    @property
    def readback_pv(self):
        return self._readback_pv

    @property
    def ramp_rate(self):
        return self._ramp_rate

    @property
    def tolerances(self):
        return self._tolerance


    @AsyncStatus.wrap
    async def set(self, new_value: float, timeout: CalculatableTimeout = CALCULATE_TIMEOUT):
        logging.debug(f"set() called with new_value={new_value}")

        old_value = await self.frequency.get_value()
        logging.debug(f"Old value = {old_value}")

        # Calculate the time to wait based on the ramp rate
        change = abs(new_value - old_value)
        wait_time = change / self._ramp_rate
        logging.debug(f"Calculated wait time = {wait_time}")

        # Define the timeout based on the ramp rate
        if timeout == CALCULATE_TIMEOUT:
            timeout = wait_time + DEFAULT_TIMEOUT
            logging.debug(f"Calculated timeout = {timeout}")

        # Set the value
        await self.frequency.set(new_value)
        await asyncio.sleep(wait_time)

        # Compare so readback and setpoint is within some tolerance
        current_values = await self.frequency.locate()
        diff = abs(current_values['readback'] - current_values['setpoint'])

        logging.debug(f"Compare setpoint = {current_values['setpoint']} and readback = {current_values['readback']}")
        logging.debug(f"Absolute difference is = {diff}")

        if diff > self._tolerance:
            raise Exception('The difference between the readback and setpoint is larger than the tolerance.')
``

[TeresiaOlsson]

For all of these devices I have been using them without the Bluesky runengine. For the masterclock I can for example use it like this:

# Connect the device
await masterclock.connect()

# Read current values
current_values = await masterclock.frequency.locate()
masterclock_setpoint = current_values['setpoint']

# Set the device
status = masterclock.set(masterclock_setpoint + 2)
# This has status.done and status.success which can be used in applications to control the execution

# Read the device
await masterclock.read() # This will read all the attributes.
# There is also an option to configure that some attributes are only for configuration and are only read if doing read_configuration()

await masterclock.frequency.read() # Read only the frequency