Simply put, a critical system is one where failure brings the potential for catastrophic harm to human life, major financial losses, serious security breaches or mission failure.
Suppose you developed a device that can measure human blood sugar levels for a medical professional to possibly prescribe insulin. There is a risk that the machine may be incorrect but the medical professional is the watchdog for the incorrect diagnosis. So, this system may be considered low risk.
If an enhanced device determines blood sugar levels and recommends the insulin, there is greater risk that the medical professional may take the results at face value and make an incorrect prescription. This system is high risk.
But if the device measure blood sugar, determines a dose of insulin is recommended, and then goes ahead and administers the insulin, this is a critical system. This might be an example of agentic AI. And testing said system is as critical as the system itself.
Simply put, a critical system is one where failure brings the potential for catastrophic harm to human life, major financial losses, serious security breaches or mission failure.
Suppose you developed a device that can measure human blood sugar levels for a medical professional to possibly prescribe insulin. There is a risk that the machine may be incorrect but the medical professional is the watchdog for the incorrect diagnosis. So, this system may be considered low risk.
If an enhanced device determines blood sugar levels and recommends the insulin, there is greater risk that the medical professional may take the results at face value and make an incorrect prescription. This system is high risk.
But if the device measure blood sugar, determines a dose of insulin is recommended, and then goes ahead and administers the insulin, this is a critical system. This might be an example of agentic AI. And testing said system is as critical as the system itself.