Designed to save lives, but destined to kill
Some of us may have been unfortunate enough to have visited a hospital at one point in our lives, either for treatment of minor injuries or perhaps something a little more drastic. I recall clambering onto the table of a large white machine that reminded me of a giant Polo mint, the purpose of which was to examine one of my knees for possible injury. I lay there, listening to the device make a whole host of bizarre sounds as it scanned and mapped the internal structures of my knee joint, wondering how it all worked. I felt perfectly safe, entombed within this expensive looking machine and I more than likely was well protected from a possible malfunction, but imagine if something had gone wrong with the safety mechanisms built into the device that deals with precise radiation delivery?
A number of years ago this is what happened with a medical appliance called the Therac-25. This machine was designed to target and eliminate cancer cells and was the later incarnation of the Therac-20 and Therac-6. During the design and build of this medical device, vital safety mechanisms were overlooked or removed altogether with the belief that good software would control and prevent any issues. The net result of those changes was a series of deaths, whereby the software that controlled the Therac-25 was liable to go haywire if a few certain conditions were met. This interesting and also frightening article on Hackaday showcases the importance of following and adhering to a strict Risk Analysis processes during the design, build, testing and use of all manner of machinery, medical or otherwise.
Imagine if you had designed a machine intended to make people better and it ended up killing them?