What the 737 MAX’s return tells us about automation

Dr Ashley Nunes  is a consultant and academic, holding positions at the Massachusetts Institute of Technology and Harvard Law School. He previously led research projects sponsored by the US Department of Defense and the Department of Transportation. In this article, he argues that the lesson learned from the 737 MAX debacle should be that automation, like any technology, requires major, and sometimes hidden, trade offs.

The longest grounding of a commercial jet is set to end. Sometime this week, the Federal Aviation Administration is expected to certify the Boeing 737 MAX fit to fly. The aeroplane was grounded in 2019 following two crashes. The first involved a Lion Air jet which killed all 189 people on board. Months later, a second MAX jet, operated by Ethiopian Airlines, crashed. It also left no survivors . The crashes triggered lawsuits and government investigations.

The purported culprit in both crashes was technology — more specifically, a new flight-control feature dubbed MCAS (short for Maneuvering Characteristics Augmentation System). The feature was designed to prevent the aeroplane’s nose from getting dangerously high by automatically lowering it. However, under certain conditions, the MCAS lowered the nose so strongly that pilots  struggled  to maintain control.

In the aftermath, Boeing drew fire on several fronts. Firstly for charging  extra  for certain safety features. Where the MAX is concerned, the company wanted airlines to pony up more cash — at least $80,000 by one  estimate  — for add-ons that would alert pilots if the aircraft’s systems were malfunctioning. But Boeing was more broadly criticised for not properly vetting its technology before selling it.

That was then, this is now. Moving forward, not only is the company  making  all safety features free, Boeing has also dedicated tens of thousands of staff hours towards fixing the MAX. Pilot training manuals have been updated, components tested and most notably, more technological safeguards added. According to Boeing’s chief engineer, when the MAX does take to the skies, it will  be , “the safest plane out there.”

Aviation has long been risky — so risky in fact that during the industry’s early days, insurers were hesitant to provide coverage. In 1912, Lloyd’s of London stopped underwriting aviation after multiple air crashes left the market with sizeable losses. Technology changed this. Engines got better, sensors smarter and displays more reliable. The result was a dramatic drop in air crashes. In 2019, the aviation industry  saw  just one crash for every 884,000 flights. That’s a  fraction  of what they were a half-century ago.

But technology also introduced challenges. Prospective Democrat candidate challenger Andrew Yang summed up one of them in two words: job losses. The tech entrepreneur focused his White House bid on the growing threat automation poses to American jobs. But it was Lisanne Bainbridge who foresaw a more pressing problem brought on by the bots. In her 1983 paper Ironies of Automation , the British psychologist theorised the more automated the system, the more crucial the human becomes. Bainbridge concluded that a designer “who tries to eliminate the operator still leaves the operator to do the tasks which the designer cannot think how to automate.” Put simply, technology only gets you so far.

Which brings us to the MAX. Boeing’s pledge  to , “make safe planes even safer” are commendable. Public and private interests are ill-served when faulty products hit the market. But automation, while addressing one set of problems, often creates others and the consequences can be deadly. The MAX crashes are but one example of this. Yet, there have been others. Robotic surgery has the potential to make medical procedures safer, both through enabling greater precision and lowering the risk of infection for the patient. Between 2000 and 2013, mistakes, malfunctions and breakdowns of surgical robots were  linked  to over 144 deaths. Researchers concluded that “despite widespread adoption of robotic systems for minimally invasive surgery . . ., a non-negligible number of technical difficulties and complications are still being experienced during procedures.” Government investigators  looking  at automation in other industries have reached similar conclusions. For all their benefits, robots remain — much like humans — imperfect.

Recommended

This reality warrants acknowledgment. Instead of blindly accepting the virtues of technology, we should become techno-sceptics. Before embracing algorithms unequivocally, we should first ask important questions. For example, what is the marginal benefit of technological progress? Are we really better off using more automation today than we used yesterday? If so, by how much? Perhaps most importantly, what is the cost of ceding greater control to machines?

Technology may have improved. But the benefits may not be ours to reap. ​