NTSB Questions Both FAA and Boeing Assumptions on 737 Max

Handling emergencies and unusual situations is a significant part of learning to fly and remaining current to fly any airplane. Flight training can’t possibly teach pilots how to handle every single event they could encounter, however. With that concept in mind, pilots are often taught to look for the similarities between one event and another to help them more quickly find a solution to the situation at hand. That kind of training demands making certain assumptions about the pilots who will occupy the flight deck.

In the case of Boeing’s 737 Max–grounded since last March following two deadly accidents–the company assumed pilots flying the relatively new jet would recognize that the failure of its maneuvering characteristics augmentation system looked similar to a runaway electric trim, and therefore decided cockpit crews didn’t need to know of the existence of the MCAS. The company probably never would have mentioned anything about MCAS had it not been for the first crash of a Lion Air Boeing 737 Max in November 2018.

Because the company believed the possibility of an MCAS failure to be a once in a lifetime event, they also assumed that uncommanded system inputs–like those the accident crews experienced–would be readily recognizable and could be counteracted by a normal movement of the flight controls. Pilots would not need exceptional piloting skills or strength and would take immediate action to fix any problems they might encounter.

The National Transportation Safety Board took Boeing's logic to task after the second Max accident in March of this year, explaining that they believed the Seattle aircraft builder overlooked a number of salient human factors conclusions. The details of the NTSB's thinking were outlined last week in a number of safety recommendations to the FAA following the Board's initial investigation into the two 737 Max accidents.

The NTSB is “concerned that the process used to evaluate the original design needs improvement because that process is still in use to certify current and future aircraft and system designs,” the Board said. Essentially, the NTSB believes Boeing overlooked some basic human factors data that proved pilots can only cope with so many emergencies at one time. Too many warnings can turn a pilot’s brain to mush, resulting in a classic loss of control. “We are making these recommendations to address assumptions about pilot recognition and response to failure conditions used during the design certification process as well as the creation of diagnostic tools to improve the prioritization and clarity of failure indications presented to pilots,” the report said.

Like other transport category aircraft the 737 Max was equipped with a number of warning systems to alert the flight crew of impending problems. Flightcrew alerting is expected to provide guidance for showing compliance with design requirements and indicates “appropriate flightcrew corrective actions that are normally defined by procedures (for example checklists) and are part of a flightcrew’s training curriculum or are considered basic airmanship,” according to the NTSB. Boeing again assumed that an MCAS failure would be encountered rarely, but that in any case would look so similar to a runaway trim situation that crews would simply see it as that and act accordingly.

Unfortunately the cockpit crews of the two Boeing accident airplanes didn’t quickly identify the problem at hand because faulty angle of attack data on the ground before takeoff induced the Boeing to create other warnings at liftoff, which the NTSB believes distracted the flight crews. On both aircraft, for instance, the stick shakers activated at takeoff. Altimeter and airspeed disagree lights also came on. The Ethiopian Airlines crew also received a master caution light. Once flaps were retracted on both airplanes, stick forces to hold the nose of the aircraft up increased significantly. And, in the case of the Ethiopian crash, these warnings occurred while the airplane was still within just a few hundred feet of the ground.

The NTSB said, “Multiple alerts and indications can increase pilots’ workload, and the combination of the alerts and indications did not trigger the accident pilots to immediately perform the runaway stabilizer procedure during the initial automatic nose down stabilizer trim input. Pilot responses differed and did not match the assumptions of pilot responses to unintended MCAS operation on which Boeing based its hazard classifications within the safety assessment and that the FAA approved and used to ensure the design safely accommodates failures. Although a number of factors, including system design, training, operation, and the pilots’ previous experiences, can affect a human’s ability to recognize and take immediate, appropriate corrective actions for failure conditions, industry experts generally recognize that an aircraft system should be designed such that the consequences of any human error are limited.”

The Board added that “While Boeing considered the possibility of uncommanded MCAS operation as part of its functional hazard assessment, it did not evaluate all the potential alerts and indications that could accompany a failure that also resulted in uncommanded MCAS operation. Therefore, neither Boeing’s system safety assessment nor its simulator tests evaluated how the combined effect of alerts and indications might impact pilots’ recognition of which procedure(s) to prioritize in responding to an unintended MCAS operation caused by an erroneous AOA input.” The NTSB worries that, with so many assumptions that have proven unreliable, training pilots to cope with these kinds of situations is problematic and with good reason.

“As early as 2002, a joint FAA-industry study recognized that while excellent guidance existed for manufacturers on various topics important to the development of system safety assessments, there were no methods available to evaluate the probability of human error in the operation of a particular system design and the existing qualitative methods for assessing human error were not very satisfactory.” The 2002 study went on to explain that the processes used to determine and validate human responses in safety assessments needed to improve.

The NTSB last week asked the FAA to ensure that system safety assessments for the 737 MAX consider the effect of all possible flight deck alerts and indications on pilot recognition and response and redesign the airplane to meet these guidelines where necessary. The Board wants the FAA to reexamine other aircraft certification reports to see if similar assumptions used in the certification of other air carrier aircraft might have slipped through the cracks up to now. The NTSB also wants the FAA to share these ideas with regulators around the world and to develop robust tools to evaluate certification assumptions long before they become a problem. Finally, the NTSB wants to see better ways for pilots to prioritize failure indications to help them improve the speed and effectiveness of their responses to multiple flight deck alerts.

It’s important to note that the NTSB’s recommendations are just that, recommendations, ideas the FAA is free to adopt or not depending upon its own evaluation of a situation.