National Aeronautics and Space Administration

Pushing the Envelope of Flight Test Safety

Technical, Physical and System Issues Behind AC-130J Flight Test Mishap

On April 21, 2015, an AC-130J, assigned to the 413th Flight Test Squadron at Eglin Air Force Base in Florida, departed controlled flight over water about 41 nautical miles south of the Air Force base. Even before the crew boarded the aircraft, missed red flags, procedural gaps, incomplete predictive data and documentation anomalies raised the stakes of an already rigorous test plan. Once on board and in the midst of executing an aggressive test maneuver, the collective oversights set the stage for an extraordinary in-flight emergency. Further complicated by insufficient instrumentation and human factors, the crew worked as a team to save their own lives, but the aircraft was not so fortunate. The cost of damages totaled $115,600,000, including the total loss of the aircraft.

140305-F-MV531-001 Under the AXZD0023 Job Order Number (23 JON), the mishap mission was designed to explore aircraft flight characteristics at the edges of the flight envelope with respect to sideslip (yaw angle from the flight path). The Mishap Pilot was assigned to fly a Steady Heading Sideslip (SHSS) test condition to 15 degrees yaw angle, triggering the aircraft’s RUDDER Special Alert warning. This maneuver, which was prohibited by the pilot’s flight manual, was authorized under an approved test plan. On the mishap flight, the pilot inadvertently exceeded 15 degrees sideslip, causing a “vertical fin stall,” which led to a departure from controlled flight.

According to the Accident Investigation Board (AIB) report, the Mishap Pilot’s inappropriate response to challenging conditions that occurred during the flight caused the accident. However, the report also revealed several underlying issues related to procedural guidance, the aircraft’s instrumentation/warning system, and the pilot’s spatial disorientation and confusion during the flight test.

Organizations such as the Air Force and NASA conduct extensive preflight planning and preparations for flight tests that reach the edge of an aircraft’s known performance envelope. These projects involve high-risk exposure to thermal, pressure and sonic environments. Dating back before Chuck Yeager broke the sound barrier in 1947, a deep understanding of how aircraft systems interact at their limits has allowed test engineers to design safety margins of time, airspeed, g acceleration and other parameters into the “test card” used by the crew. Then, the crew members are empowered to manage emergent risks in the cockpit as they see fit.

After discussing the technical causes of the AC-130J mishap identified by the AIB, this case study focuses on the underlying physical and system-related issues involved, emphasizing how mission-driven organizations can take precautions during the test planning phase to prevent similar future problems.

Background

AC-130J Aircraft

The AC-130J is a highly modified MC-130J aircraft. Its advanced features make it the most modern gunship in the Air Force inventory. The AC-130J has an advanced two-pilot flight station with fully integrated digital avionics. The aircraft’s precision strike package has a mission management console, a robust communications suite, two electro-optical/infrared sensors and advanced fire control equipment.

Air Force Test Chain of Command

The Air Force Test Center oversees developmental test and evaluation using two test wings: the 95th Test Wing and the 96th Test Wing. The 96th Test Wing supports multiple units, including the 413th Flight Test Squadron. This test squadron plans, executes and manages test and evaluation of special operations, combat search and rescue, and other missions. The U.S. Special Operations Command (USSOCOM) Detachment 1 (DET 1) conducted the AXZD0017 JON (17 JON) flight test in 2014 and the subsequent 23 JON flight test programs for the mishap aircraft.

17 JON Flight Test, a Precursor to the Mishap

On Feb. 25, 2014, the Air Force conducted the fourth flight test of the Test Directive for 17 JON, involving the AC-130J. This event resulted in a departure from controlled flight during stall testing. The test was never planned to investigate to the sideslip angles experienced later in the 23 JON mishap flight. 

According to an analysis independent of the safety investigation for the fourth 17 JON flight test, here’s what happened:

  • As airspeed slowed, the aircraft responded normally to flight control inputs until the stall warning “stick pusher” activated, pushing the control yoke forward. The pilot pushed the left rudder pedal to yaw the aircraft nose to the left as part of the test. To maintain level flight, the pilot had to counter so much yaw with an opposing roll input via the control yoke. Already held at 60 percent of full right roll control (or authority), the pilot rotated the yoke further to 90 percent to maintain heading and altitude.
  • The now cross-controlled aircraft flew to some degree sideways to its path through the air (a sideslip), increasing the angle-of-attack to the vertical fin. Airflow over the vertical tail fin separated on the downwind side, flowed around the fin (so-called “fin stall”) and pushed the rudder into a fully deflected position (so-called “rudder lock”). According to the AIB report, the fin stall and rudder lock were not perceived by the pilot. The aircraft was recovered after 4,000 feet of altitude loss.
Fin Stall

Example of fin stall. (Source: the C-130 Fin Stall Phenomenon/TAC Attack)

As a result of this event, some test members questioned if the new and slightly different aerodynamic shape of the AC-130J had degraded its handling qualities over the C-130J. Since the outer mold line on the AC-130J was modified with the addition of weapons and stores, how would these modifications impact flight characteristics? And would the modifications degrade flying qualities? Other members of the team disagreed with any need to do further testing. According to the USSOCOM DET 1 commander (lead for the entire test), “…you may have disagreed with it and you may have not completely understood exactly why, or if it made sense at all but it became more of those ones, as a function of being able to — to execute and get further testing done, we needed to play nice with — with the folks who were making the recommendations, even if we didn’t necessarily believe 100 percent in them...” The AIB report shows that the test team added 5,000 feet to the test altitude for recovery safety.

The AIB report for the April 21 incident did not address whether the test team employed Risk Management as prescribed by Air Force Instruction 63-101 under Life Cycle Systems Engineering and Environmental, Safety and Occupational Health (ESOH) responsibilities. Specifically, the report stated that ESOH “provides a safety release for the system prior to each developmental and operational test involving known system hazards to people, equipment, or the environment. The safety release identifies the hazards involved in the test and their formal risk acceptance. This is in addition to and can inform any safety release provided by the T&E [Test and Evaluation] organization.”


Lack of Predictive Flight Data

Fin stall was not a new or misunderstood hazard within the C-130 test community. In fact, it was documented for decades in Air Force guidance and Lockheed Martin technical publications. Although the test team members knew they were skirting the edge of known flight test knowledge, they were denied access to that information. (Likely, an assessment by Lockheed Martin could have been acquired through a contract.)

The Air Force had not purchased developmental engineering and test data rights from Lockheed Martin — specifically a 2013 report that contained predictive data. This report included plots from previous SHSS flight tests, which the team could have used to predict the rudder force required to achieve sideslip angles in the 15-degree-range test limit. Test team members could not even access their own 17 JON test data, since the data had to be processed by the manufacturer in accordance with contract negotiations from 2013.

To deal with this situation, the test team installed new special instrumentation, which allowed it to view rudder forces but without predictive data. One exception was a chart provided by Lockheed Martin that described when the SIDESLIP Special Alert would sound for excessive Angle of Sideslip (AoS).

23 JON Mission Planning

Test planning for 23 JON was extensive, spanning several months and including a well-documented test and safety process. Technical adequacy and safety risks were noted in the Test Directive for 23 JON. However, the test program unaccountably took the aircraft “to the edges of the aircraft envelope in sideslip 183 times” before the mishap event occurred. The test on April 21 was designed to explore whether modifications to aircraft weight, aerodynamic shape and control systems “adversely affected the aircraft’s flying qualities.” Under the mission authority of the 96th Operations Group Commander (OG/CC), the flight test was designed to include a series of flying qualities maneuvers, including SHSSs at various flap and gear configurations. According to the AIB report, successfully carrying out these maneuvers would help “demonstrate positive lateral-directional stability throughout the entire designed envelope.”

Rudder_RoundDuring the medium-risk flight test, the Mishap Pilot was expected to maneuver the aircraft in a buildup fashion to reach the SIDESLIP Special Alert (first alert) and the RUDDER Special Alert (second alert). Buildup implies a methodical and planned stepwise movement to a planned boundary flight condition that represents a less than fully understood condition of the aircraft’s controllability or structural response. The stepwise methodology is based upon an increase or decrease in a specific parameter or parameter(s) that inherently affords a controlled approach and defines the boundary condition itself.

The 23 JON test program planned to exceed the SIDESLIP Special Alert and reach the RUDDER Special Alert sideslip angle of 14.5 degrees. These Advisory Caution and Warning System special alerts were designed to advise aircrews of unsafe conditions and the need for immediate action to correct sideslip conditions caused by rudder pedal deflections or movements generated by the GAU-12/U Gatling cannon mounted on the left side of the aircraft. These special alerts included both visual and auditory cues.

Heads Up Display (HUD)

The HUD provided critical flight instrument data at windscreen level so that the pilot could include the outside horizon as much as possible in his scan and maintain proper aircraft attitude. To get to the RUDDER Special Alert, the team relied on the HUD to give the pilot indications of sideslip and the special instrumentation package for monitoring safety parameters (e.g., rudder pedal force).

The preliminary technical order for the aircraft did not make it clear that the sideslip indicator freezes once it bisects the fence. According to Davis’ report, the test pilot said this was not understood by the test team.

Waiver

The AIB report noted that the appropriate Air Force Materiel Command office signed a waiver to give “approval to intentionally maneuver the AC-130J into a sideslip resulting in a LEFT/RIGHT RUDDER alert during regression testing,” even though this would exceed the flight manual limit for the aircraft. As part of the waiver, participating aircrews were directed to review the High Sideslip Recovery Procedures (C-130J-1) and Fin Stall Recovery (C-130H-1) before flight when intentional rudder alert test points were to be flown. The AIB report explained that “the Fin Stall warning section describes the risk of fin stalls at angles of sideslip from 15–20 degrees of sideslip and at speeds from stall to 170 knots.”

The High Sideslip Recovery Procedures stated, “If either RUDDER Special Alert occurs, immediately apply the indicated rudder to center the sideslip display on the HUD.” The AIB report noted that the waiver was signed in February 2014 as part of the 17 JON flight test program and before the safety planning and first departure of the 23 JON flight test program. Therefore, its guidance could have been included in the Test Hazard Analysis Worksheet (THAW) during the Safety Review Board on July 18, 2014. However, the THAW failed to mention reviewing the Fin Stall Recovery as directed by the waiver.

Approvals/Authority

The AIB report highlighted the following indications of the approvals and authority surrounding the 23 JON flight test program:

  • The 96th Operations Group Commander reviewed and approved technical and safety considerations during the Test Approval Brief on Aug. 6, 2014.
  • While the Safety Annex was updated with Amendment 1 on Sept. 18, 2014, it “did not make any changes regarding the High Sideslip Recovery or the Fin Stall warning even though the first departure was a departure in sideslip.”
  • As a result of “test team ‘turbulence,’” the 413th Flight Test Squadron technical director removed himself from reviewing test cards on Feb. 12, 2015. While his approval was not a requirement from the Operations Group, it provided an “additional layer of review.” On March 5, 2015, the technical director rescinded his signature authority on all test planning and execution documents associated with the AC-130J development test. However, he did not indicate he wanted the test to stop. In fact, he stated that the test plan “is the best product that the Air Force Test Center could put out under the circumstances.” The test approval authority (OG/CC) was never informed that the final signature on the Method of Test had been rescinded.

What Happened

Flight Data Lockheed Martin

Animation from Lockheed Martin, based on flight data showing attitude of the aircraft and cockpit presentation after aircraft inverted during departure (Source: U.S. Air Force AIB report/ Lockheed Martin).

The following events occurred on April 21, 2015, during the 23 JON mishap flight:

  • 10:46 a.m. — After takeoff, the Mishap Pilot completed a series of flying qualities test points at 15,000 feet.
  • 12:10 p.m. — The crew began performing SHSSs with flaps at 100 percent, gear down and 140 knots. The Mishap Pilot reached SHSSs to the right but did not stabilize at the RUDDER alert. He applied as much as 278 pounds of rudder pedal force and indicated that his foot was at the end of the rudder pedal travel. The test point to the right was terminated. The Mishap Pilot proceeded to conduct SHSSs to the left at 12:16 p.m.
  • 12:16 p.m. — While completing SHSSs to the left, the Mishap Pilot stabilized the SIDESLIP Special Alert (first alert) for nearly 10 seconds, applying 125 pounds of force to the rudder pedal.
  • 12:18 p.m. — The mishap test conductor started to clear the Mishap Pilot to proceed to the second special alert.
  • Recovery events (undetermined time) — The Mishap Copilot began the recovery from the dive. He pulled the aircraft out of the dive, retracted the flaps and recovered with less than 10,000 feet of altitude. The aircraft was overstressed and reached 3.194 Gs. The flaps were oversped by more than 100 knots. The smoke alarm in the cargo compartment was triggered by powder from a fire extinguisher that broke apart by the violent movement of the aircraft.

Proximate Cause

According to the AIB report, the Mishap Pilot responded incorrectly to conditions that occurred during the flight test. The report noted that the mishap occurred because “the MP’s [Mishap Pilot’s] excessive rudder input during the test point followed by inadequate rudder input to initiate a timely recovery from a high sideslip angle due to Overcontrolled/Undercontrolled Aircraft and Wrong Choice of Action During an Operation.”

Underlying Issues

In addition to the pilot’s inappropriate responses to the circumstances and environment of the flight test, the AIB report identified the following underlying issues:   

Technical Difficulty of the Task

For pilots, flight performance is based on the touch and feel of their flight control as well as their experience base. A pilot’s proficiency is partly conferred by natural aptitude and partly earned through extensive practice. Pilots manage technical challenges during flight based on their ability and experience.

In several instances, the AIB report highlighted the technical difficulty of the flight test, which helped contribute to the mishap. Consider the following examples:

  • “Several test pilots commented on how difficult the task is due to the variation in sensitivity on the rudder pedals required for each test point condition compounded by the waffling flight characteristics of the Dutch Roll.”
  • Several test pilots expressed their hesitation “to be aggressive with the rudders and felt reducing rudder pedal force was preferred to being too aggressive with applying rudder inputs.”
  • According to the mishap plot, “…there was no other good way to do it, um, other than build down [in] speed and build up in rudder force. I mean, it was the only way that made sense as far as from a safety standpoint…”
  • an, it was the only way that made sense as far as from a safety standpoint…”

Instrumentation and Warning System Issues

The AIB report explained that the sideslip warning system’s visual cues provided limited real-time positioning information beyond the SIDESLIP Special Alert. The system froze at the RUDDER Special Alert. According to the AIB, “Since the test crews had no indication if they were beyond the RUDDER Special Alert, it is plausible the MP thought he was executing the final portion of the test point perfectly based on the visual cues alone.”

In addition, the preliminary technical order for the aircraft did not “make it clear that the sideslip indicator freezes once it bisects the fence.” According to the AIB report, this was not understood by the test team. One test pilot admitted: “What I didn’t realize…is once it gets to the second alert it does not move any further. So it stops, so, you have no indication past — if you’ve gone past that point and if so how far…so if I were to change that, I would change that mechanization somehow to, to show if I’ve gone past that how far past that I have gone.”

According to the AIB, “At best, the Warning System did little to help the MP avoid overcontrolling the aircraft. At worst, the Warning System could be misunderstood to make the MP perceive he was exactly on conditions while actually making the situation worse.”

Spatial Disorientation

The AIB report defined spatial disorientation as the Mishap Pilot’s failure to sense a position, motion or attitude of the aircraft or himself. The following statements from the Mishap Pilot suggest that spatial disorientation contributed to the mishap flight:

  • “When the FT [Flight Test engineer] called recover, I was…disoriented in a way that I don’t remember ever feeling before in an aircraft.”
  • “…the only thing I could actually latch onto was the airspeed decreasing…”
  • “We were far south in the water ranges; so I had no cultural references to ascertain motion of the aircraft; so I was fully dependent on the HUD and the instrumentation inside the aircraft.”

In addition, the AIB report indicated that the Mishap Pilot’s “reluctance to use the rudder seems more tied to spatial disorientation than anything else.” The Mishap Pilot revealed, “I think if I had known what I know now, and I knew that what we were in was a sideslip departure or a spin motion, I wouldn’t have hesitated to apply the correct procedure.”

Confusion

The AIB report defined confusion as the inability to maintain “a cohesive and orderly awareness of events and required actions.” It characterized this mental state as one of bewilderment with a lack of clear thinking. The Mishap Pilot himself revealed the role of confusion in the mishap flight. “I was trying to contemplate everything,” he said. “There were just too many things.” During the pilot’s interview, he also stated, “I couldn’t quite recognize why the aircraft was continuing to do what it was doing. Out of all the things that I could see, in my field of view, the only thing that I could recognize as status of the aircraft was the airspeed.” Confusion likely contributed to the Mishap Pilot’s inappropriate response to the conditions that occurred during the flight test.

Inadequate Procedural Guidance or Publications

According to the AIB report, the test team was not provided with adequate procedural guidance for the flight test. The AIB revealed, “The error amounted to a reasonable expectation that the edge of the envelope was 16 degrees AoS when in fact it was only 14.5 degrees AoS. This misinformation altered guidelines placed on the safety of test displays. Therefore, had the correct limits been used, a safety monitor may have called ‘terminate’ earlier, possibly preventing the rudder lock and, by extension, the departure. Admittedly, this is speculation. The actual impact is undetermined, but in my opinion, this false information was more likely than not a substantially contributing factor.”  

Communication Issues

Communication issues likely contributed to the following events related to the flight test mishap: Although senior management dictated the inclusion of recovery procedures, only a partial effort was made to carry them out.

  • The technical director removed himself from the process of reviewing test cards and providing signature authority related to test planning and execution documents.
  • There was confusion related to the actual AoS limit during the flight test mishap.
  • The test conductor failed to stop the test when the mishap Mishap Pilot did not adhere to the procedure.

Applying Lessons Learned to Current and Future NASA Missions

VAB from gsdo ksc tps mpcv noesa NASA is now involved in planning, research and test operations to a greater extent than in recent decades. In conjunction with International Space Station crew and cargo launches and recoveries, the following NASA projects are proceeding on a weekly basis:

  • X-plane projects at NASA Armstrong Flight Research Center
  • Space Launch System (SLS)/Orion systems tests and qualification assessments at multiple NASA centers
  • Government insight into commercial launch service providers (instead of more direct oversight)
  • As mentioned, flight-related projects and their associated test operations involve high-risk exposure to thermal, pressure and sonic environments, making the planning for worst-case situations vital. Proper planning for flight tests will help safeguard crews and vehicles during test operations.
  • Precautions for test planning identified from this case study include the following:
  • When working with companies holding proprietary test data, the effort to understand and even purchase data rights may be critical to successful testing and troubleshooting.
  • When members of the test team express reservations regarding the necessity of taking identified risks, do not just retreat from the test, but try to analyze the potential benefit(s) versus the potential cost. Too often, only the potential cost is considered in risk decisions. This will help separate needless risks fromneedful risks.
  • Organizational leaders should create an environment where every member of the test team not only feels empowered to express concerns but also sees that a technical concern is technically addressed.
  • Waivers and deviations are not recognized by the laws of nature.
NASA has excellent compendiums of test guidance, including the Jet Propulsion Laboratory Design Validation/Verification Guideline. Other test guidance documents are listed in the References.

Questions for Discussion

  • How do you determine the acceptable amount of risk to take during a test?
  • How important is obtaining predictive data prior to the testing phase?
  • Within your project, are there multiple levels of review and approval required prior to moving forward with test procedures?
  • If instrumentation and warning systems seem lacking, is there a process in place to report and modify these systems?

References

This is an internal NASA safety awareness training document based on information available in the public domain. The findings, proximate causes, and contributing factors identified in this case study do not necessarily represent those of the Agency. Sections of this case study were derived from multiple sources listed under References. Any misrepresentation or improper use of source material is unintentional.