JSC, the FAA, and Targets of Opportunity

As 2005 drew to a close, Michael Coffman at FAA Oklahoma City had convinced his line management that a flight demonstration of the sen­sor fusion technology would be a fine precursor to further FAA interest. FAA Oklahoma City had a problem: how best to protect its approaches of flight-check aircraft certifying instruments for the Department of Defense in combat zones. Coffman and Fox had suggested sensor fusion. If onboard video sensors in a flight-check aircraft could image a termi­nal approach corridor with a partially blended synthetic approach cor­ridor, any obstacle penetrating the synthetic corridor could be quickly identified. Coffman, using the MOU with NASA JSC signed just that July, suggested that an FAA Challenger 604 flight-check aircraft based at FAA Oklahoma City could be configured with SVS equipment to dem­onstrate the technology to NASA and FAA managers. Immediately, Fox, Coffman, Mike Abernathy of RIS, Patrick Laport and Tim Verborgh of AANA, and JSC electronics technician James Secor began discussing how to configure the Challenger 604. Fox tested his ability to scrounge excess material from JSC by acquiring an additional obsolete but ser­viceable Embedded GPS Inertial Navigation System (EGI) navigation processor (identical to the one used in the ACES van) and several pro­cessors to drive three video displays. Coffman found some FAA funds to buy three monitors, and Abernathy and RIS wrote the software nec­essary to drive three monitors with SVS displays with full-sensor fusion capability, while Laport and Verborgh developed the symbology set for the displays. The FAA bought three lipstick cameras, JSC’s Jay Estes designed a pallet to contain the EGI and processors, and a rudimentary portable system began to take shape.[1192]

The author, now a research pilot at the Johnson Space Center, became involved assisting AANA with the design of a notional instrument pro­cedure corridor at JSC’s Ellington Field flight operations base. He also obtained permission from his management chain to use JSC’s Aircraft

Operations Division to host the FAAs Challenger and provide the jet fuel it required. Verborgh, meanwhile, surveyed a number of locations on Ellington Field with the author’s help, using a borrowed portable DGPS system to create by hand a synthetic database of Ellington Field, the group not having access to expensive commercial databases. The author and JSC’s Donald Reed coordinated the flight operations and air traffic control approvals, Fox and Coffman handled the interagency approv­als, and by March 2006, the FAA Challenger 604 was at Ellington Field with the required instrumentation installed and ready for the first sen­sor fusion-guided instrument approach demonstration. Fox had bor­rowed helmet-mounted display hardware and a kneeboard computer to display selected sensor fusion scenes in the cabin, and five demonstra­tion flights were completed for over a dozen JSC Shuttle, Flight Crew Operations, and Constellation managers. In May, the flights were com­pleted to the FAA’s satisfaction. The sensor fusion software and hardware performed flawlessly, and both JSC and FAA Oklahoma City manage­ment gained confidence in the team’s capabilities, a confidence that would continue to pay dividends. For its part, JSC could not afford a more extensive, focused program, nor were Center managers uniformly convinced of the applicability of this technology to their missions. The team, however, had greatly bolstered confidence in its ability to accom­plish critically significant flight tests, demonstrating that it could do so with "shoestring” resources and support. It did so by using a small-team approach, building strong interagency partnerships, creating relation­ships with other research organizations and small businesses, relying on trust in one another’s professional abilities, and following rigorous adherence to appropriate multiagency safety reviews.

The success of the approach demonstrations allowed the team mem­bers to continue with the SVS work on a not-to-interfere basis with their regularly assigned duties. Fox persuaded his management to allow the ACES van to remotely control the JSC Scout simulated lunar rover on three trips to Meteor Crater, AZ, in 2005-2006 using the same sensor fusion software implementation as that on the Challenger flight test. Throughout the remainder of 2006, the team discussed other possibil­ities to demonstrate its system. Abernathy provided the author with a kneeboard computer, a GPS receiver, and the RIS’s LandForm software (for which JSC had rights) with a compressed, high-resolution database of the Houston area. On NASA T-38 training flights, the author evaluated the performance of the all-aspect software in anticipation of an official

evaluation as part of a potential T-38 fleet upgrade. The author had con­versations with Coffman, Fox, and Abernathy regarding the FAA’s idea of using a turret on flight-check aircraft to measure in real time the height of approach corridor obstacles. The conservations and the portability of the software and hardware inspired the author to suggest a flight test using one of JSC’s WB-57F High-Altitude Research Airplanes with the WB-57F Acquisition Validation Experiment (WAVE) sensor as a proof of concept. The WB-57F was a JSC high-altitude research airplane capable of extended flight above 60,000 feet with sensor payloads of thousands of pounds and dozens of simultaneous experiments. The WAVE was a sophis­ticated, 360-degree slewable camera tracking system developed after the Columbia accident to track Space Shuttle launches and reentries.[1193]

The author flew the WB-57F at JSC, including WAVE Shuttle tracking missions. Though hardly the optimal airframe (the sensor fusion proof of concept would be flown at only 2,000 feet altitude), the combination of a JSC airplane with a slewable, INS/GPS-supported camera system was hard to beat. The challenges were many. The two WB-57F airframes at JSC were scheduled years in advance, they were expensive for a single
experiment when designed to share costs among up to 40 simultaneous experiments, and the WAVE sensor was maintained by Southern Research Institute (SRI) in Birmingham, AL. Fortunately, Mike Abernathy of RIS spoke directly to John Wiseman of SRI, and an agreement was reached in which SRI would integrate RIS’s LandForm software into WAVE for no cost if it were allowed to use it for other potential WAVE projects.

The team sought FAA funding on the order of $30,000-$40,000 to pay for the WB-57F operation and integration costs and transport the WAVE sensor from Birmingham to Houston. In January 2007, the team invited Frederic Anderson—Manager of Aero-Nav Services at FAA Oklahoma City—to visit JSC to examine the ACES van, meet with the WB-57F Program Office and NASA Exploration Program officials, and receive a demonstration of the sensor fusion capabilities. Anderson was convinced of the potential of using the WB-57/WAVE to prove that an object on the ground could be passively, remotely measured in real time to high accuracy. He was willing to commit $40,000 of FAA money to this idea. With one challenge met, the next challenge was to find a hole in the WB-57F’s schedule.

In mid-March 2007, the author was notified that a WB-57F would be available the first week in April. In 3 weeks Fox, pushed through a Space Act Agreement to get FAA Oklahoma City funds transferred to JSC, with pivotal help from the JSC Legal Office. RIS and AANA, work­ing nonstop with SRI, integrated the sensor fusion software into the WAVE computers. Due to a schedule slip with the WB-57, the team only had a day and a half to integrate the RIS hardware into the WB-57, with the invaluable help of WB-57 engineers. Finally, on April 6, on a 45-minute flight from Ellington Field, the author and WAVE operator Dominic Del Rosso of JSC for the first time measured an object on the ground (the JSC water tower) in flight in real time using SVS technol­ogy. The video signal from the WAVE acquisition camera was blended with synthetic imagery to provide precise scaling.

The in-flight measurement was within 0.5 percent of the surveyed data. The ramifications of this accomplishment were immediate and profound: the FAA was convinced of the power of the SVS sensor fusion technology and began incorporating the capability into its planned flight-check fleet upgrade.[1194]

Building on this success, Fox, Coffman, Abernathy, and the author looked at new ways to showcase sensor fusion. In the back of their minds had been the concept of simulating a lunar approach into a vir­tual lunar base anchored over Ellington Field. The thought was to use the FAA Challenger 604 with the SVS portable pallet installed as before. The problem was money. A solution came from a collaboration between the author and his partner in a NASA JSC aircraft fleet upgrade study, astronaut Joseph Tanner. They had extra money from their fleet study budget, and Tanner was intrigued by the proposed lunar approach sim­ulation because it related to a possible future lunar approach training aircraft. The two approached Brent Jett, who was Director of Flight Crew Operations and the sponsor of their study, in addition to oversee­ing the astronauts and flight operations at JSC. Jett was impressed with the idea and approved the necessary funds to pay the operational cost of the Challenger 604. FAA Oklahoma City would provide the airplane and crew at its expense.

Once again, RIS and AANA on their own modified the software to simulate a notional lunar approach designed by the author and Fox, derived from the performance of the Challenger 604 aircraft. Coffman was able to retrieve the monitors and cameras from the approach
flight tests of 2006. Jim Secor spent a day at FAA Oklahoma City rein­stalling the SVS pallet and performing the necessary integration with Michael Coffman. The author worked with Houston Approach Control to gain approval for this simulated lunar approach with a relatively steep flightpath into Ellington Field and within the Houston Class B (Terminal Control Area) airspace. The trajectory commenced at 20,000 feet, with a steep power-off dive to 10,000 feet, at which point a 45-degree course correction maneuver was executed. The approach terminated at 2,500 feet at a simulated 150-feet altitude over a virtual lunar base anchored overhead Ellington Field. Because there was no digital database avail­able for any of the actual proposed lunar landing sites, the team used a modified database for Meteor Crater as a simulated lunar site. The team switched the coordinates to Ellington Field so that the EGI could still provide precise GPS navigation to the virtual landing site anchored overhead the airport.

In early February 2008, all was ready. The ACES van was used to val­idate the model as there was no time (or money) to do it on the airplane. One instrumentation checkout flight was flown, and several anomalies were corrected. That afternoon, for the first time, an aircraft was used to simulate a lunar approach to a notional lunar base. Sensor fusion was demonstrated on one of the monitors using the actual ambient conditions to provide Sun glare and haze challenges. These were not

representative of actual lunar issues but were indicative of the benefit of sensor fusion to mitigate the postulated 1-degree Sun angles of the south lunar pole. The second monitor showed the SVS Meteor Crater digital terrain database simulating the lunar surface and perfectly matched to the Houston landscape over which the Challenger 604 was flying.[1195] This and two more flights demonstrated the technology to a dozen astronauts and to Constellation and Orion program managers.

Four flight test experiments and three trips to Meteor Crater were completed in a 3-year period to demonstrate the SVS sensor fusion tech­nology. The United States military is using evolved versions of the orig­inal X-38 SVS and follow-on sensor fusion software with surveillance sensors on various platforms, and the FAA has contracted with RIS to develop an SVS for its flight-check fleet. Constellation managers have shown much interest in the technology, but by 2009, no decision has been reached regarding its incorporation in NASA’s space exploration plans.[1196]