Alumnus Helped Put 'Drone Net' on the Launch Pad

The game-changing technology, based on a network of passive rooftop sensors that capture electro-optical and infrared data (EO/IR), continuously scans the sky – at a much lower cost than RADAR. 

In the future, if the Drone Net’s all-sky camera and connected acoustic network detect a small unmanned aerial system (sUAS) without a flight plan, or off its flight plan, the technology will kick into gear. Specifically, the all-sky camera will cue an EO/IR camera to slew and track the sUAS with high-resolution visible and infrared imaging until the non-compliant sUAS leaves the area monitored by Drone Net – encompassing about 1 square kilometer. 

“When the all-sky and acoustic system detects something moving,” said Samuel Siewert, an assistant professor of electrical, computer and software engineering, “it activates automatically, sort of like lizard-brain intelligence. It sends a message to begin electro-optical and infrared sensing by tilting and panning the camera to redetect the sUAS in a narrow field of view. The EO/IR sensing is controlled by machine intelligence, so that it will re-detect and track, supporting the identification of drones by ground-based computing systems.” 

Ultimately, the goal of the research is to help law enforcement distinguish between responsible drone operators and possibly hostile ones by creating a database of drone “fingerprints.” The researchers, working on Embry-Riddle’s Prescott, Ariz., campus, will compare and validate data captured by the Drone Net with information from many other types of passive and active sensors.

Embry-Riddle Eagle Matthew Demi Vis, now an avionics engineer at the Jet Propulsion Laboratory, at work with a copy of the Mars Curiosity Rover.

Scanning Shipping Channels and Skies

Undergraduate research by Matthew Demi Vis – a 2017 Embry-Riddle alumnus now working as an avionics engineer with the Jet Propulsion Laboratory – developed foundational technology that helped set the Drone Net project onto the proverbial launch pad, Siewert noted. 

Initially, Vis was part of a project funded by the U.S. Department of Homeland Security and the Arctic Domain Awareness Center to identify rogue ships in new shipping channels. That project, in collaboration with the University of Alaska and the University of Colorado-Boulder, involved developing a computer-based way to remotely process data captured by sensors on buoys.

“As warmer temperatures are melting new passages through the Arctic, some vessels are illegally using them as shipping channels,” Vis explained. “These rogue ships turn off their transponders to avoid detection. Our idea was to drop something onto a buoy so that it would transmit data on vessels to the Coast Guard.”

In such remote locations, computing efficiency is essential, Vis said: “We wanted to use as little power as possible. I helped develop different ways of processing the data with different computer architectures.”

This same technology for tracking marine traffic is now being adapted to visualize UAS activity. To help Embry-Riddle’s Drone Net recognize different types of drones, Vis had to manually annotate a large collection of images. “Part of trying to get the technology set up was, a human had to go through all the video and say, `That’s a drone, that’s not a drone.’ There were 12,000 or 14,000 frames of individual pictures that had to be examined.”

Since then, Siewert said, Embry-Riddle researchers have been working on a machine-based learning system so that the Drone Net can automatically classify aerial objects. A first step toward establishing a database of drone fingerprints will be to fine-tune the way different Drone Net instruments communicate with each other, he noted.

“We think the machine-learning approach could perform as well or perhaps better than RADAR,” Siewert said. “It would let us detect, track and localize as well as determine shape, texture and color information so that we could better classify and identify aerial objects.”

A RADAR system can easily cost more than $100,000, making it impractical for many non-military uses, he noted.

Research as a Teaching Tool 

These days, Vis is busy with his work for JPL, where he said his Embry-Riddle research experience has proved invaluable.

“My undergraduate research gave me insights to different hardware and software technologies and how they interact,” he said. “My name was on two papers while I was still an undergraduate, and I got to present our findings at a conference with Dr. Siewert. All of that extra experience really opened up the doors needed for me to do what I wanted right out of school … Space exploration is my passion, and right now at JPL, I’m doing avionics integration and testing for the Europa Clipper (a mission to Jupiter’s moon).” 

Siewert and his colleagues have meanwhile been leveraging the Drone Net concept as a teaching tool. This past summer, for example, Embry-Riddle’s ICARUS (Instrumentation and Control of Robotic Unmanned Systems) Group offered a summer camp on aerial robotics. High school students took part in mock search-and-rescue missions, using a Drone Net-style multi-model sensor network – including overhead cameras, ground robots and aerial robots – to navigate a maze and rescue a person in need of help. 

“We wanted to provide the students with basic information on how you operate UAS safely and effectively in different scenarios,” Siewert said. “They might enjoy it as a game or a challenge but, we presented them with the larger picture, too, and we each engaged them in a curriculum segment based on the Drone Net research.” 

In summer 2019, Siewert also plans to teach Real-Time Systems and the Internet of Things – technologies used for Drone Net – in Embry-Riddle’s study abroad program in Ireland. 

“To have safe air traffic management in an unrestricted fashion,” Siewert said, “what we need is instrumentation so that small UAS that are compliant can do so safely and small UAS that are non-compliant can be localized and ATC can handle them in a contingency scenario.” 

The ICARUS Group at Embry-Riddle is directed by Dr. Stephen Bruder, associate professor of electrical engineering. Along with Siewert, the Drone Net team includes Embry-Riddle faculty member Dr. Mehran Andalibi, Engineering Lab Technician Jim Weber and external collaborator Dr. Iacopo Gentilini. The team has collaborated with faculty members Dr. Jennah Perry and Dr. Sarah Nilsson. Drs. Richard Stansbury and Elliott B. Bryner serve as faculty advisors to students. 

Currently, the following students are essential to the project, too: 

• Garrison Bybee
• Jonathan Buchholz
• Robert R. Noble
• Thomas Rapp
• Steve Rizor at Colorado University – Boulder