Top Menu

Community: Puzzlebox talks to Alex Doud, creator of a Parrot AR drone you steer with your mind

Alex Doud has a lot going on.

Besides researching brain-computer interfaces (BCIs) in Dr. Bin He’s Neuroengineering laboratory at the University of Minnesota and building technology that lets people steer Parrot AR drones with their minds, Doud is:

  • Publishing scientific papers
  • Finishing a combined degree in biomedical engineering and medicine
  • Helping his wife, Natalie, get a mobile and web app development agency called Synaptic Design off the ground.

Wait…a mind-controlled drone?

Doud says he has been interested in BCI technology since he was a senior in high school. Back then, he and a team of fellow students wanted to use BCI in their entry to an annual creative problem solving competition, a science/arts/engineering contest that draws over 14,000 students from all over the world.

Even though, with a total project budget of $125, BCI was out of reach for the high school team, Doud’s interest in the tech didn’t wane. In his freshman year at the University of Minnesota, he approached Dr. Bin He, who runs one of the country’s pioneering BCI labs, and asked for a job.

At first, Doud did the lab’s grunt work – washing lab equipment and helping grad students with their experiments. But, after a semester-long stint at the Santa Lucia biomedical engineering lab in Italy, he came back with a head full of ideas for BCI applications.

His first big one: build a virtual-reality helicopter simulation that people could steer with their minds.

Flying a virtual helicopter on brainwaves

In 2010, using an open-source virtual reality (VR) builder called Blender, Doud and his team created a VR simulation of the University of Minnesota’s campus, and a helicopter that could fly around it. Instead of relying on a joystick or a mouse-keyboard combination, Doud’s VR helicopter used what is called a “reductionist 2D strategy” to simulate 3D control with a brain-computer interface.

In the reductionist 2D strategy, the virtual helicopter automatically flies forward, while the pilot uses signals from an EEG with sensors over the motor cortex to steer right, left, up, and down. In this way, BCI users could steer the helicopter anywhere in virtual 3D space, even though they didn’t have true three-dimensional control.

The virtual becomes real

After the success of the mind-controlled VR helicopter, Alex became more ambitious. He decided to bring the experience of flying something with your mind into the real world. (This was at the end of 2010, right around the time that Puzzlebox published a guide to building mind-controlled helicopters with a consumer-grade EEG headset, but the two projects arose independently.)

“Everyone [in the BCI field] was doing robotic arms,” he says, “so we wanted to do something different. We wanted to add in an element of entertainment and create something approachable and fun.”

Doud decided to use the Parrot AR drone, a programmable quadricopter, and spent the next two years (2011-2013) working on a system that let people steer it with their minds.

This was not an easy task.

Unlike the software helicopter Doud and his team had built, Parrot AR drones have real weight. Their battery lives are limited. They get tossed around by unpredictable air currents. They break when they crash too hard.

As Doud puts it, simply, “When you move from the virtual world to the real world, you encounter more constraints.”

Getting people up and running and piloting the drone wasn’t trivial, either.

“You can’t just put the [EEG] cap on and fly,” he says. There’s a training period, and its length typically ranges from 20-30+ training sessions of two hours, each.

By mid-2013, despite these obstacles (and after two years of research and a dozen or so wrecked drones), Doud and his team at Dr. He’s lab had something that worked. Using the same 2D reductionist strategy Doud had deployed for the VR helicopter, trained subjects could maneuver the Parrot drone through a series of hoops that hung from the ceiling using nothing but an EEG and their minds.

You can see it for yourself in the video above.

Now that Doud and his team have demonstrated that an EEG with sensors across the motor cortex can enable people to steer drones with their minds, he wants to see the BCI community develop better, faster ways of getting new users (especially clinical patients) accustomed to using the technology.

He thinks the gaming world offers a lot of applicable lessons.

“Today, the learning curve [for using BCI] is steep,” he says, “but if you make it fun, people are much more likely to stay engaged.”

Puzzlebox and the “consumerization” of BCI

The mind-controlled Parrot Drone that Doud’s team built relies on research-grade BCI hardware that has 64 electrodes, is priced far out of the consumer-friendly range, and takes a long time to master. Doud thinks that, if and when BCI technology becomes easier and more affordable, it will offer benefits for a wide swath of people – not only the clinical patients it is usually applied on, but the general population, as well.

He envisions a world where BCI is used in industry for hands-free control systems, in video games for more immersive experiences, and in personal development for mindfulness practice.

When asked about his thoughts on Puzzlebox and our own Orbit mind-controlled helicopter, Doud says: “Without companies like Puzzlebox, the BCI community would lose touch with early adopters, who are willing to try new things and push the technology to its limits. The good thing about what Puzzlebox is doing is that you’re opening it up, so people can apply scrutiny. Nothing is being hidden in the approach you’re taking.”

Alex Doud is a combined-degree student at the University of Minnesota, where he is finishing his Masters in Biomedical Engineering and getting his degree in Medicine. On the side, he and his wife run Synaptic Design, a mobile and web application development company that specializes in user experience design. He and Natalie are also behind an iOS app called Wander, which uses location data from Foursquare to create adventurous itineraries that help you discover your city in new, unexpected ways.

© 2016 Puzzlebox Productions LLC. All Rights Reserved.