Why user interface designers must take cues from science fiction and games

John Underkoffler gave an illuminating talk about the future of computing interfaces — and how slow the tech industry has been about creating new ones — at our recent GamesBeat Summit event in Berkeley, Calif.

His talk was, unsurprisingly, wonderfully visual as he stepped through the evolution of early augmented reality. His talk was important because it came from years of work trying to bridge science, science fiction, games, and business. Underkoffler is one of the seers who wants to enable new kinds of thinking and creativity through innovations in how we interact with the digital and physical worlds.

Underkoffler did some of the foundational work at the MIT Media Lab, where he was responsible for innovations in real-time computer graphics systems, and electronic holography. He showed experiments dubbed I/O Bulb and Luminous Room. That work eventually got him noticed by Hollywood.

John Underkoffler at GamesBeat Summit 2017.

Alex McDowell, the production designer for the 2002 film Minority Report, had to build the world behind Steven Spielberg’s film based on a short story by sci-fi author Philip K. Dick. He turned to Underkoffler for the science to help knit it all together. Their job was to envision how computers would work decades in the future.

“Spielberg said, ‘Don’t tell me it’s going to be the keyboard and the mouse still,'” Underkoffler said.

Tom Cruise in Minority Report inspired lots of tech and game companies.

Underkoffler has been trying to make the vision of the Minority Report, where actor Tom Cruise uses “data gloves” and gestures to control a transparent computer, into a reality.

“The film is a small container, and not everything you think of fits into the film,” Underkoffler said.

He founded Oblong Industries in 2006 and launched Mezzanine (a version of the Minority Report gesture controls for enterprise collaborators) in 2012.

“It seemed like then was the time, to really put it into practice, to take this idea that had been prototyped in academia, in art pieces, and in this fictional setting and bring it into commercial reality where it could be used to get work done,” Underkoffler said.

Ironically, during his talk, Underkoffler had to spend a larger than normal amount of time trying to get our slide clicker to work properly. Fortunately, such clickers are going to be obsolete in the world that Underkoffler envisions for us.

Please check out the video of Underkoffler’s talk.

Dean Takahashi

Dean Takahashi is editorial director for GamesBeat at VentureBeat. He has been a tech journalist since 1988, and he has covered games as a beat since 1996. He was lead writer for GamesBeat at VentureBeat from 2008 to April 2025. Prior to that, he wrote for the San Jose Mercury News, the Red Herring, the Wall Street Journal, the Los Angeles Times, and the Dallas Times-Herald. He is the author of two books, "Opening the Xbox" and "The Xbox 360 Uncloaked." He organizes the annual GamesBeat Next, GamesBeat Summit and GamesBeat Insider Series: Hollywood and Games conferences and is a frequent speaker at gaming and tech events. He lives in the San Francisco Bay Area.