The Research Behind Minority Report

This is part 1 of a new series on interfaces!

In 2002, scenes from Minority Report looked like science fiction. But the idea started years earlier at MIT.

John Underkoffler was researching how computers could understand and respond to physical space through projects like the Luminous Room.

Originally started as a project under Professor Hiroshi Ishii’s Tangible Media Group at MIT, the Luminous Room was interactive, projecting information onto surfaces, and responding to human touch.

And this early work was spotted by Alex McDowell, a production designer working with Steven Spielberg.

Alex intro’d the team to Spielberg, and they were brought onto the film as science and technology advisors.

John helped translate that research into the film’s famous gesture interface—and taught Tom Cruise its visual language.

The work was done with incredible care, treating it like an R&D effort. The team’s design team created an 80-page encyclopedic vision of the storyworld. This document – dubbed the “2050 Bible” had a series of white papers written by John Underkoffler explaining his vision and science of interfacing.

Talk about worldbuilding!

24 years later, the movie is still a classic, and the UI gestures became a hallmark for interaction designers everywhere.

After the movie, John continued developing the idea through his company Oblong Industries and its g-speak system. Oblong’s customers include NASA, PwC, IBM, Fujitsu, and Accenture, along with other Fortune 500 companies

And yet as I look back through this history I realize that mass adoption of spatial computing hasn’t quite happened yet. Even after Apple’s launch of the Vision Pro, we haven’t seen much progress.

What do you think is stopping us?