Not everyone will be able to afford the HoloLens. And, at first, most people probably won’t want to strap computers to their faces. But the complete 3D ecosystem will be here and Microsoft is going great lengths to ensure that it will be a leader in what they dub mixed reality. So, to bridge the gap between tomorrow’s world of face computers and today’s world of pocket computers, Microsoft Research has developed an app called MobileFusion (PDF), capable of turning almost any mobile phone into a 3D scanner.
Peter Ondruska, a Ph.D. candidate at Oxford University and a Microsoft Research intern on the project, says of the app, “What this system effectively allows us to do is to take something similar to a picture, but it’s a full 3D object.” Unlike Google’s Project Tango, the upcoming smartphone with built-in 3D scanning capabilities, or the Structure Sensor, a 3D scanning peripheral for the iPhone, MobileFusion can operate on a smartphone without any added devices. And, unlike Autodesk’s 123D Catch, no Wi-Fi is needed for capturing a 3D model. This makes it ideal for researchers out in the field and away from an internet connection. Pushmeet Kohli, a principal research scientist with Microsoft Research, adds, “Everything happens on the phone itself.”
Shahram Izadi, another principal on the project, previously worked with Kohli to develop a similar tool called Kinect Fusion, working with the Microsoft Kinect. The two, along with Ondruska, wanted “go to the next level, in the sense of usability,” according to Kohli, by shrinking it down to work with a smartphone. What drives the ability to capture scans in 3D is an algorithm relying on the phone’s camera. Izadi says, “This is really about the accessibility and ubiquity of 3D scanning.” He continues, “The great starting point was to take a sensor that everyone has in their pocket, which is the camera you have on your mobile phone.”
The app is currently being perfected for current mobile platforms, such as Windows Phone, Android and iPhone and will be presented at the International Symposium on Mixed and Augmented Reality in October. From the looks of it, MobileFusion has the potential to 3D scan an object, creating a continuous, full-color model, in about 20 seconds. Whether or not the quality of the models will be as good as those created by devices specifically designed for 3D capture, like Tango and the Structure, is unclear to me. I would guess that they may resemble those of 123D Catch, which don’t really compare to those made with the Structure, in my opinion. But it could improve on the user-friendliness of the 123D Catch app, which I’ve found to be a bit of a pain to work with. Maybe we’ll know for sure in October! If you’d like to pore of the researchers’ paper now, however, you can read it here.