While mixed reality is already impacting the field of medicine, with doctors using patients specific 3D models to create 3D printed implants and surgical models, the next big industry to be affected by this emerging ecosystem could be the multi-billion dollar world of gaming, set to reach $107 billion by 2017. A slew of VR and AR systems will be hitting the market this year and, coupled with 3D scanning personal devices from Google and Apple, we will likely see the digital realm and physical realm blend more and more. As that occurs, there’s no doubt that people will want to see their own faces reflected in the apps that they play. And one research outfit at the University of Southern California is already working to make that a reality, using an inexpensive toolkit for personal 3D scanning.
Lead by Dr. Ari Shapiro at the USC Institute for Creative Technologies, the SmartBody project allows users to scan themselves with a variety of low-cost scanning systems for 3D animation and other applications. Once a 3D scan is performed with a Microsoft Kinect, Intel RealSense 3D camera, or Structure Sensor from Occipital, the SmartBody software is capable of performing a variety of functions for a user’s personal avatar. This includes rigging the mesh for use in games and 3D videos, as well as manipulating the mesh to make it taller, shorter, fatter, or thinner. The team has even developed gesture recognition and lip syncing features for recording motion with a Kinect or programming a 3D model to move its mouth along with an audio clip.
Dr. Shapiro, the head of the Character Animation and Simulation research group, tells The Daily Mail, “We’re looking at all aspects of what makes people, people. We’re looking at how we can model a person so that it’s convincingly a person, so that it captures the essence of them. We’re trying to create a digital person that thinks, acts, and behaves like a regular person.”
As many 3DPI readers probably know, advanced CAD packages and 3D scanners are outrageously expensive. So, to create an accurate 3D model of oneself is not an easy task for average consumers. Fortunately, a number of low-cost 3D scanners, like those being used by the USC research group, have put entry-level 3D scanning into the hands of users. For this reason, Shapiro is working on software that will enable the use of 3D scans in gaming with greater ease.
He tells USC News, “We’re giving everyone the ability to scan and animate themselves for free. We’re trying to foster innovation. While tools to create games and capture 3-D exist, the toolchain to bring the entire process together typically requires expert artistic intervention and a complicated set of processes. We are providing tools and software that, without any expertise, allow you to create and animate a 3-D version of yourself in four minutes The community can now develop interesting applications with it. The applications could extend beyond games and into social media, communication, training and more.”
The applications, Shapiro says, go far beyond video games. Having a virtual avatar is useful for long-distance communication, seeing a virtual doctor, or training the military and police force. Describing the possible uses of an avatar as a “proxy for social interaction”, Shapiro says, “The body communicates a lot. When you know a person, there’s meaning to [the interaction]. I can see a revolution in social interaction using your own 3-D avatar as a means of communication.”
Over the past couple of weekends, I’ve been working on incorporating my own 3D likeness into a Unity game populated by 3D scans of my own apartment (above). Though it doesn’t use Shapiro’s tools, only available for Windows at the moment, the joy of animating myself, a character that I’m quite attached to, is pretty indescribable. If they’re not already working on it, I’m sure that, as the mixed reality ecosystem takes off over the course of the next, there will be plenty of platforms urging you to import your own environment into a game. Or exploring the world (i.e., Facebook) in VR, you’ll soon be able to 3D print items directly from your virtual environment into the real world. Mark my words!
