Michael Balzer is many things. He’s the host of the All Things 3D podcast, covering every 3D technology imaginable. He’s an engineer and developer, working with the Commodore Amiga in the early 90’s to create such pioneering programs as Perfect Vision, Color Splitter, the award-winning ChromaKey+, and he co-designed Perfect Sound 500 for Amiga with Anthony Wood, who is now the CEO of ROKU. Mike is also a loving husband, who 3D printed his wife’s skull in order to help doctors plan treatment for her brain tumor.
Now that 3D technology is finally coming into its own, though, Mike is an avid enthusiast of 3D scanning, 3D printing, VR, and more, leading him to be considered, in many realms, as a “power user” of a handful of devices. After exploring many 3D scanners, Mike tells me that he still finds the Structure Sensor, a 3D sensing device for iOS, to have the best value. And, for this reason, he has become one of the Structure Sensor’s biggest power users. More than that, Balzer has taken this $379 scanner and, almost single-handedly, created an entire ecosystem of peripherals, ranging from lens systems to a VR app with potential for surgical planning.
So, as this Structure Sensor engineer preps to unveil his medical VR app, ORSurgery, at Stanford’s MedX Conference this weekend, that Mike was able to give this Mike a complete demonstration of all of his hardware, plus his latest app that could lead to something of great significance for the medical world.
With a price under $400, the Structure, designed by Occipital, takes advantage of the iPad’s awesome computing power (and the iPhone 6’s, if you’re an expert like Mike) to create a mobile 3D sensing device. Improving upon the technology PrimeSense developed for the original Kinect (before being acquired by Apple), the Structure Sensor combines with the iDevice’s own camera to capture full-color 3D scans. To improve the quality of the Structure’s scans, Balzer designed 4eyes, a pair of glass lenses that, when placed over the scanner, give it, roughly, an extra 10 cm of scanning proximity, while increasing surface detail by 30%.
As Mike showed me over Google Hangouts, by hooking his iPhone 6 + Structure + 4eyes up to his Apple TV, this allows the device to get about 10 cm closer to an object, so as to capture smaller objects in greater detail. Though typical users, like me, might rely on Occipital’s own apps or the powerful and intuitive itSeez3D app, a power user, like Mike has his own app, NEODiSCAN+, which he designed for greater flexibility and control over the scanning process. With a few swipes and button pushes, his app was able to increase the clarity of and color of the scan instantly, fully taking advantage of the combined power of his iPhone and Structure.
A power user, really an unofficial team member, doesn’t just stop with an app and a lens system, though. Balzer has designed an external light for his iPhone, too. The NEODiBRIGHT LED Light Bar shines light all around a subject, reducing shadows and achieving consistent lighting. Given the huge improvements made to the camera on the iPhone 6s, however, Mike envisions, with Nostradamus like certainty, that aspiring filmmakers (and professionals?) will use Apple’s latest phablet for their video projects, for which such a lighting tool, and microphone adapters currently in the works, will be essential.
Other peripherals, like the NEODiMOUNT Grip & Shoot Handle add extra mobility and functionality to the setup. One interation of the NEODiMOUNT incorporates a Bluetooth connection that gives users the ability to control scans from the device from the handle itself, essentially turning it into a handheld scanner, à la the Artec EVA.
Occipital, however, isn’t looking just to create a 3D scanning device. Instead the firm wants to set the stage for what they call “Spatial Computing”, building augmented reality and virtual reality apps for the Structure, as well. Because the Structure can sense the 3D world around it and register a user’s location with positional tracking, it’s possible to use an iPad as a digital interface for a virtual environment laid over the physical environment. The most obvious demonstration of this possibility, so far, has been Occipital’s own S.T.A.R. Ops VR game embedded below.
As the co-host of All Things 3D, Balzer is well aware of what the future of 3D computing will look like. Just as Mike played a role in the early days of personal computing, with the Amiga, he is helping to pioneer the world of 3D computing with the Structure, pushing its capabilities to reach its full potential, as Mike envisions it. In fact, after witnessing Occipital’s VR demo at CES, this year, Mike kicked his Structure development into high gear, significantly advancing the creation of a VR headset designed specifically for the Structure.
To take advantage of the device’s VR capabilities, the engineer reached out to French VR headset designers Homido. The Homido headset was made to allow just about any smartphone to be clipped onto the front, a universal Google Cardboard system with a price of about $70. By combining a Homido headset with his own NEODiMOUNT case, for mounting the Structure to the iPhone, Balzer and Occipital were able to create what may be the first depth-sensing VR headset.
Now that he had a VR headset to go with his Structure Sensor, it was time for this power user to put it to a powerful use. Mike has since been working on a number of apps that bring depth VR to the iPhone, one of which, in particular, he’ll be demoing at Stanford’s Med X this weekend. Over Google Hangouts, I watched as Mike wore the VR headset and walked around a virtual operating theater, occupied by a team of doctors and a patient whose face bore a striking resemblance to the app’s creator. The heartbeat of the anesthetized patient echoed loudly, while medical equipment bleeped away. The whole time that the engineer explored the VR room, he was also wandering around his living room, thanks to Occipital’s positional tracking algorithms, embedding the user in a virtual environment mapped over the physical one, insuring that Mike wouldn’t bump into an ottoman like Dick Van Dyke.
Though I still haven’t gotten my hands on my own Structure or a new iPhone, Mike did send me a pair of cost-effective VR glasses to follow his video tutorial. Albeit, I couldn’t actually move around the OR room, the low-cost Go4D glasses allowed me to feel the experience Mike’s movements. And these $22 glasses have worked better than any other similar Google Cardboard-style headset. If you’ve got your own VR headset, you can follow along with Mike’s tutorial, too, using the video embedded below. In October, you should be able to download the app, as well as a medical scanning app he’s developed called BioStep Capture.
For now, ORSurgery only teaches users about the various doctors in the operating theater and their respective surgical equipment, but it’s easy to imagine how it could be extended, if Mike can get his hands on some investment capital. With a VR camera rig, for instance, an actual operating room could be grabbed, adding more authenticity to the experience. Attach a haptic device to the platform and med students could practice their surgical skills. Upgraded equipment, such as those offered by 3D Systems, could be used for actual rehearsals of complicated procedures.
In the meantime, Mike has a great proof-of-concept, while he works up similar proofs-of-concept for his other ideas, including an MRI simulator, which could help patients prep for the claustrophobic sensation of being placed into an MRI machine; “Art House”, a curated art gallery simulator where “only the good art is displayed, so you can skip all of the bad stuff”; and “Alps”, which may be one of the most intriguing, as it relates to Balzer’s work with burn victims. Alps was inspired by a study that Mike read suggesting that cooling imagery can soothe those with severe burns. The app takes this even further by providing a simulated environment, potentially providing even an greater calming effect.
This week, Mike is also heading to the children’s burn ward at Shriners Hospital, where he will be using his customized Structure Sensor system and accompanying apps to scan the ward’s patients. The resulting 3D models will be used to tailor clothes for the kids, when sizing them any other way might be difficult without causing pain.
Many of the peripherals for the Structure are already available for purchase on his company’s website (via Square and ecwid), but he’s planning on launching a Kickstarter in the near future, selling the products in bundles. And, while he can make some dough off of all of the effort he put into designing them, it’s heart-warming to know that he’s really trying to create useful and life-enhancing applications for them, too. One such application is helping his wife treat her tumor, about which he and his wife discuss in the TEDx talk below:
If anyone understands the direction that the 3D ecosystem is heading in, its Mike Balzer. As AR, VR, 3D scanning, 3D printing, and haptic devices merge into a single mixed reality network of devices, Balzer will be well-prepared to bring objects from the physical world, into the digital and vice versa. In fact, in the future, we may look back on his efforts as one of the pioneers of the mixed reality space in the first place.
