Augmented Reality with the HoloLens
Over an incredibly fun yet sleepless weekend at Microsoft's #holohacks in June 2016 at The Brewery Artist Lofts in Los Angeles, I worked with Kokiri Lab to develop a prototype of Project Nourished using Microsoft's Hololens prototype. We aimed to tackle the issue of making a device, then marketed towards primarily business and industrial application, at becoming a tool to help address the issues of childhood obesity by setting forth a more interesting way for children to look at their food and begin to imprint on them healthy eating choices for the rest of their lives.
Using the Hololens' deceptively easy SDK and Unity integration, I worked to build an event-driven story system that took a few pieces from the available Hololens Academy examples and expanded their functionality in stereotypical hacky hackathon fashion to get the pieces we needed to convey our vision quickly. Approaching this still-in-development device for the first time, I was eager to just play around and see what it was capable of. After a few hours of being lost in child-like awe with everyone else, I started to get a feel for the device's strengths and weaknesses and set out to adjust development to cater to it. We needed something simple, engaging, and able to be used hands-free at the dinner table.
What problems did we run into?
One important issue to deal with was the idea that you have a cropped field of view with a harsh invisible edge cutting off objects abruptly as you move your head. Rather than let things sharply disappear off-screen which would detract from user presence, I implemented a method that allowed for animated objects to gradually fade off screen as they either left the current field of view or went beyond the bounds of the dinner table. I love building within constraints like this and Hololens, for as incredible a device it is and as thorough of documentation it has , absolutely has a ton of limitations to be aware of and build for.
Another limitation was it's graphical capabilities relative to most current gen mobile devices. Given our hook, using a low poly cartoon-esc world really worked in our favor as we could have quite a bit of art onscreen yet still needed to be aware of shading performance. The artists and I made a few efficiency tweaks as well as applied special Hololens shaders that forgoes some passes, lowers sampling, and allows for a smooth opaque to transparent transition.
Showed it off, please work please work, and...
We won the social division of #holohacks LA and couldn't be more proud at what we learned! I'm really thankful for Microsoft having made themselves available over the weekend to make this happen. For more info, check out Kokiri Lab's VR project at www.projectnourished.com
Dates: June 2016
APIs/Tools: Unity 5, Modo, Hololens SDK
Languages: C#
