Nearly a year in since it was released commercially, consumer VR still has a lot of kinks it needs to work out, one of which is how to show would be consumers what VR is like, without actually having to experience it in person.
Google Research in collaboration with Daydream, took it to another level by not only allowing us what one engaged in VR sees but we get to see their face as they go about in VR. In essence, Google wants to see the entire head.
We have seen videos of people engaged in Virtual Reality posted on YouTube. However, we cannot actually see what they are seeing in virtual space. That is now solved as can be seen in the videos covered in a previous article here. The thing is the headset obscures their face.
Google Research and Daydream came up with a solution by creating a spin in mixed reality. They made the headset transparent allowing us to see the person's face while engaged in VR.
For those of us seeing a person engaged in VR, or we are in the VR environment with them, we see their gestures as they move about in a simulated environment. However, we cannot actually see them, more so their face, except for the gestures they make inside. This is where Google Research and Daydream come in.
Google Research and Daydream collaborated and called it, rather blandly as, Headset Removal. According to Google's Keyword Blog, they used a combination of machine learning, 3D computer vision and advance rendering techniques, to remove the headsets and show the person's face in mixed reality. The process involves creating a digital construct of a person's face.
Google Research and Daydream constructed a 3D model of the subject's face by scanning and took eye movements as they blink and look at different directions. Accordingly, the head or the face is mimicked digitally even though it is obscured by the headset and placed over the headset making the subject look as if wearing a transparent scuba mask.
Finally, Google Research and Daydream superimposes the scanned head of the subject and composite it into the virtual plane complete with expressions. According to the blog, this allows viewers to see expressions of those engaged in VR.
Practical applications of this Google Research and Daydream collaboration seeing the subject's reactions while in a conference or playing a favorite game together. It is just the beginning, said Google. They are currently exploring ways to make the project accessible to YouTubers, according to Engadget.
