Google IO: With Project Starline, Google dreams of almost face-to-face virtual meetings

Google is making in-person communication possible through a video call with Project Starline.

For over a year, a large number of people have not hugged their friends, family, and close ones because the pandemic made physical contact dreadful. And even though the pandemic is receding in some countries, people are far from being comfortable going back to the old days, leading to what doctors call touch deprivation. No one can tell when things will be back to being normal, but Google is certainly trying to bring people together as if they are right in front of them. At Google IO, the software giant announced Project Starline, a breakthrough in the world of technology that combines both hardware and software to create life-size images and videos of the person you are on call with. As Google describes it, “imagine looking through a sort of magic window, and through that window, you see another person, life-size and in three dimensions. You can talk naturally, gesture and make eye contact.” To put it simply, Project Starline uses technological advances of both hardware and software to create a 3D avatar that looks exactly like how the person you are calling would. But these are better simulated and feel real. Although you cannot touch the person on the other side of this screen -- which is how Project Starline’s face-to-face interaction is supposed to work -- you can talk and make gestures and eye contact with this person the same way you would do in real life. Project Starline tries to make online communication a lot better and to achieve that, there is a mix of several technologies working in tandem under the hood. According to Google, the project is a result of “research in computer vision, machine learning, spatial audio and real-time compression.” And all of this is happening in a highly advanced setup that has something called the light field display system, which Google thinks is a breakthrough to create a sense of volume and depth to the video of the person, without requiring a mixed-reality headset or high-tech glasses. “The effect is the feeling of a person sitting just across from you, like they are right there.”