I tried 3D video chat through Google Starline. This is a smooth work in progress

At the Code 2023 conference, I tried out the latest version of Google’s Project Starline, the company’s tool for making video chats more engaging by adding depth, such as 3D video. It’s tough for a crowd tired of years of Zoom chats on tiny computer screens during pandemic quarantines, but I found it refreshing to have more humanity in remote calls.

Project Starline, which was first announced at Google I/O 2021, is a supercharged video chat that uses cameras and software to simulate depth on a flat screen, so that a person can see through the screen with an artificial 3D effect. going out. CNET’s Scott Stein was impressed with the first publicly available version of Starline he tested last October, which took up a booth full of technology and sensors. The new, much more portable version does a similar degree of remote chatting, using 3D to convey much more body language cues than a 2D video call. Google believes this is more engaging for both people in the conversation.

“We’ve found, especially at Google, that you can use it to build a strong, new relationship — because you trust people,” Andrew Nartaker, general manager of the Starline project, told me in preparation for the launch. , you understand them.” I got to work with the new Starline, which was introduced in May.

I walked into the Project Starline demo room, a hotel room decorated with white decorative trim on the walls, to sit in front of a desk and the Starline unit itself. The latest version of Project Starline has shrunk to the size of a modified flat-screen TV, ringed by three sets of twin cameras, one on each side and on top.

All six cameras track my exact posture and movements, not just to relay my position to whoever I’m talking to, but to see exactly where my eyes are—and each of my eyeballs is a They show the same picture with a little difference. A position to create a 3D effect This is like a 21st century version of the blue and red 3D glasses of yesteryear.

Google found that colleagues who knew each other were able to use standard video calls just fine, but those who hadn’t met in person couldn’t connect via Zoom calls.

Instead, the Starline uses a TV-sized display, and its cameras show your subject from the waist up, but with depth to show people moving, moving, and looking directly at your screen. There is a small arc of fabric that curves outward at the bottom of the large screen, which guides the user’s eye to how far the 3D image of their conversation partner extends from the screen. The Starline team believes that this setup is much better for people to communicate remotely.

A man is sitting at a table in front of a Starline project television screen, holding his hand a few feet from the screen "Catch" The apple that his interlocutor was holding.

Google’s Project Starline prototype allows participants to have a remote conversation with depth-of-field effects—though it’s hard to see from this angle, the seated reporter extends his hand below where the other person is holding the apple.

David Lumb/CNET

“You read body language and all these little things that are missing [in conventional video calls] Nonverbal cues, hand gestures, gestures, and even leaning toward or away from the subject — all of that stops in video calls because fit people tighten up, Nartaker said. In the frame of a small webcam

3D chat, across the office or the world

Starline has the advantage of being limited to one-on-one calls rather than Zoom group chats as well as being on big screens, but as I sat through my demo, I could see how the new and improved Starline would help participants. invites to link. More comfortable with your conversation partner.

As for me, I spoke to Nartker from several rooms away, each connected to his Starline prototype via the hotel’s wired internet network. While Google didn’t share the exact speeds it was getting, it’s clear that the Starline network doesn’t require a specialized network — as far as I can tell, any standard work or home network should be fine for a lag-free connection. Nartaker came in clear and didn’t hold back, although there was occasional roughness around the edges of his arms and lower body, which I chalked up to the complexity of compositing for depth.

As we talked, I tried to pick up on these unspoken body language tics that Nartaker described. Seeing her angle towards me or away from me was subtle feedback on whether she was interested in what I was saying – something much easier to gauge with the 3D effect on the screen, which was convincing (we’ve come a long way). We’ve gone through stereoscopic 3D on the Nintendo 3DS). This 3D persisted despite me wearing glasses (apparently his cameras were looking through mine), and it was comforting to look into Nartaker’s eyes as we chatted. As we spoke, I found myself leaning inward and pointing more clearly, which naturally matched the level of Nartaker’s gestures.

Because I participated more, I thought less about the emails I had to write or the meetings I had to attend. In video calls, people multitask because they don’t get caught. But Nartaker held out an apple and I could see it hovering on the other side of the TV screen. He could tell where I was looking, and thus steer the conversation spatially—which would be useful, for example, if he was referring to a corporate presentation or guiding care during a telemedicine call.

A man is sitting behind a table in a white hotel room in front of a television "Starline project" on that.

At the Code 2023 conference, Google showed its Project Starline prototype to companies and got interesting feedback.

David Lumb/CNET

But Starline’s team noticed something else: When using their system for chat, participants remembered more than the video call. Nartaker explained that because people remember the world through spatial memory—where people stood, what they were wearing, how tall they were, what the room looked like—Starline Chat gives people a clearer recollection of what happened to have

“There’s little difference in how you walk away from a productivity session or something thinking more about what happened,” Nartaker said.

I remembered a lot of our 10-minute Starline conversation, especially how Nartaker moved and pointed, what he pointed at, and how he behaved while talking. Obviously, it has advantages for corporate applications, where a Starline setup can bridge the gap between low-cost video calling and expensive in-person travel. While Google hasn’t revealed the cost of a Starline unit (it’s still a prototype), it’s likely to be cheaper than private jet flights for executives or travel accommodations for entire teams.

The first of many Starline products?

Google announced its new Starline via blog post in May around Google I/O 2023, and is taking prototypes of the phone to select locations to drum up interest. The team brought up a Starline (Nartker accidentally pointed it out, suggesting that it was more than just a project name, and maybe inside a project name. producing product name) to the Code Conference to invite potential partners to try it out for themselves (they’ve already worked with Salesforce to refine the Starline system) as well as develop new ways to use it for feedback to the Google team.

“It’s one of the ideas that came up repeatedly at the code conference with this crowd that’s really trying to eliminate travel,” Nartaker says. “Where am I going today, and why do I usually go in person to these important important meetings or negotiations or business deals. Can I do them via Starline instead?”

Currently, both participants in a one-to-one call need a Starline unit to use the system, which means sending TV, camera and speaker units to the location and having a Google engineer on site to operate it. It takes a full day to set up, so it’s still not possible to just roll into Starline and instantly sync it for a corporate call.

But the prototypes are much more mobile than older versions, Nartker said, and point the way to scaling Starline down to smaller proportions thanks to Google’s innovations in artificial intelligence. And this is just one of several formats that the team is tinkering with.

“We are thinking [Starline] As a very flexible communication technology that can take different forms and factors. You can imagine the crowd coming.”

The team continues to improve by identifying the participant’s objects and bodies and replicating them on both sides of the contact. With enough development and software innovation, one day video chatters may not need to set up multiple custom cameras to achieve the same depth effect in order to feel more connected and maintain more video chats. Nartker was coy about when we’d be able to use the Starline on a laptop with a webcam, or if it would be possible, but he didn’t rule it out.

“I think in the long arc, that’s a big North Star goal at this point,” Nartaker said. “We’re really looking at this kind of prototype where, with enough cameras, we can create an effect that’s useful for people being together. Maybe over time, we’ll be able to do that with simpler systems.” give.”

#video #chat #Google #Starline #smooth #work #progress
Image Source : www.cnet.com

Leave a Comment