Microsoft aims for more engaging video conferences with 'Together Mode'

As "work from home" becomes the new normal owing to the COVID-19 pandemic, an nonnegligible byproduct has emerged from the latest work practice: fatigue or distraction due to back-to-back video conferencing.

Now Microsoft Corp is riding to the rescue. Its teamwork app Microsoft Teams has unveiled this week a new option called "Together Mode" to make meeting participants more attentive to one other and more focused on shared goals.

By employing AI segmentation technology, the model unites meeting attendees in a common virtual background such as in a theater or an auditorium, instead of being separated in boxes in the conventional way. The functionality will become available to customers in August.

The design serves to allow participants to more easily pick up on nonverbal cues, and make back and forth conversation feel more natural, said Jaron Lanier, a researcher and scientist at Microsoft.

"The initial design has been optimized to help people during the pandemic. It brings less fatigue from frequent video meetings, a better sense of connection with others, and more effective meetings," he said.

According to Lanier, early-stage experimenting has shown people responding more positively to meetings under the new mode, as assessed by gauges like testing for retention of information memory of who is present, logging time spent, how much time was devoted to tasks like negotiation, and the frequency of people turning off their cameras.

In developing the new mode, technologies are combined with the latest scientific understanding of cognition and social perception. For instance, people in Together Mode know where others are in a shared virtual space. That means your brain can keep track of what other people are signaling or emoting in a natural way, relying on social/spatial perception, and people can intuitively signal each other nonverbally.

"In a grid, you don’t know where other people are on the screen, relative to you, from their point of view, so natural glances and other subtle cues are impossible," he said.

The new mode also helps mitigate the common eye contact problem that is common in classic video conferencing: people appear to be looking in the wrong direction because of the upper-center position of the camera on a device.

Using an illusion that is based on unique geometry, everyone is looking at the whole group through a big virtual mirror, so that people intuitively position themselves to look as if they are reacting to one another appropriately, thus enhancing the level of engagement.

Lanier said currently choice and reservation of seats are not yet available, and the company is looking to extend the Together Mode to other online conferencing scenarios, as well as exploring simultaneous translation of live transcriptions in the future.

The functionality happened to be launched in sync with the ongoing World Artificial Intelligence Conference 2020 in Shanghai, where Microsoft’s AI chatbot Xiaoice made an appearance with three other virtual avatars by Chinese tech companies in a music video during the event’s opening ceremony.

The company is also unveiling other new functionalities like dynamic view and video filters.