Engaging autopilot

Residents enjoy augmented reality-powered tours of Hangzhou's scenic spots aboard the Asian Games Village's self-driving bus service

Two petite buses continually navigate the thoroughfares of the Hangzhou Asian Games Village. What sets them apart is their notable lack of human drivers.

Moreover, when settling into one of these vehicles for a village tour, a remarkable feature comes to life on the windows. As you peer outside, not only do you witness the natural splendor of the surroundings, but also the breathtaking, but virtual, landscapes of Hangzhou.

These buses can be aptly described by two key terms: "autonomous driving" and "augmented reality" (AR), embodying the cutting-edge fusion of technology and imagination.

Using advanced technologies like visual perception, light detection and ranging (LIDAR), and multi-sensor fusion, the AR smart buses in the Asian Games Village have achieved a high degree of autonomous driving within specific areas and under particular conditions.

Onboard, there is a 30-degree, 8-mega-pixel camera capable of perceiving vehicles at a range of 600 meters and pedestrians within 300 meters. It can autonomously devise and execute avoidance strategies.

Once passengers are seated in the AR smart bus and have securely fastened their seatbelts, Tang Rong, the safety officer, just needs to simply press the automatic driving button and the vehicle seamlessly transitions into fully autonomous cruising mode.

Tang explains that these vehicles are entirely self-driving, and in typical situations, human intervention isn't necessary. "We only assume control in case of emergencies to ensure passengers' safety," Tang said.

The vehicle has the capability to continuously create a 3D model of the surrounding traffic environment in real-time. This enables it to engage in multi-modal behavior predictions, thereby enhancing safety measures for its journey.

According to Zhu Hongbo, a commercial director from Sense Time, the AR smart buses in the Asian Games Village come equipped with onboard AR capabilities, creating a virtual reality environment and space.

"As passengers gaze through the vehicle windows, they can enjoy a seamless, interactive experience that combines the virtual and physical worlds outside," Zhu said.

As the vehicle advances, the screens on the front and right side of the bus not only present real-time images of the external surroundings, but also superimpose virtual scenes, including the Hangzhou Asian Games mascots, various sports events and picturesque Hangzhou landscapes.

The fusion of these elements, accompanied by sound effects, music and mascot narration, conjures a mesmerizing spectacle where the boundaries between the virtual and real worlds blur before the passengers' eyes.

With charming scenes of quaint bridges over meandering streams, blooming lotus flowers on the West Lake, thrilling dragon boat races, and intense volleyball matches, the AR production team has dedicated 18 months to meticulously crafting and refining their work.

Through extensive testing and adjustments, they have strived to achieve a flawless fusion of cutting-edge virtual imagery and the authentic backdrop of the Asian Games Village.

Since April, the smart bus has commenced a series of safety tests and began offering services for various events. Each of the two sightseeing routes has been meticulously designed.

"We conducted numerous on-site surveys and road tests during the preliminary stages, ultimately finalizing the setup of two AR sightseeing routes in the athletes' village and media village," stated Zheng Sihan, the traffic and logistics center service scheduling supervisor for the Asian Games Village operation team.

"Our aim is to prioritize safety while also showcasing the theme of 'China's unique features and the intelligent Asian Games', providing visitors to the village with a digitalized mobile experience," Zheng said.