Sony Honda Mobility unveils their AFEELA Prototype at CES 2024

Sony Honda Mobility Inc. (SHM) will exhibit the AFEELA Prototype at CES® 2024 in Las Vegas, Nevada, USA beginning on Tuesday, January 9, 2024 (local time).

In advance of CES® being opened to the public, Sony’s press conference was held on Monday, January 8 (local time). Representative Director, President and COO Izumi Kawanishi took the stage to unveil the evolved prototype for the first time. In his speech, he introduced its progress on two aspects–“AI for ADAS” and “Mobility as a Creative Entertainment Space” under the theme of redefining the relationship between people and mobility. A summary of the speech and an overview of the exhibit are below.

Summary of the speech

*SHM aims to revolutionize how people move by redefining the relationship between people and mobility.

*SHM aims to realize AD/ADAS through the use of a number of sensing devices and AI in mobility.

*To create our new user experience in mobility with vehicle data and sensing data, SHM is developing new entertainment and gaming features utilizing virtual space with Epic Games.

*Aiming for mobility that continuously evolves according to user preferences, SHM will provide an open mobility development environment where creators and developers can freely develop applications and services that run on AFEELA.

*SHM has also entered into an agreement with Microsoft to develop a conversational personal agent, using Microsoft Azure OpenAI Service.

*SHM is starting a collaboration with Polyphony Digital in vehicle development. To develop vehicles that fuse the virtual and the real, mainly in the area of human senses by combining Polyphony Digital’s simulation technology with the development of actual vehicles by SHM.

AI for ADAS – Using AI for ADAS

SHM aims to realize AD/ADAS through the use of a number of sensing devices and AI in mobility. SHM will adopt Vision Transformer^* for perception and machine learning for path planning. Also, a redundant system will be developed to secure safety. In this system, Qualcomm Technologies’ high-performance SoCs for automotive applications are used.

New user experience in mobility with sensing data

At CES®2023, SHM announced a collaboration with Epic Games to create new value in mobility services and entertainment. At this time, SHM disclosed the progress of this partnership. SHM unveiled a simulator making use of virtual spaces to create our new user experience in mobility.

This tool simulates external environmental condition such as information of other vehicles, pedestrians, terrain, and weather. By combining this technology with AR, users will enjoy an immersive experience that also enhances safety. SHM will provide a visually stunning 3D map which extensive metadata from the internet is overlaid. Map data can also be used to broaden the possibilities within the development of gaming and entertainment features.

To realize these features by incorporating sensing and vehicle data, SHM uses the powerful gaming engine, Unreal Engine 5. SHM fully uses the latest version, Unreal Engine 5.3, and will continue to implement updates for the latest Unreal Engine advancements.

Mobility as a Creative Entertainment Space – Opening the mobility development environment (tentative name: AFEELA Co-Creation Program)

SHM will nurture AFEELA as mobility with intelligence. In addition, SHM will pursue new possibilities on top of conventional car values, so that mobility can become a unique and endearing presence for users. Rather than rely solely on keeping the knowledge closed inhouse, SHM will provide an environment where creators and developers can freely develop applications and services that run on AFEELA, as well as a digital platform for co-creation and the expression of creativity.

SHM would like to explore the relationship between people and mobility in a broad, deep, and open way. We want to promote an open development environment for the free expression of digital gadgets so that users will become endeared to AFEELA. We hope that this initiative will bring about changes in mobility, evolving into a service that connects mobility and society. Together with not only users but also creators and developers, SHM will share this creative community with the world.

Conversational personal agent 

SHM has also entered into an agreement with Microsoft to develop a conversational personal agent, using Microsoft Azure OpenAI Service.

Collaboration with Polyphony Digital in vehicle development

To develop vehicles that fuse the virtual and the real, mainly in the area of human senses by combining Polyphony Digital’s simulation technology with the development of actual vehicles by SHM.

[Exhibited Items]

Prototype – The concept remains unchanged from the prototype that was unveiled for the first time at CES® 2023 but has evolved toward mass production.

Simulator – To create our new user experience in mobility with vehicle data and sensing data, SHM is developing new entertainment and gaming features utilizing virtual spaces with Epic Games. The simulator, which allows visitors to experience the concept of UI overlaid with AR, is exhibited as an example.

Vision Transformer – Transformer (Deep learning model for natural language processing) specialized in image recognition.