BakedAvatar

BakedAvatar is a new representation for real-time neural avatar synthesis that can be deployed in standard polygon rasterization pipelines. The method extracts deformable multi-layer meshes from learned head isosurfaces and computes expression, pose, and perspective-dependent appearances that can be baked into static textures, thereby providing support for real-time 4D avatar synthesis. We propose a three-stage neural avatar synthesis pipeline, including learning continuous deformations, manifolds, and radiation fields, extracting hierarchical meshes and textures, and fine-tuning texture details through differential rasterization. Experimental results show that our representation produces comprehensive results comparable to other state-of-the-art methods and significantly reduces the required inference time. We further demonstrate a variety of avatar synthesis results from monocular videos, including view synthesis, facial reconstruction, expression editing, and pose editing, all at interactive frame rates.

Demand group:

"Real-time avatar synthesis in VR/AR, telepresence, video games and other applications"

Example of usage scenario:

Real-time avatar synthesis in VR/AR applications

Avatar synthesis in video games

Real-time avatar synthesis in telepresence

Product features:

Learn about continuous deformations, manifolds, and radiation fields

Extract layered meshes and textures

Fine-tune texture details with differential rasterization