IMAvatar

3D morphable face model based on neural implicit fields

Released in: I M Avatar: Implicit Morphable Head Avatars from Videos

Contributor:

Summary

Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details. Neural volumetric representations approach photorealism but are hard to animate and do not generalize well to unseen expressions. To tackle this problem, we propose IMavatar (Implicit Morphable avatar), a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, this work represents the expression- and pose- related deformations via learned blendshapes and skinning fields. These attributes are pose-independent and can be used to morph the canonical geometry and texture fields given novel expression and pose parameters. The model employs ray marching and iterative root-finding to locate the canonical surface intersection for each pixel. A key contribution is a novel analytical gradient formulation that enables end-to-end training of IMavatars from videos. The authors show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods.

2022

Year Released

Key Links & Stats

IMAvatar

I M Avatar: Implicit Morphable Head Avatars from Videos

@InProceedings{zheng2022IMavatar, title={{I} {M} {Avatar}: Implicit Morphable Head Avatars from Videos}, author={Zheng, Yufeng and Abrevaya, Victoria Fernández and Bühler, Marcel C. and Chen, Xu and Black, Michael J. and Hilliges, Otmar}, booktitle = {Computer Vision and Pattern Recognition (CVPR)}, year = {2022} }

ML Tasks

  1. Face Animation
  2. Facial Modeling

ML Platform

  1. Not Applicable

Modalities

  1. 3D Asset

Verticals

  1. Digital Human

CG Platform

  1. Not Applicable

Related organizations

ETH Zurich

Max Planck Institute for Intelligent Systems, Tubingen

Max Planck ETH Center for Learning Systems