SOAP: Style-Omniscient Animatable Portraits

¹MBZUAI   ²Westlake University   ³Pinscreen

Paper

Code

Video

Abstract

SOAP awakens the 3D princess from 2D stylized photos. Unlike other works that directly drive the 2D photos, SOAP reconstructs well-rigged 3D avatars, with detailed geometry and all-around texture, from just a single stylized picture. The main challenges lie in 1) How to accurately deform and correctly rig one head template, to adapt to a diverse range of styles, shapes, and haircuts. 2) How to maintain the cross-view consistency, both in color and shape. To tackle these challenges, we created a large-scale 3D head dataset with 24K avatars and trained a multi-view diffuser. This model generalizes across realistic humans and cartoon characters, significantly outperforming those generic ones. Additionally, we introduce an adaptive fit-and-rig pipeline that deforms, rigs, and paints parametric head models, i.e., FLAME, to produce high-resolution (20K+ faces) mesh-based avatars with consistent texture, teeth and eyes, fine-grained skinning weights and head parsing labels. This enables the stylized 3D avatar to be freely animated with extreme expressions, natural eye movements, and lifelike lip motions.

Method

Reconstructions

Animation

Applications

Mesh Gallery

Video

BibTeX

@article{  
    title={{SOAP: Style-Omniscient Animatable Portraits }},
    author={Liao, Tingting and Zheng, Yujian and Karmanov, Adilbek and Hu, Liwen and Jin, Leyang and Xiu, Yuliang and Hao Li},
    booktitle={International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH)},
    year={2025}
}