This work targets a novel text-driven whole-body motion generation task, which takes a given textual
description as input and aims at generating high-quality, diverse, and coherent facial expressions, hand
gestures, and body motions simultaneously.
Previous works on text-driven motion generation tasks mainly have two limitations: they ignore the key role
of fine-grained hand and face controlling in vivid whole-body motion generation, and lack a good alignment
between text and motion.
To address such limitations, we propose a Text-aligned whOle-body Motion
generATiOn framework, named
HumanTOMATO, which is the first attempt to our knowledge towards applicable holistic motion generation in
this research area.
To tackle this challenging task, our solution includes two key designs: (1) a Holistic
(aka H²VQ) and a Hierarchical-GPT for fine-grained body and hand motion reconstruction and generation
with two structured codebooks;
and (2) a pre-trained text-motion-alignment model to help generated motion align with the input textual
Comprehensive experiments verify that our model has significant advantages in both the quality of generated
motions and their alignment with text.