DreamActor-M1 From ByteDance
Holistic & expressive human animation with custom controls
Listed in categories:
VideoDesign ToolsGitHub
Description
DreamActorM1 is a DiT-based human animation framework that utilizes hybrid guidance to achieve fine-grained holistic controllability, multiscale adaptability, and long-term temporal coherence. It can imitate human behaviors captured from videos, producing highly expressive and realistic human videos across multiple scales, from portrait to full-body animations. The resultant videos are temporally consistent, identity-preserved, and of high fidelity.
How to use DreamActor-M1 From ByteDance?
To use DreamActorM1, users need to provide a reference image or multiple frames from an input video. The framework then extracts body skeletons and head spheres from the driving frames, encodes them, and combines them with the video latent to generate expressive animations. Users can control various aspects of the animation, including facial expressions and body movements, through the hybrid guidance system.
Core features of DreamActor-M1 From ByteDance:
1️⃣
Fine-grained holistic controllability
2️⃣
Multiscale adaptability
3️⃣
Long-term temporal coherence
4️⃣
Robust control of facial expressions and body movements
5️⃣
Support for audio-driven facial animation with lip-sync capabilities
Why could be used DreamActor-M1 From ByteDance?
| # | Use case | Status | |
|---|---|---|---|
| # 1 | Creating expressive human animations for films and games | ✅ | |
| # 2 | Generating realistic avatars for virtual reality environments | ✅ | |
| # 3 | Producing animated content for social media and marketing | ✅ | |
Who developed DreamActor-M1 From ByteDance?
DreamActorM1 is developed by a team of researchers including Yuxuan Luo, Zhengkun Rong, Lizhen Wang, Longhao Zhang, Tianshu Hu, and Yongming Zhu, affiliated with Bytedance Intelligent Creation. The team has made equal contributions to the development of this innovative human animation framework.