VACE is an all-in-one AI model jointly developed by Alibaba, Tongyi Lab, and the Wan team, specifically designed for video creation and editing.
It supports multiple tasks, including:
Reference-to-Video Generation
Video-to-Video Editing
Masked Video-to-Video Editing
What makes VACE unique is that users can freely combine these tasks to explore more creative possibilities and simplify workflows.
VACE offers a series of "Anything" features to meet various video creation and editing needs
Freely move objects in videos while maintaining natural visual effects
Replace objects in videos while maintaining consistency in motion and context
Generate videos based on reference images while maintaining style and content consistency
Expand video field of view, adding reasonable and coherent additional content
Bring static content to life with vivid animation effects, creating engaging videos
VACE utilizes Diffusion Transformer technology to generate and edit high-quality videos while maintaining consistency between temporal and spatial dynamics.
This unified approach simplifies user workflows, reduces the need for multiple separate tools, and improves overall efficiency in the video creation and editing process.
VACE's deep integration with Wan 2.1 enhances functionality for specific video editing tasks
Utilize Wan 2.1 to provide precise video control capabilities, implementing advanced features like pose control
Replace objects in videos through simple text prompts, such as changing a lemon into an apple
Use reference images to replace objects in videos, maintaining style consistency and contextual integration
In the MimicPC workflow, users can input prompt words in the "WanVideo TextEncode" node to replace objects in videos.
Additionally, enabling the "WanVideo TeaCache" node can accelerate video generation, though it may reduce video quality.
Users can adjust parameters such as width, height, frame rate, and number of frames to customize video resolution and length. Community discussions suggest setting Step=30 for good 2D video effects and Step=50 for clearer real-person facial textures.
VACE supports inputs of any resolution, but optimal results are achieved within specific video size ranges
| Model | Download Link | Video Size | License |
|---|---|---|---|
| VACE-Wan2.1-1.3B-Preview | ~ 81 x 480 x 832 | Apache-2.0 | |
| VACE-LTX-Video-0.9 | ~ 97 x 512 x 768 | RAIL-M | |
| Wan2.1-VACE-1.3B | Coming Soon | ~ 81 x 480 x 832 | Apache-2.0 |
| Wan2.1-VACE-14B | Coming Soon | ~ 81 x 720 x 1080 | Apache-2.0 |
Perform end-to-end inference using the command-line interface provided by the official GitHub repository:
python vace/vace_pipeline.py --base wan --task depth --video assets/videos/test.mp4 --prompt 'xxx'
Output will be saved to the ./results/ directory
Launch interactive Gradio demos using the following commands:
python vace/gradios/preprocess_demo.py
python vace/gradios/vace_wan_demo.py
python vace/gradios/vace_ltx_demo.py
Discussions on community platforms (such as Reddit) highlight VACE's advanced features, like Pose Control and ControlNets, which offer unique advantages compared to other models (like Hunyuan).
User comments like "ControlNets for videos? Awesome!" reflect excitement about its potential for precise video editing.
The community is also looking forward to its open-source release, making comparisons with platforms like Pika Labs, and generally expressing enthusiasm about its potential.
"This looks so cool!"
Reddit User
"ControlNets for videos? Awesome!"
Community Member
"At this rate, I'm gonna be the star of all the classics in a year or 2. $1.99 matinee fee!"
Tech Enthusiast
Explore more VACE-related resources to understand its features and applications
Provides examples and demonstrations, such as video re-rendering with content, structure, subject, posture, and motion preservation
Visit Official PageProvides additional models and resources, supporting different application scenarios and tasks
Browse Hugging Face CollectionProvides models and resources in a Chinese environment, suitable for Chinese users
Browse ModelScope CollectionProvides datasets and tools for evaluating video generation and editing quality
Explore BenchmarkProvides tools for video annotation and data preparation, supporting model training and evaluation
Get Annotation ToolsCommunity-driven YouTube tutorials demonstrating how to use VACE with tools like ComfyUI
Watch Tutorial Videos