Tencent has open-sourced a tool named MOFA-Video, which presents a very comprehensive and unique way of video control.
This tool has a variety of powerful features. Among them, it supports controlling the motion direction of video content with arrow keys, a control method similar to the operation principle of a motion brush. Users can use the direction indicated by the arrow to precisely control the elements in the video to move in the desired direction. For example, in a video scene containing movement of people or objects, users can use the arrow to make people move left, right, up, or down, or make objects move along a specific trajectory, thus achieving flexible control over the motion of video content.
In addition, MOFA-Video also supports a highly innovative feature, which is the ability to transfer facial expressions from the original video to a newly generated face video. This means that users can select a video containing rich facial expressions as the source video, then extract the facial expression characteristics and apply them to a new face video. Whether it's happiness, sadness, surprise, or other complex expressions, they can all be presented in the new video, bringing more possibilities and creative space for video creation.
What's more surprising is that the above two control methods are not mutually exclusive; they can be used simultaneously in the same scene. This provides video creators with a richer and more complex means of creation. For example, in a video scene, creators can control the motion direction of an object with an arrow while transferring the facial expressions of a character to another character, thus creating a unique and interesting video effect.
In order to achieve these two complex and unique control methods, Tencent's R&D team has conducted careful design and development. They designed multiple domain-aware motion adapters, which play a crucial role in the video generation process. They can sense different video domains, including motion domains, expression domains, etc., and control the motion during the video generation process according to the user's control instructions. Through the collaborative work of these adapters, MOFA-Video can achieve such powerful and diverse video control functions.