Omnihuman: The new AI of Bytedance creates realistic videos from one photo

Bytedance Researchers have developed an AI system that turns single photos into realistic videos of people who talk, sing and move naturally – a breakthrough that can change digital fun and communications.

The new system called OmnihumanHe generates videos with the whole body that show that people gestures and move in ways that match their speech, exceeding the previous AI models that could only revive faces or upper bodies.

How Omnihuman uses 18 700 hours of training training to create a realistic movement

“Human animation end -to -end has suffered remarkable progress in recent years,” the researchers from Bytedance wrote in A document published on ARXIVS “However, existing methods are still struggling for scaling as large common video production models, limiting their potential in real applications,” ”

The team trained Omnihuman more than 18,700 hours of human video data using a new approach that combines many types of inputs – text, audio and body movements. This “Universal Conditions” training strategy allows AI to learn From much larger and more diverse data sets than previous methods.

Ai Video Generation’s breakthrough shows the movement of the whole body and natural gestures

“Our key idea is that the inclusion of multiple conditioning signals, such as text, audio and posture, can significantly reduce data waste during training,” the research team explained.

Technology is making significant progress AI generated mediaDemonstration of opportunities that range from creating videos of people who are speaking to depicting topics that play musical instruments. When testing, Omnihuman outperforms existing systems through many quality indicators.

Tech Giants Race to develop next -generation AI video systems

Development appears against the background of increased competition in AI video generation, with companies such as Google., Meta and Microsoft Pursuit of such technologies. Bytedance breakthrough can give an advantage to Tiktok’s mother in this rapidly developing field.

Experts in the industry say that such technology can transform entertainment production, educational content and digital communications. However, it also raises concerns about the potential abuse of the creation of synthetic media for deceitS

Researchers will present their discoveries at an upcoming computer vision conference, although they have not yet indicated when or which of them.

Daily information on business use cases with VB Daily

If you want to impress your boss, VB Daily covers you. We give you the inner spoon for what companies do with generative AI, from regulatory changes to practical implementation, so you can share information about the maximum return on investment.

Read our Privacy Policy

Thanks for subscribing. See more VB newsletters hereS

An error occurred.