After “AI Sun Yanzi,” another tech breakthrough has emerged.
Recently, videos of top singer Taylor Swift (nicknamed “Mei Mei” by Chinese fans) speaking fluent Mandarin have been widely spread across major social media platforms. In the video, when Swift speaks Chinese, her pronunciation is precise, her tone resembles her native tongue, and even the lip movements match perfectly.
Furthermore, viral videos circulating online also feature Mr. Bean and Emma Watson speaking Chinese and Cai Ming speaking English. Beyond English-Chinese exchanges, international users have even tried translating English into Japanese and other six languages, with impressive results.
How is it achieved?
These videos have been made using the AI video tool called HeyGen. Creators simply upload the video and choose the desired language for translation. The tool then automatically translates, adjusts the tone, and matches lip movements.
However, HeyGen isn’t perfect. Video subtitles need to be added by the creators. Also, some users have noted that when characters in the video speak another language, the emotional accuracy can be lacking.
Despite these flaws, HeyGen’s popularity is surging. A paid version has been released, but it offers a free trial that produces videos up to 1 minute long. Some creators mentioned a backlog of 7,000 videos awaiting translation.
Chinese Company Behind HeyGen Turns Profitable Within 7 Months of Launch
HeyGen is a tool designed for video creators, but translation is not its only feature. At a glance, HeyGen has powerful capabilities revolving around video creation, including face-swapping for broadcasters, text-to-video voiceovers, virtual host creation, and voice cloning.
The creation of digital humans is its flagship feature. As per HeyGen, even without a camera, studio, or actors, ordinary individuals can make videos. Users only need to upload a short 2-minute clip to produce a digital human resembling a real person, capable of adjusting even subtle gestures and facial expressions.
Joshua Xu personally tested the platform, producing a digital version of himself.
Currently, their clientele includes NVIDIA, Amazon, Volvo, and more. The product supports over 300 voices, 40 languages, and offers more than 100 digital human templates and materials for sectors like advertising, e-commerce, news, etc. It also allows users to create content from a PPT.
Data from Tianyancha indicates that Shiyun Tech, the company behind HeyGen, has secured investments amounting to tens of millions from venture capitalists including Sequoia China and ZhenFund.
It’s noteworthy that HeyGen is already a profitable AI application. They have devised detailed pricing plans, categorizing between individual and corporate versions with monthly or annual payment options. There are premium plugins available at an added cost. For ordinary users, the annual plan averages to around $24 per month.
In a blog post released at the end of April this year, the company revealed that 178 days post-launch (launched in July 2022), the company’s annual recurring revenue (ARR) had reached 1 million USD, achieving profitability in just seven months.
Joshua Xu has stated that HeyGen has accumulated millions of video data and is considering the development of a large model (LLM) specifically for the video domain.