Update: China's technology industry overseer has begun an official inquiry into how the developers of viral face-swapping app Zao handle user data, responding to an online outcry over privacy.
The Ministry of Industry and Information Technology asked executives at Momo Inc. to "rectify" the app, according to a statement posted to its official social media account. It asked Momo, better known as a developer of hookup and live-streaming services, to only collect and use personal data according to local laws. The company should also assess security risks on new platforms and prevent the use of information in online fraud, according to the statement.
Zao's developers said in a statement they would abide by the laws, regulations and requirements of the authorities, and would implement higher standards to secure user data.
The app launched recently and went viral in the world's biggest smartphone market, topping the free downloads chart on China's Apple iOS store.
Zao goes viral
An app that allows users to virtually graft their faces onto actors in scenes from movies and television shows became the most popular iPhone download in China over the weekend, raising questions about the speed with which "deepfake" videos can blur truth online.
To use Zao, an iOS app developed by a subsidiary of Chinese developer Momo, users upload an image of themselves. Then, the program automatically transposes the image onto an actor's face from a selection of video clips in just a few seconds. Those clips include scenes from movies and television shows with celebrities such as Leonardo DiCaprio, 'Game of Thrones' actor Kit Harington and others.
"Best application of 'Deepfake'-style AI facial replacement I've ever seen," wrote Twitter user Allan Xia.
Zao is only available in China. After launching on Friday, the app quickly gained popularity and was the most downloaded iPhone app in China on Monday, according to data and analytics company App Annie.
But the app also quickly created controversy. According to RADII China, the app's user agreement initially gave Zao widespread rights to use the photos and the videos generated with its technology free. The company later amended its terms of service.
Momo did not immediately respond to a request for comment.
While photoshopping has long made images susceptible to realistic-looking changes, deepfake videos are the latest frontier for manipulation as technology advances. Made popular by powerful and widely available artificial-intelligence software developed by Google, these realistic videos have quickly multiplied across the Internet.
Some of the more sinister uses have included revenge videos with women's heads pasted into porn videos. And many researchers in the field are concerned about potential uses of this type of technology as the United States approaches the 2020 elections.
Zao's popular app also raises privacy questions following the rise to the top of the app stores this summer of FaceApp, a program that takes photos of people and "ages" them using artificial intelligence. Privacy advocates quickly began warning about the Russian-made app's vague legalese. Suspicions were raised that the app could be part of a disinformation campaign or secretly downloading photo albums. Leaders of the Democratic Party warned campaigns to delete the app "immediately."
Among the high-profile individuals who have had their own images weaponized against them is House Speaker Nancy Pelosi, D-Calif., who was the subject of a video distorted to make it sound as though she were drunkenly slurring her words. The video and the speed with which it was disseminated made it clear that even simple manipulations could be used to shape public perceptions.
When Facebook said it would not remove the Pelosi video, stating that it did not violate any of its policies, two artists released a deepfake video of Facebook CEO Mark Zuckerberg bragging about misusing "stolen data" from users. That video also remained on the platform.
User @nikkmitchell tweeted that he was in awe of the app. "On a side note to the developers, please don't be evil," he added.