Meet Sora, OpenAI's AI that generates lifelike video from text input

Artificial intelligence expert OpenAI has taken a quantum leap forward with its new AI called Sora. It is a new text-to-video model. That is, it generates realistic-looking videos from user-entered text prompts. Along with the announcement, Sam Altman’s company also discussed the steps they are taking to prevent malicious use of Sora as disinformation or hate speech.

OpenAI’s Sora is the new AI that creates videos with the text you type

OpenAI introduced Sora last February 15th through a short series of posts on its official account on X, Elon Musk’s social network. In the first of these posts, we can see a generated video showing a vibrant Tokyo with snowflakes and lots of cherry blossom petals. The result, as you can see in the video itself, is so realistic it’s scary.

In other words: Sora is to video what OpenAI’s DALL-E3 is to images. As the company reports, in order to prevent malicious use of Sora as much as possible, it is working hand in hand with a team of experts in areas such as misinformation or bias, who are conscientiously looking for details in the generation capabilities of this AI model. In other words, a group of specialists deliberately “misuses” Sora to prevent harmful uses of this artificial intelligence in the future. This would be conducive to Sora using hidden prompts to combat prejudices such as racism, as DALL-E3 does.

Indeed, concerns about the unethical use of AI are growing. Several OpenAI researchers even published a letter to the board of directors urging them to take action to prevent the “destruction of humanity.

For now, Sora is only available to red teamers, users who will work closely with the company that owns ChatGPT to look for any evidence of potentially harmful use. OpenAI will also provide access to Sora to various artists, designers, and filmmakers to get feedback on how to calibrate this model for the future.

According to OpenAI, “Sora is able to generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world.” And the company isn’t just saying that; the accompanying videos demonstrate that it will be increasingly difficult to distinguish a real video from an AI-generated one.

“The current model has weaknesses. It may struggle with accurately simulating the physics of a complex scene, and may not understand specific instances of cause and effect. For example, a person might take a bite out of a cookie, but afterward, the cookie may not have a bite mark,” OpenAI justifies itself with the fact that Sora is still in the testing phase.

In any case, after its announcement, Sora has generated a lot of buzz on social media, not only because of the fact that the line between fiction and reality is becoming increasingly blurred, but also because of the dangers and potential malicious uses that this type of tool can have.

According to Ralph Brooks, data scientist, and machine learning engineer, OpenAI’s Sora would use Epic Games’ Unreal Engine 5 to simulate the scenes users want to create. After that, the model would return one based on the calculations that Fortnite’s graphics engine has processed.