Luma AI Launches AI-Powered Text-to-Video Generation Platform Dream Machine

Luma AI launched its artificial intelligence (AI) text-to-video generation model, Dream Machine, globally on Wednesday. The platform can generate videos up to five seconds long from simple or descriptive text prompts. The AI ​​model can generate videos in various styles such as cinematic, animation, realistic and more. The AI ​​firm claims Dream Machine was trained entirely on video and is capable of generating “physically accurate, coherent and event-driven photos.” The platform is currently free to access and use, but is likely to have a daily generation limit.

Luma AI debuts Dream Machine

According to the website, the Dream Machine AI model is based on a transformer model and was trained directly on videos. Typically, large language models (LLMs) are trained on text and images and then transferred to videos, as they require a deeper understanding of space and motion. “Dream Machine is our first step in building a universal imagination engine,” the company added.

Dream Machine joins video generation platforms like Runway AI and Pika 1.0 that are also available in the public domain and offer video generation between three and five seconds. Gadgets 360 tested the platform and found the platform's quick stickiness to be inferior. Struggle with multiple characters or when the message is too complex. However, compared to the other two, it is capable of generating higher quality cinematic videos.

The AI ​​platform takes 120 seconds to generate a video, which the company says will have 120 different frames. Dream Machine is also said to understand how people, animals and objects interact with the physical world and create videos with accurate physics and character consistency.

However, Luma AI also highlighted several limitations in the current mode, such as movement, text, morphing, and the infamous Janus issue, where instead of showing consistent 3D output, the AI ​​model shows multiple canonical views of an object in different directions.

Luma AI's Dream Machine creates a video about Stormtroopers
Photo credit: Luma AI/Dream Machine

Luma AI has not shared any technical details about the AI ​​model, so information about parameter sizes, benchmarks, architecture, and training methods is not known. The company also did not share any details about the acquisition of the training data. In particular, Gadgets 360 was able to generate several videos with copyrighted characters.

To test the platform, enthusiasts can go to the website and click on the “Try Now” button. Users will need to register on the platform before they can generate videos.