NVIDIA releases Cosmos 3 Omnimodal world modelson HF

Posted by RobotRobotWhatDoUSee@reddit | LocalLLaMA | View on Reddit | 9 comments

https://huggingface.co/nvidia/Cosmos3-Super-Text2Image

Nano: 16B

Super: 64B

Cosmos3 is a collection of Omnimodal world models capable of generating dynamic, high-quality video, image, audio, and action commands from combinations of text, image, video, and action trajectory inputs. It serves as a foundational building block for a broad range of Physical AI applications and research spanning world understanding, world generation, simulation, and embodied policy learning.

Haven't seen much here yet.

Some twitter discussion: https://x.com/victormustar/status/2061354267546427595

[-]

Dany0@reddit

I'm not a Yann LeCun fanboy but I have his words stuck in my head, how can a model do anything in the real world without being able to predict the future?

This model is a great test case for that

Yeah, I'm keeping an eye on his JEPA ideas. Also, I like the idea of an internal feedback loop, especially after reading "I'm a strange loop" by Douglas Hofstadter. We kinda simulate it through chain-of-thought, but latent-space reasoning might be much better.

Borkato@reddit

Is it good

FusionCow@reddit

it's okay at best, but what's impressive is that the model can literally do any modality in, and modality out. but the 32b is actually a 65b, and a 65b model is hefty. the 8b model (which is really a 16b), isn't worth it

Eyelbee@reddit

If it's better than nano banana pro I wouldn't call it okay, there haven't been an open model in that tier of image models

it isn't

Can it run on 48gb vram

xlltt@reddit

total_size 129187722208 , BF16