cosmos
NVIDIA Cosmos is an open platform of world models, datasets, and tools that enables developers to build Physical AI for robots, autonomous vehicles, smart infrastructure, and more.
Explore NVIDIA Cosmos 3's Reasoner and Generator modes. Learn which mode suits vision-language understanding and media creation tasks for optimal results.
How to Use Cosmos 3 for Embodied Reasoning and Next-Action PredictionLeverage Cosmos 3 an omnimodal world model for embodied reasoning and next-action prediction. Use its causal transformer Reasoner surface to plan tasks from video and text inputs.
Running Supervised Fine-Tuning on Cosmos 3 Vision Generators: A Complete GuideLearn to run supervised fine-tuning on Cosmos 3 vision generators with NVIDIA's cookbook scripts. Automate dataset prep, multi-GPU training, and checkpoint export for your custom video data.
Implementing Video-to-Video Transformation with Prompt Guidance in Cosmos 3Learn how to implement video-to-video transformation with prompt guidance in Cosmos 3. This guide covers using textual prompts and spatial controls for powerful video generation.
Using Cosmos 3 for Physical Plausibility Analysis and Classification: A Complete GuideLearn how to use Cosmos 3 for physical plausibility analysis and classification. This guide covers the omni-model's two-step workflow for video synthesis and LLM-vision classification.
Troubleshooting CUDA Driver Compatibility with PyTorch Backends for Cosmos 3Fix CUDA driver compatibility errors in Cosmos 3 with PyTorch. Ensure your CUDA wheel tag matches your NVIDIA driver for successful GPU acceleration. Avoid torch.cuda.is_available() failing.
Implementing 2D Grounding and Bounding Box Localization with Cosmos 3Implement 2D grounding and bounding box localization with Cosmos 3. Our Reasoner surface converts image-text prompts into JSON bounding boxes using a Mixture-of-Transformers.
Understanding mRoPE Position Embedding for Multimodal Reasoning in Cosmos 3Explore mRoPE position embedding for multimodal reasoning in NVIDIA Cosmos 3. Learn how this 3-axis embedding unifies vision, audio, and robot actions in transformers.
How to Configure Video Resolution, Frame Rate, and Aspect Ratio in Cosmos 3Learn to configure video resolution, frame rate, and aspect ratio in NVIDIA Cosmos 3. Control these settings using environment variables or CLI arguments for optimal video generation.
Running Inverse Dynamics Predictions for Autonomous Vehicle Ego-Motion with Cosmos 3Learn to run inverse dynamics predictions for autonomous vehicle ego-motion with Cosmos 3. Estimate 9-DOF vehicle pose deltas from video using PyTorch or REST APIs.
How to Set Up the Guardrail Safety Checker for Cosmos 3 GeneratorLearn to set up the Guardrail safety checker for the NVIDIA Cosmos 3 Generator. Easily enable or disable this essential feature for secure AI model generation.
Configuring Action Policy Predictions for DROID Robot Embodiment in Cosmos 3Learn to configure DROID robot action policy predictions in Cosmos 3. Set up the environment, register the experiment, and run the training script for seamless embodiment.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →