DreamFusion is a website that generates relightable 3D objects from text captions using a pretrained text-to-image diffusion model. The generated objects can be viewed from any angle, relit by arbitrary illumination, or composited into any 3D environment.
DreamFusion offers a gallery of hundreds of generated assets and allows users to generate 3D objects themselves. The resulting 3D models are coherent, with high-quality normals, surface geometry, and depth, and can be exported to meshes for easy integration into 3D renderers or modeling software.
DreamFusionroach requires no 3D training data and no modifications to the image diffusion model, demonstrating the effectiveness of pretrained image diffusion models as priors.
DreamFusion Features
- Text-to-3D Synthesis: Generates relightable 3D objects with high-fidelity appearance, depth, and normals from a given caption.
- Pretrained Text-to-Image Diffusion Prior: Uses a pretrained 2D text-to-image diffusion model as a prior for optimization of a parametric image generator.
- No 3D Training Data Required: Circumvents the need for large-scale datasets of labeled 3D assets and efficient architectures for denoising 3D data.
- Composable Objects: Objects can be composited into any 3D environment.
- Mesh Exports: Generated NeRF models can be exported to meshes using the marching cubes algorithm for easy integration into 3D renderers or modeling software.
- Score Distillation Sampling: Uses Score Distillation Sampling (SDS) to generate samples from a diffusion model by optimizing a loss function. SDS allows for optimization in an arbitrary parameter space, such as a 3D space, as long as mapping back to images is differentiable.
- High-Quality Results: Produces coherent 3D models with high-quality normals, surface geometry, and depth, and are relightable with a Lambertian shading model.
Share:
