BlockFusion

What is BlockFusion?

BlockFusion is an advanced model for generating 3D scenes that can seamlessly integrate new blocks. It uses a dataset of 3D blocks randomly cropped from complete 3D scene meshes. Each training block is transformed into a hybrid neural field consisting of tetrahedra with geometric features followed by a multi-layer perceptron (MLP) for decoding signed distance values. Variational autoencoders compress the tetrahedra into a latent space which undergoes denoising diffusion.

During the generation process, new blocks are added to extend the scene. These blocks overlap with the current scene, and the existing latent tetrahedra are extrapolated to fill these new blocks. This extrapolation uses feature samples from overlapping tetrahedra to guide the generation process, ensuring that the transition is semantically and geometrically meaningful.

BlockFusion also includes a 2D layout regulation mechanism to control the placement and arrangement of scene elements. The model has been shown to generate diverse, geometrically consistent, and high-quality large 3D indoor and outdoor scenes.

Who Can Use BlockFusion?

BlockFusion can be useful for developers, designers, and architects working on games, virtual reality applications, or architectural projects.

Example Scenarios

Game Development: Generate varied game environments.

Virtual Reality Applications: Create realistic virtual spaces.

Architectural Design: Visualize building interiors and exteriors.

Key Features

Generates detailed 3D scenes

Integrates new blocks seamlessly

Extrapolates existing latent tetrahedra to fill new blocks

Controls placement and arrangement of scene elements