San Francisco, NFAPost: Nvidia has unveiled a new feature called ‘Neuralangelo’ that works by analysing a 2D video clip of an object that has been filmed from various angles and then do a 3D reconstruction of it.
Neuralangelo selects several frames of reference to analyse what the object would look like and its results can be used for 3D asset generation.
“The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world. This tool will eventually enable developers to import detailed objects — whether small statues or massive buildings — into virtual environments for video games or industrial digital twins,” said Ming-Yu Liu, Senior Director of Research and Co-Author of the paper on the AI model.
The company says that the AI model can generate “3D structures with intricate details and structures”, and these models can then be imported into editing and design applications for further tweaking.
Neuralangelo works by analysing a 2D video clip of an object that has been filmed from various angles. It selects several frames of reference from the video to get an idea of what the object would look like from various angles. This allows it to get a sense of depth, size, and shape of the object.
Once analysed, the AI model then creates a rough 3D representation of the object, and then optimises it by adding details and textures. The resulting 3D model can then be used in, virtual reality applications, digital twins or robotics development.