New AI That Generates 3-D Models From Text

The Shap-E AI system from OpenAI (creators of ChatGPT) and available for open-source download, can create 3D models from text. 

What Is It? 

According to the Cornell University researchers behind the system, Shap-E is “a conditional generative model for 3D assets” which, unlike other 3D generative models “generates the parameters of implicit functions that can be rendered as both textured meshes and neural radiance fields” rather than producing a single output representation. In short, Shap-E is a big step forward in simplifying the generation of 3D models that can generate “complex and diverse 3D assets” in seconds (when trained on a large dataset of paired 3D and text data). 

Trained In Two Stages 

OpenAI says that Shap-E works so well (and better than the previous Point-E models) because it is trained in two stages: first, the encoder that deterministically maps 3D assets into the parameters of an implicit function is trained and second, we train a conditional diffusion model on outputs of the encoder.  

Faster & Better 

When compared to Point-E, which is an explicit generative model over point clouds, the new Shap-E converges faster and reaches comparable or better sample quality despite modelling a higher-dimensional, multi-representation output space.  

Examples 

Examples of the text-conditional models created by Shap-E and the simple prompts used to create them can be seen at the foot of this page. For example, the first model was created simply by using the text prompt “a chair that looks like an avocado.” 

Where & How? 

Shap-E can be downloaded free here https://github.com/openai/shap-e/ and accessed on Microsoft Paint 3D.   

Challenge 

Even though Shap-E simplifies 3D model making, some tech commentators have reported that getting the new model installed and running can be complicated and take some time, and that a lot of system resources are required to use it. 

3D Printing 

There are also developments in the 3D printing world, e.g., with advanced models like the Chinese Artillery 3D Sidewinder. Today’s advanced, large 3D printers are moving more into the production technology stage from one-off manufacturing and prototyping, and additive manufacturing (constructing a three-dimensional object from a CAD model or a digital 3D model) could soon be fed by models created by AI models that use text or even speech to create items. 

What Does This Mean For Your Business? 

The Shap-E represents a big step forward in speeding up and simplifying the making of 3D models and making it more accessible to all kinds of businesses. Being able to quickly visualise a concept or idea and chop and change it with just words, and to create a model that can be used to make a 3D printed item is used in many industries and fields for making prototypes, components, end products, reconstructed fossils, or artefacts (and much more) could deliver huge benefits to many businesses. Although Shap-E, as shown by the example models, is still at an early (but still impressive) stage, linking things like AI text (or speech) model generators to 3D printers could dramatically speed up, cut costs, and simplify areas of manufacturing that would have proved much more challenging before.

For the moment, having a free 3D model generator for general usage that works on simple text inputs makes 3D modelling more accessible to any business and will help with creativity and conceptualising, and we have yet to see its true value and range. Reports indicate that it may be a little challenging to install but as it’s from OpenAI it comes with a good pedigree and once up and working, it may prove a useful tool for many businesses, as well as being fun!