The physical and digital worlds are becoming increasingly connected. From digital twins and immersive environments to virtual prototyping and simulation, there is a growing need to bring real-world objects into digital space. But converting a physical object into a useful 3D representation is not as simple as pointing a camera at it and generating a model.
There are several technologies that can capture different aspects of an object, and each has its own strengths. Some are focused on geometry, some on appearance, while newer approaches are beginning to use AI to generate complete 3D assets from images.
Different Ways of Capturing a Physical Object
When it comes to capturing the geometric information of a physical object, point clouds remain one of the most useful representations. A point cloud essentially records millions of points in three-dimensional space, providing information about the shape and spatial structure of an object. These can be generated using LiDAR, laser scanners, depth cameras, and other sensing technologies. Point clouds are particularly valuable when accuracy matters. In applications such as inspection, reverse engineering and aerospace manufacturing, understanding the actual geometry of a component can be more important than simply making it look realistic. However, a point cloud primarily tells us where the surfaces are. It does not necessarily capture the complete visual appearance of an object.
This is where photogrammetry becomes interesting. Instead of directly measuring the object, photogrammetry uses multiple photographs taken from different viewpoints to reconstruct its three-dimensional structure. By identifying common features across hundreds or even thousands of images, software can estimate camera positions and triangulate the geometry of the object. The resulting data can then be converted into point clouds, meshes and textured models.
Photogrammetry has made 3D capture considerably more accessible because it can work with relatively ordinary cameras. At the same time, reflective, transparent or textureless surfaces can still present challenges, making the technique highly dependent on image quality and capture conditions.
From Geometry to Appearance
Another major development in this field is Gaussian Splatting. Unlike traditional mesh-based approaches, Gaussian Splatting represents a scene using large numbers of three-dimensional Gaussians, or ellipsoids, that contain properties such as position, scale, orientation, opacity and appearance.
This creates a different way of thinking about 3D capture. A point cloud is primarily concerned with the geometry and spatial location of what has been captured. Gaussian Splatting, on the other hand, is particularly effective at reproducing how an object or environment looks from different viewpoints.
This makes it extremely useful for visualisation and immersive applications. It does not necessarily replace a CAD model or a geometrically accurate mesh, especially when the objective is engineering modification or manufacturing. Instead, it provides another representation that can preserve visual information remarkably well.
The distinction is important because a “3D model” does not always mean the same thing. A point cloud, mesh, CAD model and Gaussian Splat can all represent the same physical object, but they contain and reproduce different types of information.
The Rise of Generative 3D
The field is now moving beyond simply capturing what exists. Generative AI is beginning to create 3D representations directly from images.
Microsoft’s TRELLIS.2 is an interesting example of this shift. The system uses an O-Voxel representation, a sparse voxel structure designed to capture both geometry and appearance, including physically based material properties. It represents a different approach from conventional scanning and reconstruction because the model is learning how 3D objects can be represented and generating the result from an input image.
This opens up an interesting question. Are we always trying to create an exact digital replica of the physical object, or are we sometimes trying to create a useful digital representation of it?
The distinction becomes particularly important in engineering. A generative model may produce an object that looks convincing but does not necessarily represent the exact dimensions or internal geometry of the physical component. For visualisation, this may be acceptable. For engineering and manufacturing, accuracy becomes critical.
Why a Hybrid Approach Makes Sense
Rather than looking at these technologies as competing solutions, there is an opportunity to combine them. A physical object could be captured through photographs, depth sensors and scanning systems, producing multiple forms of digital information.
Point clouds could provide accurate geometric data, while photogrammetry could help reconstruct the object from images. Gaussian Splatting could preserve its visual appearance, while a conventional mesh or CAD model could support engineering workflows.
This is also where OpenUSD becomes increasingly relevant. The Alliance for OpenUSD has introduced a native schema for 3D Gaussian Splats in OpenUSD 26.03, reflecting the broader movement toward supporting different types of 3D representations within a common ecosystem.
For SyncXR, this is particularly interesting. If we want to bring physical objects into an immersive engineering environment, the answer may not be to choose one universal representation. A hybrid pipeline could allow geometry, appearance, materials and engineering data to coexist, with each representation being used for what it does best.
The Future of Physical-to-Digital Capture
The way we capture reality is changing rapidly. Point clouds continue to provide valuable geometric information, photogrammetry allows us to reconstruct objects using images, and Gaussian Splatting is creating new possibilities for realistic visualisation. At the same time, generative models are beginning to challenge the traditional idea of how a 3D asset should be created.
The future may not be about one technology replacing another. Instead, it could be about combining multiple representations to create richer and more useful digital twins.
As the boundaries between physical and digital environments continue to blur, the real challenge may no longer be simply converting an object into 3D. It will be about deciding what information we need to capture, how we represent it, and how we make those representations work together.
Leave A Comment