How Cortex actually builds a 3D model.

The walkable dollhouse view is the most visible part of a Matterport tour, but underneath it is a sequence of 360° panoramas, processed by Matterport's own AI engine, Cortex, into a dimensionally accurate space. Understanding how that actually works opens up useful capture strategy.

How Cortex actually builds the model

Every capture uploads as a sequence of panoramas. Cortex compares visual reference points across adjacent panoramas to work out how they relate to each other in space: that's photogrammetry, at its core. When the camera also has a depth sensor (LiDAR or infrared, like the Pro2), Cortex combines that depth data with the visual alignment to build the actual dimensionally accurate mesh, the part that makes a space measurable, not just walkable.

Why panorama-only capture is sometimes the right call

Not every space needs measurement data. A residential amenity space captured for Priya Living in Gurugram, part of an ongoing relationship that's since covered Hyderabad too, is a real example: shot entirely on 360° panoramic hardware, no depth sensor involved. For a space where visual navigation is the actual goal, that's a perfectly reasonable choice, not a compromise. Expansive exteriors, large open venues, or anywhere direct sunlight would interfere with depth sensing are all better served by panorama-only capture than by forcing full geometric scanning where it isn't needed.

Both Priya Living projects also involved working closely with the architects directly, useful context for capture planning that a scan alone wouldn't provide, and a good reminder that this work is rarely just showing up and pressing a button.

The real problem with water

Infrared depth sensors reliably fail over water. The water absorbs the infrared signal instead of reflecting it back, leaving a real gap in the depth data. A swimming pool without any correction shows up as a literal hole in the geometric mesh. Cortex handles this by analyzing the panoramic imagery surrounding the gap and generating a plausible mesh to fill it in. It's not perfect for complex geometry, but for a standard pool shape, it works well enough that most viewers would never notice anything was missing.

Where this data could go next

Spatial photos (a 360° panorama with depth data built in, viewable on headsets like Apple Vision Pro) are a real emerging format, though not something used in any workflow here; Vision Pro hasn't launched in India. Regardless, a space captured today with real depth data attached stays usable if that changes, since the underlying geometry doesn't need to be recaptured to support a newer output format later.