Representation

Turning scattered images and videos into a computable, updatable world state for reasoning.

Seeing is not yet understanding

An image is not a world. It hides occluded structure, breaks temporal consistency, and entangles objects with lighting, viewpoint, and motion. We seek representations that retain what matters while remaining continuously updatable.

images & videos→world representation→reasoning & rendering

Reading 3D understanding from visual features

Vision foundation models excel at 2D tasks, but depth or matching scores alone cannot tell us whether they understand 3D. Feat2GS reads foundation-model features into 3D Gaussian attributes and separates geometry from texture, making novel-view synthesis both a reconstruction tool and a probe of spatial understanding. LoftUp brings high-resolution images, coordinates, and low-resolution features together through coordinate-based cross-attention, returning semantic evidence to sharp pixel boundaries.

Feat2GS — probing geometry and texture awareness through novel-view synthesis.
LoftUp — high-resolution visual features.

Disentangling the visible world

A representation that renders well is not necessarily easy to reason about or edit. Ref-GS separates direction, roughness, geometry, and lighting to reduce the ambiguity between surface normal and viewing direction. NeST places texture in a neural shell around the surface, freeing Gaussians to focus on geometry and sampling. It preserves detail with fewer primitives and supports direct textured-mesh extraction. A useful representation should not only look right; it should place distinct sources of variation into variables that can be analyzed and controlled independently.

NeST — fine texture detail with a compact, disentangled representation.

From one-off reconstruction to online world memory

The real world does not appear all at once. Easi3R separates camera and object motion from the attention of a pretrained model without additional training; Human3R jointly recovers people, scene, and camera trajectories in a shared world frame. For long video streams, TTT3R updates memory according to confidence in state-observation alignment, while R³ uses confidence-weighted relative constraints for offline and bounded-memory streaming reconstruction. GlobalSplat aggregates multi-view evidence into global scene tokens before decoding explicit geometry, bridging compact latent state and usable 3D structure.

Easi3R — disentangling camera and object motion without training.
Human3R — people and scene reconstructed online in one world frame.
TTT3R — maintaining reconstruction quality beyond the training context length. R³ extends this line with confidence-weighted relative constraints.

Representation does not end with a static 3D file. It is a world state that can be read from vision, disentangled, updated over time, and scaled within finite compute and memory.

Feature → 3D

Feat2GS · LoftUp

Disentangled appearance

Ref-GS · NeST

Dynamic & human-scene reconstruction

Easi3R · Human3R

Long-context world memory

TTT3R · R³ · GlobalSplat