Residual Primitive Fitting of 3D Shapes with SuperFrusta

Brown University
Adobe Research

T.L.D.R.: We introduce a framework for converting 3D shapes into compact and editable assemblies of analytic primitives. Our approach combines SuperFrustum, an expressive, editable, and optimizable primitive, with ResFit, an iterative fitting algorithm. Together, they achieve state-of-the-art reconstruction quality while using significantly fewer primitives than prior work.

NEWS

  • [December 2025] Code coming soon!
  • [December 2025] Paper and project page released.

Qualitative Results

Primitive Assembly Shaders

We infer parsimonious assemblies of SuperFrusta that provide an SDF function that is sufficiently accurate as well as computationally cheap that it can be visualized with real-time sphere tracing. Click on a mesh on the left side to load a shader containing the inferred SuperFrusta assembly on the right. The loaded WebGL shader is sphere tracing the geometry in real-time.

Note: We developed SySL, a library for compiling symbolic scene expressions into shader code (and the surrounding glue). It will be released soon!

Mesh Render

Select a shape below

SuperFrusta Assembly (GLSL Shader)

Select a Shape

Example 1
Example 2
Example 3
Example 4
Example 5
Example 6
Example 7
Example 8
Example 9
Example 10
Example 11
Example 12
Example 13
Example 14
Example 15
Example 16
Example 17
Example 18
Example 19
Example 20

Controls

  • Camera Rotate: Click + Drag
  • Camera Pan: Shift + Drag

Additional Qualitative Results

Our primitive assemblies capture a wide range of shapes, including hollow forms (wine glass), curved & toroidal parts (bikes, bowls), intricate geometry (robots, grapes), and smooth organic shapes (horse, bear). The assemblies are simultaneously accurate, compact, and semantically interpretable.

Our method reconstructs target shapes with high geometric fidelity and produces interpretable assemblies using compact, minimally-overlapping primitives. In contrast, prior methods often lose fine structure and generate assemblies with substantial primitive overlap.

Interactively Editing Textured Primitive Assemblies

We can convert textured meshes into textured primitive assemblies, which makes many edits significantly easier to perform. More importantly, we can render and interact with these assemblies directly in a simple WebGL shader—highlighting the efficiency and deployability of our representation. The compact analytic form of SuperFrusta enables real-time sphere tracing with interactive editing capabilities, all running efficiently in the browser. More details here.

Controls

  • Camera Rotate: Click + Drag
  • Camera Pan: Shift + Drag
  • Parametric Editing: Click a primitive to select it. A left-side panel exposes its parameters for real-time editing.
  • Translate: G (constrain with X, Y, or Z)
  • Scale: S
  • Cancel Operation: Esc

Abstract

We introduce a framework for converting 3D shapes into compact and editable assemblies of analytic primitives, directly addressing the persistent trade-off between reconstruction fidelity and parsimony. Our approach combines two key contributions: a novel primitive, termed SuperFrustum, and an iterative fitting algorithm, Residual Primitive Fitting (ResFit). SuperFrustum is an analytical primitive that is simultaneously (1) expressive, being able to model various common solids such as cylinders, spheres, cones and their tapered and bent forms, (2) editable, being compactly parameterized with only 8 parameters, and (3) optimizable, with a signed distance field differentiable w.r.t. its parameters almost everywhere. ResFit is an unsupervised procedure that interleaves global shape analysis with local primitive optimization, iteratively fitting primitives to the unexplained residual of a shape to discover parsimonious yet accurate decompositions for each input shape. On diverse 3D benchmarks, our method achieves state-of-the-art results, improving IoU by over 9 points while using nearly half as many primitives as prior work. The resulting assemblies bridge the gap between dense 3D data and human-controllable design, producing high-fidelity and editable shape programs.

SuperFrustum: An Expressive, Editable & Optimizable Primitive

An ideal primitive for inverse graphics must be expressive enough for diverse forms, editable via intuitive controls, and robustly optimizable.

Existing primitive families often fall short on one or more of these criteria. We introduce SuperFrustum, a unified analytic primitive designed to meet all three desiderata.

Primitive Expressive Editable Optimizable
Superquadrics
Algebraic Surfaces
Multi-type
SuperFrusta

Expressive

SuperFrustum spans a wide range of common solids—cuboids, cylinders, cones, spheres, and toroidal variants—and can produce more complex shapes such as bent, hollow, or smoothly capped forms.

Side Note: SuperFrustum can be parameterized to represent many of the "Geons" - 3D forms corresponding to the simple parts of an object in Biederman's recognition-by-components theory. While algerbaic surfaces can represent a wide range of shape they fail to model canonical solids and geons.

Editable

SuperFrustum is compactly parameterized with only 8 intuitive parameters:

  • Scale (s): Anisotropic scaling along each axis
  • Rounding (r): Profile rounding
  • Dilation (d): Uniform dilation
  • Taper (t): Tapering along the axis
  • Bulge (b): Axial bulging
  • Onion (o): Shell/hollowing thickness

This compact parameterization makes the primitives easy to manipulate and edit by users, bridging the gap between dense 3D data and human-controllable design.

Optimizable

SuperFrustum admits a signed distance field (SDF) that is C⁰-continuous and fully differentiable (almost everywhere) with respect to all parameters.

This enables robust inverse modeling and gradient-based optimization. As a result, SuperFrustum can be reliably fitted to target geometry through standard gradient descent without the need for non-differentiable heuristics.

SuperFrustum optimization

The design of SuperFrustum builds on analytic functions uncovered by the Shadertoy and Demoscene communities in their pursuit of highly expressive analytic forms with minimal description length [sdUberprim, sdSuperprim, ChamferBox SP, Inigo Quilez's SDF Functions]. We find that, when carefully adapted, these formulations are exceptionally well-suited for inverse modeling.

ResFit: Residual Primitive Fitting

To achieve parsimonious assemblies, an expressive primitive must be paired with an equally effective inference algorithm. We propose Residual Primitive Fitting (ResFit), an unsupervised procedure that tightly interleaves global shape analysis with local primitive optimization.

Interleaving Shape Decomposition and Primitive Optimization

Instead of optimizing a large set of primitives jointly from scratch, ResFit first analyzes the input geometry to propose initial structures based on global cues. These primitives are then refined via gradient descent to conform to the local geometry.

The resulting assembly is subtracted from the target shape, and the process repeats on the unexplained residual. By alternating between proposing global structure and optimizing local parameters, ResFit allows these two signals to mutually inform each other, producing assemblies that are both compact and high-fidelity.

Morphological Shape Decomposition

ResFit initializes primitives from volumetric regions produced by Morphological Shape Decomposition (MSD). MSD is an iterative "peel the thickest part first" technique: at each step, it finds the largest connected region of roughly uniform thickness, extracts it, removes it from the shape, and repeats on the residual. This yields regions that form suitable initialization seeds for SuperFrusta—capturing non-convex structures such as bicycle tires, curved tails, and bowl rims that convex decomposition methods would over-partition.

ResFit in Action

The iterative analysis-optimization loop enables self-correction as new parts are added. To prevent over-parameterization, we seed few primitives per round and employ parsimony-aware optimization: a soft regularizer penalizes redundancy during fitting, while hard pruning removes parts that degrade the objective. To address under-parameterization, primitives are optimized based on their local support, and the full assembly is re-optimized in each round. This allows the system to converge toward a compact and coherent structure.

Resetting the Quality-Parsimony Pareto Frontier

Our approach redefines the reconstruction–parsimony Pareto Frontier. Compared to state-of-the-art methods like Marching Primitives (MPS) and Primitive Anything (PA), we achieve markedly lower reconstruction error using significantly fewer primitives.

On two different 3D benchmarks, our method improves IoU by 6-9 points while using roughly half as many primitives and reducing volumetric overlap by over . The inferred primitives also demonstrate higher semantic coherence, with more meaningful distinctions between parts.

Comparative Qualitative Results

Our method produces assemblies with higher geometric fidelity and significantly less primitive overlap compared to prior methods. While baselines often generate fragmented or heavily overlapping primitives, our approach yields compact, semantically meaningful decompositions.

Applications

Our representation enables several downstream applications that combine visual quality, editability, and analytic structure.

Editable & Deployable Assets

Our primitives are simultaneously compact, editable, and capable of high-fidelity reconstruction, allowing them to serve directly as deployable 3D assets. By assigning per-primitive spherical textures, the resulting textured assemblies can be directly deployed in real-time sphere-traced scenes while remaining fully editable.

Inferring CSG Programs

Although our framework is designed for smooth, soft-union assemblies, it can also infer discrete Constructive Solid Geometry (CSG) programs composed of canonical solids (cuboids, cylinders, cones, spheres). Our approach achieves nearly the same CSG reconstruction accuracy as dedicated methods while using fewer primitives.

Image to Primitive Assembly

Our method can seamlessly be combined with 3D generative models to achieve image-to-primitive assembly. By using state-of-the-art text/image-to-3D models followed by ResFit, we can convert any image into an editable primitive assembly.

Semantic Segmentation Enrichment

ResFit can help enhance coarse semantic labels. By intersecting coarse semantic annotations with our primitive assemblies, we can subdivide large parts into functionally meaningful subcomponents without drifting outside their semantic boundaries.

Failure Cases

While ResFit performs reliably across a broad range of shapes, we observe several characteristic failure modes:

  • (a) Non-uniform hollow structures: Shell-like shapes (helmets, hats, shoes) with thin walls enclosing large voids require many primitives without explicit negative-space operations.
  • (b) Multi-arc or winding structures: Each primitive contains only a single bending arc, so long spirals or multi-turn curves get fragmented into many short segments.
  • (c) Overlapping assemblies: Categories like chairs may require overlapping primitives when local curvature cannot be matched by the current primitive's shape space.
  • (d) Flat curved surfaces: When parts are extremely thin (e.g., dragon wings), gradient magnitudes become very small, leading to poor fits.
  • (e) Irregular thickness in hollow containers: Inconsistent wall thickness causes MSD to decompose shapes into thin slats instead of coherent onioned cylinders.
  • (f) Semantic spillover: Achieving high geometric fidelity sometimes requires primitives whose spatial support crosses semantic boundaries (e.g., cheek extending into eye region).

BibTeX

@misc{ganeshan2025superfit,
  title={Residual Primitive Fitting of 3D Shapes with SuperFrusta}, 
  author={Aditya Ganeshan and Matheus Gadelha and Thibault Groueix and Zhiqin Chen and Siddhartha Chaudhuri and Vladimir Kim and Wang Yifan and Daniel Ritchie},
  year={2025},
  eprint={2512.00000},
  archivePrefix={arXiv},
  primaryClass={cs.GR},
  url={https://arxiv.org/abs/2512.00000}, 
}
Zoomed image
100%
Scroll to zoom • Drag to pan • Click background or press Esc to close