Surface Patch Reconstruction

 Our objective is to reconstruct a complex surface subject to occlusion through touch. The idea is to view a surface as a collection of patches. We would like to represent a surface as a graph where every node corresponds to a patch and every edge corresponds the boundary curve between the two patches defining the edge.

We are currently working on the reconstruction a single patch and have made very good progress. Our algorithm consists of three steps:

  1. Pick a reference point p on an object's surface and use a touch sensor to generate data points along three curves concurrent at p.

  2. Estimate the curvatures at p as well as the tangent plane. This provides enough information for us to solve for the principal curvatures and directions at p.

  3. In the local Darboux frame at p, we fit a polynomial (whose quadratic terms are determined by the two principal curvatures) over the three data curves to yield the patch description.
The main difficulty is that one-dimensional tactile data does not provide enough constraints for fitting. We first tried interpolation over the polar angle between the three data curves. The left figures below shows two different views of a mouse with the intended local patch circled in red ink. The right figures displays the reconstructed patch in corresponding views. But interpolation does not exact capture the desired "smooth" transition from one data curve to the other on the surface. It becomes a wild guess when the Gaussian curvature is negative at the reference point. Secondly, it does not work well with shapes that have non-circular projections onto the tangent plane at p.

The objective function for fitting needed to include a term that could measure the ``degree of folding'' of the surface fit so a minimization would avoid fitting results of this type. We made use of total absolute Gaussian curvature which integrates the absolute value of the Gaussian curvature over the surface patch. Incorporation of this term into the least-squares fitting has yielded patches with high accuracy. The figures below show the reconstruction of two (marked) areas, one convex and the other concave, on a broken plastic bottle over (white) data points. We matched them against large number of new (red) data points, and the average distances from these points to the reconstructed patches were between 0.06mm and 0.15mm.


The following figures show the patches reconstructed over the top regions of a mouse, a nut, a shell, and a pebble, respectively.




The objective of this work is twofold. From basic research point of view, we try to understand the minimum amount of tactile data sufficient for reconstructing a surface, or at least a patch. In practice, surfaces are often built from range data, which are subjected to camera occlusion and also do not have the required precision sometimes. A situation may arise where very fine details about certain parts of an object (e.g., a bone in robot-assisted surgery) are needed. Probing or tracking with a touch sensor mounted on a high precision robot can be a good solution.


This research is supported by an NSF CAREER Award 0133681.

Last updated on September 13, 2006.

Yan-Bin Jia
jia@cs.iastate.edu