home   |   research   |   teaching   |   publications   |   biosketch   |   CV

Fast and Accurate Feature Matching
for Laparoscopic Images

Gustavo Puerto and Gian-Luca Mariottini


The ability to recover those tracked image features that were lost due to a prolonged occlusion, a sudden endoscopic-camera retraction, or a strong illumination change, is paramount in surgical vision. As any tracker will inevitably fail after the aforementioned events, it is of primary need to examine the laparoscopic video before the occlusion and after to identify image similarities (feature matching).
Our goal is to research novel feature-matching methods that, compared to feature tracking, do not make any restrictive assumption about the sequential nature of the two images or about the organ motion. Additionally, our goal is to reliably estimate the image mapping of features on deformable surfaces. These mappings will then be used to accurately find the position of the lost features in the image after occlusion. Restoring previously-tracked features is paramount in many SfM and Augmented-reality applications.

Our Contributions

Members of the ASTRA Robotics Lab are working on novel feature-matching algorithms that can find a larger number of image correspondences at an increased speed, and with a higher accuracy, and robustness to image clutter.
We recently designed a method, called Hierarchical Multi-Affine (HMA), which robustly matches clusters of image features on the observed objects' surface (see figure below).
Figure 1: From a set of initial (appearance-based) matches, HMA retrieves a set of refined (or final) matches, and estimates a set of local affine transformations.
HMA improves over existing methods by estimating a set of multiple and spatially distributed affine transformations. As illustrated in the above illustration, each of these transformations can map keypoints (or any other generic image-point) from the training image to corresponding features (or image point) on the query image. In estimating these transformations, HMA simultaneously computes the set of final matches (inliers) that support these local affine transformations (AT). Since these multiple ATs locally adapt more precisely to the (non-planar) object surface, HMA can i) retrieve a larger number of correct matches than a single AT, and ii) can estimate a set of highly-precise image transformation.
Figure 2: The video of the experiments.


  1. G. Puerto and G.L. Mariottini, "A comparative study of correspondence-search algorithms in MIS images", Medical Image Computing and Computer Assisted Interventions (MICCAI12), Nice, France, 2012
  2. G. Puerto and G.L. Mariottini, "Robust Feature-Matching Algorithm for Tracking Recovery from Occlusion in Augmented-Reality Systems (ARSs) for Minimally Invasive Surgery (MIS)", 27th Southern Biomedical Engineering Conference, April 29th- May 1st, Arlington, TX, 2011
  3. G. Puerto, M. Adibi, J. Cadeddu1 and G.L. Mariottini, "Adaptive Multi-Affine (AMA) Feature-Matching Algorithm and its Application to Minimally-Invasive Surgery Images", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2371 - 2376, Sept. 25-30, San Francisco, California, 2011

File translated from TEX by TTH, version 3.85.
On 7 Sep 2012, 13:12.