tobycollins/IPPE

IPPE: A very fast and accurate way to compute the 3D pose of a planar surface with correspondences

Infinitesimal Plane-based Pose Estimation (IPPE): A very fast and accurate way to compute a camera's pose from a single image of a planar object using 4 or more point correspondences. This has uses in several applications, including augmented reality, 3D tracking and pose estimation with planar markers, and 3D scene understanding.
This is the author's Matlab implementation from the peer reviewed paper "Infinitesimal Plane-based Pose Estimation" by Toby Collins and Adrien Bartoli, published in the International Journal of Computer Vision, September 2014. A copy of the author's pre-print version can be found here: http://isit.u-clermont1.fr/~ab/Publications/Collins_Bartoli_IJCV14.pdf. A c++ implementation is available on the linked github page. Feel free to contact Toby ([e-mail]) if you have any questions about the paper and IPPE. We hope you find this code useful and if so please cite our paper in your work:

@article{ year={2014}, issn={0920-5691}, journal={International Journal of Computer Vision}, volume={109}, number={3}, doi={10.1007/s11263-014-0725-5}, title={Infinitesimal Plane-Based Pose Estimation}, url={http://dx.doi.org/10.1007/s11263-014-0725-5}, publisher={Springer US}, keywords={Plane; Pose; SfM; PnP; Homography}, author={Collins, Toby and Bartoli, Adrien}, pages={252-286}, language={English} }

IPPE has the significant benefit of being able to handle ambiguous cases. Specifically, it always returns two candidate pose solutions and their respective reprojection errors. These are sorted with the first pose having the lowest reprojection error. It is possible to reject the second pose if its reprojection error is significantly worse than the first. This can be done with a likelihood ratio test.

The second pose is needed if the problem is ambiguous, which means there are two valid pose solutions. The problem is ambiguous when the projection of the object is close to affine, which in practice happens if it is small or viewed from a large distance. In these cases there are generally two pose solutions that can correctly align the correspondences (up to noise), so it is impossible to select the right pose using the reprojection error. IPPE gives you both the solutions, rather than just a single solution (which in ambiguous cases would be wrong 50% of the time). This problem is suffered by OpenCV's solvePnP default iterative solver because it only returns one solution. Geometrically, the two poses roughly correspond to a flip of the object about a plane whose normal passes through the line-of-sight from the camera centre to the object's centre. For more details about the ambiguity, please refer to the IPPE paper.

We give two Matlab demo functions for running IPPE.

The first demo: IPPE_demo1, shows how to use IPPE to solve a perspective camera's pose with a plane and a single image using ransac-based feature matching. Two feature methods are supported, which are affine-sift (ASIFT) and SURF. Other can be easily added. This demo loops through a set of 5 images and computes the camera's pose relative to the plane for each image.

The second demo: IPPE_demo2, shows and example of using IPPE to solve a camera's pose with a plane and simulated point correspondences.

IPPE_demo1 requires the Affine SIFT (ASIFT) library for affine-invariant SIFT matching: http://www.ipol.im/pub/art/2011/my-asift/ This is easy to compile with cmake, and you should put the executable demo_ASIFT in the demo_ASIFT_src folder.

Optionally, you may need Bouguet's calibration toolbox: http://www.vision.caltech.edu/bouguetj/calib_doc/ This is used for performing lens undistortion on an image.