CAMERA POSE ESTIMATION APPARATUS AND METHOD FOR AUGMENTED REALITY IMAGING
First Claim
1. An apparatus for providing an estimate for a 3D camera pose relative to a scene from 2D image data of a 2D image frame provided by said camera, the apparatus comprising(a) a candidate 2D key points detector having inputs for receiving camera data from a camera data source and the 2D image data, and an output for providing candidate 2D key points,(b) a detected 3D observations detector having inputs for receiving pre-recorded scene data and the candidate 2D key points, and an output for providing detected 3D observations,(c) a detected 3D camera pose estimator having inputs for receiving the camera data, the detected 3D observations and the candidate 2D key points, and an output for providing a detected 3D camera pose estimate,(d) a 2D image data and candidate 2D key points storage having inputs for receiving the detected 2D candidate key points and the 2D image data, and a control input for receiving a control signal related to a 3D camera pose estimate output, so as to store the 2D image data and the candidate 2D key points related to the 3D camera pose estimate output and to output a previous 2D image data and candidate 2D key points related to a previous 3D camera pose estimate output,(e) a previous 3D camera pose estimate storage having an input for receiving the 3D camera pose estimate output, and adapted to store the 3D camera pose estimate output and to output the said previous 3D camera pose estimate,(f) a tracked 3D observations detector having inputs for receiving the 2D image data, the candidate 2D key points, the camera data, the previous 2D image data and candidate 2D key points, the previous 3D camera pose estimate and 3D scene model data from a 3D scene model data source, and an output for providing tracked 3D observations,(g) a pose estimate selector having inputs for receiving the detected camera pose estimate and the said previous 3D camera pose estimate, and adapted to output as a selected 3D camera pose estimate one of the said detected 3D camera pose estimate and the said previous 3D camera pose estimate, and(h) a 3D camera pose estimator having inputs for receiving the camera data, the detected 3D observations, the tracked 3D observations and the selected 3D camera pose estimate, and an output for providing the 3D camera pose estimate output.
2 Assignments
0 Petitions
Accused Products
Abstract
An apparatus for providing an estimate for a 3D camera pose relative to a scene from 2D image data of 2D image frame provided by said camera. A candidate 2D key points detector determines candidate 2D key points from the 2D image frame. A detected 3D observations detector determines detected 3D observations from pre-recorded scene data and the candidate 2D key points. A detected 3D camera pose estimator determines a detected 3D camera pose estimate from the camera data, the detected 3D observations and the candidate 2D key points. A first storage stores the detected 2D candidate key points and the 2D image data, and outputs in response to a 3D camera pose estimate output previous 2D image data and candidate 2D key points related to a previous 3D camera pose estimate output. A second storage stores and outputs a previous 3D camera pose estimate. A tracked 3D observations detector determines tracked 3D observations from the 2D image data, the candidate 2D key points, the camera data, the previous 2D image data and candidate 2D key points, the previous 3D camera pose estimate and 3D scene model data. A pose estimate selector outputs a selected one of the detected camera pose estimate and the previous 3D camera pose estimate. A 3D camera pose estimator computes and outputs the 3D camera pose estimate from the camera data, the detected 3D observations, the tracked 3D observations and the selected 3D camera pose estimate.
88 Citations
13 Claims
-
1. An apparatus for providing an estimate for a 3D camera pose relative to a scene from 2D image data of a 2D image frame provided by said camera, the apparatus comprising
(a) a candidate 2D key points detector having inputs for receiving camera data from a camera data source and the 2D image data, and an output for providing candidate 2D key points, (b) a detected 3D observations detector having inputs for receiving pre-recorded scene data and the candidate 2D key points, and an output for providing detected 3D observations, (c) a detected 3D camera pose estimator having inputs for receiving the camera data, the detected 3D observations and the candidate 2D key points, and an output for providing a detected 3D camera pose estimate, (d) a 2D image data and candidate 2D key points storage having inputs for receiving the detected 2D candidate key points and the 2D image data, and a control input for receiving a control signal related to a 3D camera pose estimate output, so as to store the 2D image data and the candidate 2D key points related to the 3D camera pose estimate output and to output a previous 2D image data and candidate 2D key points related to a previous 3D camera pose estimate output, (e) a previous 3D camera pose estimate storage having an input for receiving the 3D camera pose estimate output, and adapted to store the 3D camera pose estimate output and to output the said previous 3D camera pose estimate, (f) a tracked 3D observations detector having inputs for receiving the 2D image data, the candidate 2D key points, the camera data, the previous 2D image data and candidate 2D key points, the previous 3D camera pose estimate and 3D scene model data from a 3D scene model data source, and an output for providing tracked 3D observations, (g) a pose estimate selector having inputs for receiving the detected camera pose estimate and the said previous 3D camera pose estimate, and adapted to output as a selected 3D camera pose estimate one of the said detected 3D camera pose estimate and the said previous 3D camera pose estimate, and (h) a 3D camera pose estimator having inputs for receiving the camera data, the detected 3D observations, the tracked 3D observations and the selected 3D camera pose estimate, and an output for providing the 3D camera pose estimate output.
Specification