System and method for position determination

• US 20050190972A1
• Filed: 02/11/2005
• Published: 09/01/2005
• Est. Priority Date: 02/11/2004
• Status: Abandoned Application

• Current Assignee: British Broadcasting Corporation
• Original Assignee: British Broadcasting Corporation
• Inventors: Chandaria, Jigna, Thomas, Graham Alexander, Fraser, Hannah Margaret, Grau, Oliver, Brightwell, Peter John
• Application Number:
  US11/055,703
• Publication Number:
  US 20050190972A1
• Time in Patent Office: Days
• US Class Current: 382/218
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: H04N 5/2224 related to virtual studio applications

Abstract

A method for determining the position and orientation of a camera, which may not rely on the use of special markers. A set of reference images may be stored, together with camera pose and feature information for each image. A first estimate of camera position is determined by comparing the current camera image with the set of reference images. A refined estimate can be obtained using features from the current image matched in a subset of similar reference images, and in particular, the 3D positions of those features. A consistent 3D model of all stored feature information need not be provided.

Claims

1. A method of determining an estimate of the pose of a camera, the method comprising:

storing a plurality of reference images corresponding to a respective plurality of camera poses, the images including a plurality of reference features;

storing a measure of three dimensional position information for the plurality of reference features;

obtaining a current camera image from the camera;

selecting one of the plurality of reference images as a current reference image based on the current camera image; and

providing an initial estimate of the pose of the camera based on the camera pose corresponding to the current reference image.

2. A method according to claim 1, wherein the measure of three-dimensional position information is stored as a depth map for each reference image.

3. A method according to claim 1, wherein the estimate of the pose has six degrees of freedom.

4. A method according to claim 3, wherein the estimate of pose comprises a three-dimensional estimate of position and an estimate of orientation including pan and tilt.

5. A method according to claim 1, wherein the current reference image is selected by comparing the current image to at least some of the plurality of reference images.

6. A method according to claim 5 wherein the current image is compared to only a subset of the plurality of reference images in at least one comparison step.

7. A method according to claim 6 wherein the subset is selected based on at least one of:

a previous estimate of pose or position;

a further input of a measure of one of the group consisting of;

pose or position or motion; and

the results of an initial comparison step.

8. The method according to claim 7, wherein the further input of a measure is from a position or motion sensor.

9. A method according to claim 8, wherein the comparison includes comparing derivative measures of image content.

10. A method according to claim 1, comprising refining the initial estimate of pose based on the position of reference features.

11. A method according to claim 10 wherein a confidence measure is stored for each of the features.

12. A method according to claim 1, wherein first portions of the reference images are identified as being suited to providing reference features.

13. A method according to claim 1, wherein second portions of the reference images are identified as being unsuited to providing reference features.

14. A method according to claim 1, wherein each reference images comprises a plurality of pixels in a plurality of regions and includes a measure of depth and a measure of suitability as a reference feature for each region.

15. A method according to claim 1, including processing a plurality of images obtained to provide said reference images and to store the measures of three-dimensional position prior to using the reference images to determine pose for a current image.

16. A method according to claim 1, further comprising updating or adding to the store of reference images and/or the reference features based on the current image.

17. A method according to claim 1, of providing a real-time output of current camera pose wherein a current position estimate is updated when a new camera image replaces the current camera image.

18. A method according to claim 17, wherein a current position estimate is updated at least 20 times per second.

19. A method according to claim 18, wherein a current position estimate is updated for every camera frame.

20. A method according to claim 17, wherein an initial estimate of camera pose is obtained in an initialisation process and wherein movement is tracked from frame to frame, wherein movement tracking is performed using fewer comparison operations than the initialisation process.

21. A method according to claim 20, wherein a validation process is performed in which more comparisons are performed than in the movement tracking process and wherein the results of the validation process are compared to the results of the tracking process.

22. A method according to claim 1, wherein at least one reference feature includes an edge.

23. A method according to claim 1, wherein a measure of gradient is associated with at least some reference features.

24. A method according to claim 1, wherein a measure of at least one further camera parameter is obtained.

25. A method according to claim 24, wherein a zoom or a measure of focus is obtained.

26. A method according to claim 1, wherein the reference images comprise a plurality of images at different resolutions.

27. A method according to claim 1, wherein images are obtained from a plurality of cameras coupled together at known relative orientations.

28. A method according to claim 27, wherein images are obtained from the plurality of cameras are three cameras coupled together at known mutually orthogonal fixed orientations.

29. A method according to claim 27, wherein one camera is designated as a studio image camera and the other camera(s) are provided to enhance pose estimation for the studio camera.

30. A method according to claim 1, wherein the camera is a studio camera, the method further comprising processing the camera image to derive an output image.

31. A method according to claim 30, further comprising processing the camera image to derive a broadcast quality image.

32. A method of determining an estimate of the pose of a camera comprising:

storing a plurality of reference images containing reference features and associated depth information, the images being associated with reference pose information;

obtaining a current camera image; and

deriving an estimate of the pose of the camera by comparing the current camera image to the reference images, wherein the estimated pose is based on the reference pose information and reference features for a plurality of reference images and associated depth information without resolving the reference images into a consistent three-dimensional model.

33. A method of compiling a database of reference images for use in determining camera pose, the method comprising:

storing a plurality of images from mutually different poses;

storing camera pose information for each image;

identifying reference features within the images; and

storing identifiers of the reference features and a measure of three dimensional position of the features.

34. The method of claim 33 comprising storing a depth map.

35. A database of images and associated positional information compiled by the method of claim 34.