Automatic detection of panoramic camera position and orientation table parameters

• US 20070058879A1
• Filed: 09/15/2005
• Published: 03/15/2007
• Est. Priority Date: 09/15/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Zhengyou, Liu, Zicheng, Cutler, Ross, Chang, Ya
• Granted Patent: US 7,630,571 B2
• Time in Patent Office: Days
• US Class Current: 382/254
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/68 of symmetry
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A panoramic camera is configured to automatically determine parameters of a table upon which the camera is situated as well as positional information of the camera relative to the table. In an initialization stage, table edges are detected to create an edge map. A Hough transformation-like symmetry voting operation is performed to clean up the edge map and to determine camera offset, camera orientation and camera tilt. The table is then fit to a table model to determine table parameters. In an operational stage, table edges are detected to create an edge map and the table model is fit to the edge map. The output can then be used for further panoramic image processing such as head size normalization, zooming, compensation for camera movement, etc.

Claims

1. One or more computer-readable media containing executable instructions that, when executed, implement a method comprising:

detecting edges that appear in a camera image, the edges including table edges of a table upon which a camera taking the camera image is situated;

determining which of the edges are table edges;

determining camera position parameters that identify a location of the camera relative to the table edges;

deriving a model of the table; and

deriving table parameters;

storing the table edges in memory.

2. The one or more computer-readable media as recited in claim 1, wherein:

the table is at least laterally symmetric; and

the determining which of the edges are table edges further comprises identifying an edge point on a first edge as a table edge point on a table edge if there is a symmetrically corresponding edge point on a second edge.

3. The one or more computer-readable media as recited in claim 2, wherein symmetry voting is used to determine which of the edges are table edges.

4. The one or more computer-readable media as recited in claim 1, wherein the camera position parameters further comprise an offset from a table center along a first medial axis of the table and an offset from the table center along a second medial axis of the table.

5. The one or more computer-readable media as recited in claim 4, wherein the camera position parameters further comprise a direction and magnitude of a camera tilt.

6. The one or more computer-readable media as recited in claim 1, further comprising storing the table model in computer-readable memory.

7. The one or more computer-readable media as recited in claim 1, wherein the detecting edges is performed when the table is uncluttered and the one or more computer-readable media further comprise instructions that, when executed, perform the additional steps of:

detecting edges that appear in a camera image included the table when the table is cluttered;

fitting the table model to the detected edges; and

using the table parameter information to normalize head sizes in the camera image.

8. A camera device, comprising:

means for capturing a panoramic image;

an edge detector configured to identify edges included in the panoramic image to create an edge map that includes edges corresponding to table edges of a table on which the panoramic camera is situated;

a voting module configured to filter the edge map to derive a symmetry-enhanced edge map and to derive camera position parameters; and

a fitting module configured to fit the symmetry-enhanced edge map to a table model to derive table dimensions.

9. The camera device as recited in claim 8, wherein the voting module is further configured to discard an edge point if there is no symmetrically corresponding edge point in the edge map.

10. The camera device as recited in claim 8, wherein the fitting module is further configured to utilize a trigonometric fitting function to fit the symmetry-enhanced edge map to the table model.

11. The camera device as recited in claim 8, wherein the fitting module is further configured to utilize a quadratic fitting function to fit the symmetry-enhanced edge map to the table model.

12. The camera device as recited in claim 8, wherein the camera position parameters further comprise a first camera offset from a table center along a first table axis and a second camera offset from the table center along a second table axis.

13. The camera device as recited in claim 8, further comprising an actuatable training initiation button that, when actuated, initiates a process to derive the table dimensions.

14. The camera device as recited in claim 8, wherein:

the edge map further comprises a first edge map that is created when the table is uncluttered;

the edge detector is further configured to create a second edge map when the table is cluttered; and

the fitting module is further configured to fit the table model to the second edge map.

15. The camera device as recited in claim 14, further comprising a head size normalization module configured to normalize head sizes of people seated around at least a portion of the perimeter of the table in the panoramic image.

16. A method, comprising:

creating a first edge map from a camera image that includes a table having table edges, the first edge map being created when no objects are situated on top of the table in the camera image;

identifying the table edges from the first edge map;

determining a camera position;

fitting the first edge map to a table model to identify table parameters;

creating a second edge map when one or more objects are situated on top of the table in the camera image;

fitting the table model to the second edge map; and

performing a procedure on the camera image that utilizes the table parameters.

17. The method as recited in claim 16, wherein the performing a procedure on the camera image further comprises performing a head size normalization on the camera image to normalize head sizes of people appearing in the camera image.

18. The method as recited in claim 16, wherein the determining a camera position further comprising determining at least a first camera offset and a second camera offset, the first camera offset being a distance that the camera is located from a table center along a first table axis, the second camera offset being a distance that the camera is located from the table center along a second table axis.

19. The method as recited in claim 16, wherein the determining a camera position further comprising determining a direction and magnitude of a camera tilt.

20. The method as recited in claim 16, wherein the fitting the first edge map step further comprises utilizing a quadratic function to fit the first edge map to a table model.