Pointing direction calibration in video conferencing and other camera-based system applications

• US 6,531,999 B1
• Filed: 07/13/2000
• Issued: 03/11/2003
• Est. Priority Date: 07/13/2000
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Trajkovic, Miroslav
• Primary Examiner(s): Saras, Steven
• Assistant Examiner(s): Spencer, William C.
• Application Number:
  US09/615,879
• Time in Patent Office: 971 Days
• Field of Search: 345/157, 345/863, 345/162, 345/178
• US Class Current: 345/157
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A camera-based system implements a calibration procedure for determining, for a given system user, calibration information that characterizes the manner in which the user points within the system. The system directs the user to point separately at each point in a set of two or more points at different positions on a display. The system then detects, for each of the points pointed to by the user, a location of a pointing finger of the user in an image generated by each camera in a set of cameras of the system, such that for each of the points the location of the pointing finger and the corresponding point on the display specify a line. The lines associated with the points are then used to compute a reference point or other pointing direction calibration information for the user. The calibration information for the user may be stored with user-identifying information in a database of the system, and is utilized in subsequent pointing direction determinations for that user.

Claims

1. A method of performing a calibration of a pointing direction for a user in a camera-based system, the method comprising the steps of:

directing the user to point separately at each point in a set of two or more points at different positions in the system;

detecting for each of the points pointed to by the user a location of a pointing finger of the user in an image generated by each of one or more cameras of the system, such that for each of the points the location of the pointing finger and the corresponding point specify a line;

computing pointing direction calibration information for the user based at least in part on the lines associated with the two or more points; and

determining a line of symmetry of the user in two or more images generated by cameras of the system, and verifying the calibration information by determining if a relationship between the calibration information and the line of symmetry of the user meets a specified criterion.

2. The method of claim 1 wherein the system comprises at least two cameras, and the detecting step comprises determining the location of the pointing finger of the user in both an image generated by a first camera and an image generated by a second camera.

3. The method of claim 1 further comprising the steps of determining user-identifying information for the user, and storing the calibration information for the user in association with the user-identifying information.

4. The method of claim 1 wherein the computing step includes determining the calibration information for the user at least in part as an intersection of the lines associated with the two or more points.

5. The method of claim 1 wherein the calibration information comprises a reference point associated with the user.

6. The method of claim 5 wherein the reference point comprises a point on or near a head of the user.

7. The method of claim 1 wherein at least a subset of the points comprise icons presented on a display of the system.

8. The method of claim 7 wherein only a single one of the icons is presented on the display at a given point in time.

9. The method of claim 7 wherein at least a subset of the icons are presented simultaneously on the display.

10. An apparatus for use in performing a calibration of a pointing direction for a user in a camera-based system, the apparatus comprising:

a plurality of cameras; and

a processing system coupled to the plurality of cameras, the processing system being operative;

(i) to direct the user to point separately at each point in a set of two or more points at different positions in the camera-based system;

(ii) to detect for each of the points pointed to by the user a location of a pointing finger of the user in an image generated by each of at least a subset of the plurality of cameras, such that for each of the points the location of the pointing finger and the corresponding point specify a line;

(iii) to compute pointing direction calibration information for the user based at least in part on the lines associated with the two or more points; and

(iv) to determine a line of symmetry of the user in two or more images generated by the cameras, and to verify the calibration information by determining if a relationship between the calibration information and the line of symmetry of the user meets a specified criterion.

11. The apparatus of claim 10 wherein the processing system determines the location of the pointing finger of the user in both an image generated by a first one of the cameras and an image generated by a second one of the cameras.

12. The apparatus of claim 10 wherein the processing system is further operative to determine user-identifying information for the user, and to store the calibration information for the user in association with the user-identifying information.

13. The apparatus of claim 10 wherein the processing system determines the calibration information for the user at least in part as an intersection of the lines associated with the two or more points.

14. The apparatus of claim 10 wherein the calibration information comprises a reference point associated with the user.

15. The apparatus of claim 14 wherein the reference point comprises a point on or near a head of the user.

16. The apparatus of claim 10 wherein at least a subset of the points comprise icons presented on a display of the camera-based system.

17. The apparatus of claim 16 wherein only a single one of the icons is presented on the display at a given point in time.

18. The apparatus of claim 16 wherein at least a subset of the icons are presented simultaneously on the display.

19. An article of manufacture comprising a storage medium for storing one or more programs for use in performing a calibration of a pointing direction for a user in a camera-based system, wherein the one or more programs when executed by a processor implement the steps of:

directing the user to point separately at each point in a set of two or more points at different positions in the system;

detecting for each of the points pointed to by the user a location of a pointing finger of the user in an image generated by each of one or more cameras of the system, such that for each of the points the location of the pointing finger and the corresponding point specify a line;

computing pointing direction calibration information for the user based at least in part on the lines associated with the two or more points; and

determining a line of symmetry of the user in two or more images generated by cameras of the system, and verifying the calibration information by determining if a relationship between the calibration information and the line of symmetry of the user meets a specified criterion.