Head tracking and color video acquisition via near infrared luminance keying

• US 20020118861A1
• Filed: 02/15/2001
• Published: 08/29/2002
• Est. Priority Date: 02/15/2001
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Iyer, Subramonium, Jouppi, Norman
• Granted Patent: US 6,785,402 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images

Abstract

A system and corresponding method for head tracking and color video acquisition via near infrared luminance keying is disclosed. The head of the local user is tracked in real time by uniformly illuminating a rear projection screen that surrounds a display cube with light having a wavelength in the near-infrared spectrum. A near-infrared image of the head of the local user is acquired by a near-infrared camera equipped with filters that discern the difference between the illuminated rear projection screen and any foreground illumination. A color image of the head of a local user and the projection of a remote location are acquired by a color camera, placed in close proximity to the near-infrared camera. A bounding box is then placed around the head of the local user in the near-infrared image. This bounding box is then translated to the view space of the color camera. This translated image is then used to crop the color image. The cropped color image is then transmitted to the remote location.

Claims

1. A method of detecting and tracking the head location of a user within an area comprising the steps of:

(a) uniformly illuminating a rear projection screen that is located in a position behind the user with light having a wavelength in the near-infrared spectrum;

(b) projecting an image of a location remote from the user onto at least a portion of the rear projection screen, where the image projector emits light having a wavelength outside of the near-infrared spectrum;

(c) defining the boundary of the head location based on detected near-infrared luminance differences within the area with respect to the rear projection screen.

2. The method of claim 1, further comprising the steps of:

(d) providing a border around the defined boundary; and

(e) positioning the bordered object on a viewing screen relative to the projected image.

3. The method of claim 1, wherein the illuminated area of step (c) comprises a background area and any detected luminance differences are considered foreground information.

4. The method of claim 3, wherein the foreground information represents the head location of the user.

5. The method of claim 2, wherein step (d) further comprises the steps of:

(d1) scanning the pixels representing the bounded image;

(d2) determining the top portion of the foreground image;

(d3) moving down a predetermined distance from the top portion of the foreground image of step (d2);

(d4) determining the leftmost portion of the foreground image, the leftmost portion provided by the pixel located at the left edge of the foreground image of step (d3);

(d5) determining the rightmost portion of the foreground image, the rightmost portion provided by the pixel located at the right edge of the foreground image of step (d3); and

(d6) providing a border around the image bounded by the pixel locations determined in steps (d2)-(d5).

6. The method of claim 5, wherein determining the top potion of the foreground image, comprises the step of:

detecting the presence of n adjacent pixels having a pixel luminance value greater than a threshold level, where n is an integer greater than one.

7. The method of claim 6, wherein n equals 10.

8. The method of claim 2, further comprising the steps of:

(f) acquiring a color image of the user; and

(g) transmitting a cropped version of the color image to the location remote from the user, the cropped color image being based in part on the defined boundary of step (d).

9. A system for detecting and tracking the head location of a user within an area, comprising:

a rear projection screen located in a position substantially behind the user;

means for illuminating the rear projection screen with light having a wavelength in the near-infrared spectrum;

means for projecting a video of a location remote from the user onto the rear projection screen, the projecting means emitting light having a wavelength outside of the near-infrared spectrum; and

a camera unit located in a position substantially in front of the user, the camera unit including a near-infrared camera and a color camera maintained in a stacked relationship with respect to each other, wherein the near-infrared camera detects head location based on luminance differences between the illuminated rear projection screen and an a video object in relative proximity to the projection screen.

10. The system of claim 9, further including means for processing the information detected by the near-infrared camera and the color camera.

11. The system of claim 10, wherein the color camera acquires a color image of the user, and the processing means further includes means for formatting the color image and transmitting the formatted colored image to the remote location.

12. The system of claim 9, wherein the video provided by the projecting means is a visible light video.

13. The system of claim 9, wherein the illumination of the rear projection screen provided by the illuminating means is substantially uniform.

14. The system of claim 13, wherein the illuminating means comprises a group of incandescent lights covered with visible-cut near-infrared pass filters.

15. The system of claim 9, wherein the near-infrared camera is operative to detect the illuminated rear projection screen and any illumination differences relative to the rear projection screen, and the color camera is operative to detect the image projected onto the rear projection screen and the user.

16. The system of claim 15, wherein the near-infrared camera further includes visible-cut near-infrared pass filters.

18. The system of claim 17, further including a processor configured to receive information from the near-infrared camera and the color camera, the processor further configured to generate a cropped color image in response to the information provided by the near-infrared camera and the color camera and transmit the cropped color image to a location remote from the user.

17. A head detection and tracking system, comprising:

a rear projection screen located in a position substantially behind a user;

an illuminator operative to uniformly illuminate the rear projection screen with light having a wavelength in the near-infrared spectrum;

a projector operative to project a video image of a location remote from the user onto the rear projection screen, the projector emitting light having a wavelength outside of the near-infrared spectrum; and

a camera unit located in a position substantially in front of the user, the camera unit including a near-infrared camera and a color camera maintained in a stacked relationship with respect to each other, wherein the near-infrared camera detects the head location of the user based on luminance differences between the uniformly illuminated rear projection screen and a video object in relative proximity to the rear projection screen.