Method and apparatus for tracking listener's head position for virtual stereo acoustics

• US 8,331,614 B2
• Filed: 12/28/2006
• Issued: 12/11/2012
• Est. Priority Date: 03/28/2006
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Mannerheim, Pal, Kim, Youngtae, Nelson, Phillip Arthur
• Primary Examiner(s): Wang, Claire X
• Assistant Examiner(s): VANCHY JR, MICHAEL J
• Application Number:
  US11/646,472
• Publication Number:
  US 20070230743A1
• Time in Patent Office: 2,175 Days
• Field of Search: 382/103, 382/154
• US Class Current: 382/103
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

A method and apparatus for tracking a listener'"'"'s head position for virtual stereo acoustics. The method of tracking the head position of a listener includes: obtaining face images of the listener using two image pickup units; tracking the skin color of an image, thereby obtaining the two-dimensional (2D) coordinate value of the listener'"'"'s position; and obtaining the distance between the image pickup units and the listener using triangulation.

Claims

1. A method of tracking a position of a head of a listener comprising a skin region, the method comprising:

obtaining two images using two image pickup units;

determining the head of the listener in at least one of the two images by detecting an edge of the skin region of the head of the listener;

tracking a skin color of one of the images, thereby obtaining a 2-dimensional (2D) coordinate value of the position, wherein a Gaussian skin classifier is applied only to a region of interest (ROI) consisting of the head of the listener substantially bound by the detected edge of the skin region of the head in order to reduce computation required to achieve color tracking; and

obtaining a distance between the image pickup units and the listener using stereo area correlation.

2. The method of claim 1, wherein the obtaining of the 2D coordinate value of the listener'"'"'s position comprises:

generating a binary image of the skin region by detecting the edge of the skin region of the at least one of the images;

expanding the skin binary image using a line-structuring element;

detecting the skin region corresponding to the head by performing morphological image processing; and

obtaining a center of the head.

3. The method of claim 2, wherein the method is performed in units of frames.

4. The method of claim 1, wherein the obtaining the distance between the image pickup units and the listener comprises:

obtaining the two images using the two image pickup units from different angles of vision, respectively;

obtaining a distance (x_r−

x_l) between the two images; and

obtaining the distance between the image pickup units and the listener using the following equation;

Zo=fd/(x_r−

x_l), and

wherein Zo is the distance and assuming that coordinates of a 3-dimensional position of the listener are P(Xo, Yo, Zo), a focal distance of each of the image pickup units is f, and a distance between the image pickup units is d.

12. A non-transitory computer readable recording medium having embodied thereon a computer program for executing the method of claim 1.

5. An apparatus for tracking a position of a head of a listener comprising a skin region, the apparatus comprising:

a first image pickup unit capturing a first image of the face of the listener;

a second image pickup unit capturing a second image of the face of the listener from a second angle of vision different from the first;

a 2-dimensional (2D) coordinate value generation unit generating a 2D coordinate value of the position by tracking a skin color of the image, wherein the 2D coordinate value generation unit comprises a skin region detection unit determining the head of the listener in at least one of the two images by detecting an edge of the skin region of the head and detecting the skin region using a Gaussian skin classifier that is applied only to a region of interest (ROI) consisting of the head of the listener substantially bound by the detected edge of the skin region of the head in order to reduce computation required to achieve color tracking; and

a distance calculation unit calculating a distance between the image pickup units and the listener using stereo area correlation.

6. The apparatus of claim 5, wherein the 2D coordinate value generation unit comprises:

a binary image generation unit generating a binary image of the skin region by detecting the edge of the skin region of the first image;

a binary image expansion unit expanding the skin binary image using a line-structuring element;

a head detection unit detecting the skin region corresponding to the head by performing morphological image processing; and

a head coordinate generation unit obtaining a center of the head.

7. The apparatus of claim 6, wherein the tracking of the position of the head is performed in units of frames.

8. The apparatus of claim 5, wherein the distance calculation unit comprises:

an image acquisition unit obtaining the two images of the listener using the two image pickup units from different angles of vision, respectively;

an inter-image distance obtaining unit obtaining a distance (x_r−

x_l) between the two images; and

a listener'"'"'s position calculation unit obtaining the distance between the image pickup units and the listener by using the following equation;

Zo=fd/(x_r−

x_l), and

wherein Zo is the distance and the coordinates of the 3-dimensional position of the listener are P(Xo, Yo, Zo), the focal distance of each of the image pickup units is f, and the distance between the image pickup units is d.

9. An apparatus, comprising:

two image pickup units respectively capturing two images, wherein a head of a listener is determined from at least one of the two images by detecting an edge of a skin region of the head;

a 2-dimensional coordinate value generation unit generating a 2D coordinate value of a position of the head by tracking a skin color region of one of the captured images, wherein a Gaussian skin classifier is applied only to a region of interest (ROI) consisting of the head of the listener substantially bound by the detected edge of the skin region of the head in order to reduce computation required to achieve color tracking;

a distance calculation unit calculating a distance from the image pickup units to the listener using stereo area correlation of the two images; and

a listener'"'"'s position calculation unit setting a location of a sweet spot in a multi-channel audio signal to coincide with the 2D coordinate position of the head generated by the 2-D coordinate value generation unit.

10. A method, comprising:

capturing two images from different perspectives via two image capturing units;

determining a head of a listener in at least one of the two images by detecting an edge of a skin region of the head of the listener;

determining a 2D coordinate position of the face by tracking a skin color region of one of the captured images, wherein a Gaussian skin classifier is applied only to a region of interest (ROI) consisting of the head of the listener substantially bound by the detected edge of the skin region of the head in order to reduce computation required to achieve color tracking;

calculating a distance from the image capturing units to a head of the listener via triangulation based on the two images; and

setting a location of a sweet spot in a multi-channel audio signal to coincide with the determined 2D coordinate position of the face.

13. A non-transitory computer readable recording medium having embodied thereon a computer program for executing the method of claim 10.

11. A method of resetting a location of a sweet spot in a multi-channel audio signal, the method comprising:

determining a position of a head of a listener by capturing two images from different perspectives via two image capturing units and by detecting an edge of a skin region of the head of the listener;

determining a 2D coordinate position of a face by tracking a skin color region of one of the captured images and calculating a distance from the image capturing units to the head of the listener via triangulation based on the two images, wherein a Gaussian skin classifier is applied only to a region of interest (ROI) consisting of the head of the listener substantially bound by the detected edge of the skin region of the head in order to reduce computation required to achieve color tracking; and

resetting the location of the sweet spot in the multi-channel audio signal to coincide with the determined position of the head of the listener.

14. A non-transitory computer readable recording medium having embodied thereon a computer program for executing the method of claim 11.

15. A method of resetting a location of a sweet spot in a multi-channel audio signal with respect to a location of a head of a listener, the method comprising:

tracking a head position of the listener by capturing two images from different perspectives via two image capturing units and by detecting an edge of a skin region of the head of the listener;

generating a 2D coordinate value of a position of the face by tracking a skin color region of one of the two captured images, wherein a Gaussian skin classifier is applied only to a region of interest (ROI) consisting of the head of the listener substantially bound by the detected edge of the skin region of the head in order to reduce computation required to achieve color tracking;

calculating a distance from the image pickup units to the listener using stereo area correlation of the two images; and

setting the location of the sweet spot in the multi-channel audio signal to coincide with the generated 2D coordinate position of the face.