Multimedia device, multiple image sensors having different types and method for controlling the same

• US 8,577,092 B2
• Filed: 08/30/2011
• Issued: 11/05/2013
• Est. Priority Date: 11/11/2010
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Taehyeong, Kim, Sangki, Kang, Mingoo, Cho, Yongwon, Im, Soungmin
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): Harandi, Siamak
• Application Number:
  US13/220,729
• Publication Number:
  US 20120114194A1
• Time in Patent Office: 798 Days
• Field of Search: None
• US Class Current: 382/115
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04H 60/27 Arrangements for recording or accumulating broadcast information or broadcast-related information
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04H 60/65 for using the result on users' side

Abstract

A method for controlling a multimedia device, which uses a plurality of heterogeneous image sensors, comprises the steps of identifying at least one user located in the periphery of the multimedia device under the control of a first image sensor for extracting depth data and a second image sensor for extracting color data; accessing a memory that stores a preference channel per user; extracting information on the preference channel corresponding to the identified user, from the accessed memory; performing a control operation for switching to the preference channel based on the extracted information; and monitoring change of a user located in front of the multimedia device.

Claims

1. A method for managing information in a display device, comprising:

acquiring a first image;

acquiring a second image;

detecting a person in the first image;

determining coordinate information of the person in the first image;

extracting a feature of the person from the second image based on the coordinate information;

comparing the extracted feature to reference information;

recognizing the person based on the comparison;

determining whether a single user or a plurality of users have been identified;

extracting information on a preference channel corresponding to the single user by accessing a memory that stores a favorite list per user when the single user is determined to have been identified, wherein the favorite list per user includes information regarding a broadcast program of a channel viewed twice or more by the single user at a specific time on a specific day; and

performing a control operation for switching to the preference channel based on the extracted information.

2. The method as claimed in claim 1, further comprising:

taking the first image located in a periphery of the display device by using a first image sensor;

extracting depth data from the taken first image;

detecting a face of the user by using the extracted depth data;

taking the second image on the detected face of the user by using a second image sensor;

extracting feature information from the taken second image;

accessing a memory that stores data corresponding to the extracted feature information; and

extracting information identifying a specific user stored in the memory.

3. The method as claimed in claim 1, further comprising:

displaying a guide message indicating the preference channel; and

switching to the preference channel when a signal selecting the preference channel is applied.

4. A multimedia device that uses a plurality of heterogeneous image sensors, the multimedia device comprising:

a first image sensor to acquire a first image;

a second image sensor to acquire a second image;

a processor to detect a person in the first image and to extract a feature of the person from the second image, wherein the processor determines coordinate information of the person in the first image and extracts the feature of the person in the second image based on the coordinate information, wherein the first and second image sensors have overlapping fields of view, the coordinate information provides an indication of a distance to the person, and the processor compares the extracted feature to reference information and recognizes the person based on the comparison, and wherein the processor includes a decision module to determine whether a single user has been identified or a plurality of users have been identified;

a memory to store a preference channel per user, wherein the memory includes;

a first generator for generating a favorite list per user, which stores a broadcast program of a channel viewed twice or more by the single user at a specific time on a specific day, and

a second generator for generating a family favorite list that stores a broadcast program of a channel viewed twice or more by the plurality of users at a specific time on a specific day;

a display module for displaying a guide message indicating the corresponding preference channel; and

a controller for performing a control operation for switching to the corresponding preference channel when a signal selecting the corresponding preference channel is applied.

5. The multimedia device as claimed in claim 4, wherein the first image sensor corresponds to a depth camera, and the second image sensor corresponds to an RGB camera.

6. The multimedia device as claimed in claim 4, wherein the processor extracts information on a preference channel corresponding to the single user by accessing a memory that stores a favorite list per user generated by the first generator when the single user has been identified in accordance with the determined result of the decision module.

7. The multimedia device as claimed in claim 4, wherein the processor extracts information on a preference channel corresponding to the plurality of users by accessing a memory that stores the family favorite list generated by the second generator when the plurality of users have been identified in accordance with the determined result of the decision module.

8. The multimedia device as claimed in claim 4, wherein the first image sensor and the second image sensor are embedded in the multimedia device, or the first image sensor and the second image sensor are designed as a separate external module.

9. The multimedia device as claimed in claim 8, wherein when the first image sensor and the second image sensor are embedded in the multimedia device,

the processor takes a first image located in a periphery of the multimedia device by using the first image sensor, extracts depth data from the taken first image, detects a face of the user by using the extracted depth data, takes a second image on the detected face of the user by using the second image sensor, extracts feature information from the taken second image, accesses a memory that stores data corresponding to the extracted feature information, and extracts information identifying a specific user stored in the memory.

10. A method for managing information in a display device, comprising:

acquiring an image;

detecting a person in the image;

determining coordinate information of the person in the image;

extracting a feature of the person from the image based on the coordinate information;

comparing the extracted feature to reference information;

recognizing the person based on the comparison,

determining whether a single user or a plurality of users have been identified;

extracting information on a preference channel corresponding to a plurality of users by accessing a memory that stores a family favorite list when a plurality of users have been identified, wherein the family favorite list includes info nation regarding a broadcast program of a channel viewed twice or more by the plurality of users at a specific time on a specific day; and

performing a control operation for switching to the preference channel based on the extracted information.