Image processing apparatus, content delivery system, image processing method, and program

• US 8,971,635 B2
• Filed: 01/20/2011
• Issued: 03/03/2015
• Est. Priority Date: 03/02/2010
• Status: Active Grant

• Current Assignee: NEC Solution Innovators, Ltd (NEC Corporation)
• Original Assignee: NEC Solution Innovators, Ltd (NEC Corporation)
• Inventors: Kodaira, Takayuki, Imaizumi, Satoshi
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US13/582,198
• Publication Number:
  US 20120321197A1
• Time in Patent Office: 1,503 Days
• Field of Search: 382/118, 382/116, 382/165, 382/195, 382/190, 382/103, 382/203, 382/117, 382/159, 382/289, 382/284, 348/348, 340/345
• US Class Current: 382/195
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

Disclosed herein is an image processing apparatus including an upper body feature data storage unit (110) which stores upper body directional feature data, which indicates the upper body of a person and indicates that the upper body is facing a specific direction, for each of a plurality of directions in conjunction with directional data indicative of the direction of the upper body directional feature data; and an upper body detection unit (140) which extracts upper body image data indicative of the upper body of the person from image data by reading the plurality of upper body directional feature data stored in the upper body feature data storage unit (110) in conjunction with the directional data and using each of the plurality of upper body directional feature data.

Claims

1. An image processing apparatus, comprising:

an upper body feature data storage device configured to store upper body directional feature data, which indicates an upper body including ahead and shoulders of a person and indicates that the upper body is facing a specific direction, for each of a plurality of directions in conjunction with directional data indicative of a direction of the upper body directional feature data; and

an upper body detection device configured to extract upper body image data indicative of the upper body of the person from image data by reading the plurality of upper body directional feature data stored in the upper body feature data storage device in conjunction with the directional data and using each of the plurality of upper body directional feature data;

a facial feature data storage device configured to store facial directional feature data indicative of a face of the person in conjunction with the directional data;

a facial image detection device configured to obtain the upper body image data in conjunction with the directional data corresponding to the upper body directional feature data used when the upper body image data is extracted, which reads the facial directional feature data corresponding to the obtained directional data from the facial feature data storage device, and configured to extract facial image data indicative of the face of the person from the upper body image data using the read facial directional feature data; and

a facial image storage device configured to store the facial image data detected by the facial image detection device.

2. The image processing apparatus according to claim 1, further comprising:

an upper body image storage device configured to store the upper body image data, detected by the upper body detection device, in conjunction with the directional data corresponding to the upper body directional feature data used when the upper body image data is extracted.

3. The image processing apparatus according to claim 1,

wherein the facial image detection device does not read the upper body image data which is in conjunction with at least one predetermined directional data but reads the upper body image data which is in conjunction with a remaining directional data.

4. The image processing apparatus according to claim 1, further comprising:

an attribute data storage device configured to store attribute feature data indicative of features of a face for each attribute; and

an attribute detection device configured to perform classification on the facial image data for each age and sex by reading and using the attribute feature data stored in the attribute data storage device.

5. The image processing apparatus according to claim 4,

wherein the attribute detection device is configured to perform classification on only the facial image data corresponding to the predetermined directional data for each age and sex.

6. The image processing apparatus according to claim 1,

wherein the image data is generated by imaging a front of an image display apparatus.

7. The image processing apparatus according to claim 6,

wherein the directional data is classified into visual directional data which indicates that an upper body indicated by the upper body image data is facing the image display apparatus and non-visual directional data which indicates that the upper body is not facing the image display apparatus, and

wherein the image processing apparatus further comprises:

an upper body image storage device configured to store the upper body image data, detected by the upper body detection device, in conjunction with the directional data corresponding to the upper body directional feature data used when the upper body image data is extracted; and

an audience rating calculation device configured to calculate a ratio of a number of the upper body image data corresponding to the visual directional data to a total number of the upper body image data as an audience rating using the upper body image data stored in the upper body image storage device.

8. The image processing apparatus according to claim 1,

wherein the image data is generated by imaging a front of an image display apparatus,

wherein the facial feature data storage device configured to store the facial feature data for each direction of the face,

wherein the facial image storage device is configured to store the facial image data in conjunction with a direction corresponding to the facial feature data used when the facial image data is extracted and in conjunction with the directional data corresponding to the upper body directional feature data used when the upper body image data, from which the facial image data is extracted, is extracted, and

wherein the image processing apparatus further comprises an audience rating calculation device configured to calculate a number of the facial image data, estimated that the image display apparatus is viewed, using a combination of the direction of the face stored in the facial image storage device and the directional data, and configured to calculate a ratio of the calculated number to a total number of the facial image data as an audience rating.

9. The image processing apparatus according to claim 6,

wherein the directional data is classified into visual directional data which indicates that the upper body indicated by the upper body image data is facing the image display apparatus and non-visual directional data which indicates that the upper body is not facing the image display apparatus,

wherein the upper body detection device is configured to immediately obtain the image data from an imaging apparatus configured to generate the image data, and is configured to extract the upper body image data from the immediately obtained image data, and

wherein the image processing apparatus further comprises a display control device configured to control the image display apparatus based on a number of the upper body image data which is extracted from the immediately obtained image data and corresponds to the visual directional data.

10. The image processing apparatus according to claim 6,

wherein the upper body detection device is configured to immediately obtain the image data from the imaging apparatus configured to generate the image data, and is configured to extract the upper body image data from the immediately obtained image data, and

wherein the image display apparatus is controlled based on the directional data corresponding to the upper body image data extracted from the immediately obtained image data.

11. An image processing apparatus, comprising:

an image acquisition device configured to obtain image data; and

a first image processing device configured to extract upper body image data indicative of an upper body including a head and shoulders of a person from the image data obtained from the image acquisition device, and configured to specify a direction of a body indicated by the upper body image data; and

a second image processing device configured to read a facial directional feature data indicative of a face of the person from a facial feature data storage device configured to store facial directional feature data in conjunction with a direction of a body using the specified direction by the first image processing device, and configured to extract facial image data indicative of the face of the person from the upper body image data extracted by the first image process device using the read facial directional feature data.

12. A content delivery apparatus, comprising:

a content storage device configured to store image content;

an image display apparatus configured to receive the image content stored in the content storage device over a communication network, and configured to display the received image content;

an imaging apparatus configured to image an area in front of the image display apparatus and configured to generate image data; and

an image processing apparatus configured to receive the image data over the communication network and configured to process the received image data,

wherein the image processing apparatus comprises;

an upper body feature data storage device configured to store upper body directional feature data, which indicates an upper body including ahead and shoulders of a person and indicates that the upper body is facing a specific direction, for each of a plurality of directions in conjunction with directional data indicative of a direction of the upper body directional feature data; and

an upper body detection device configured to extract upper body image data indicative of the upper body of the person from image data by reading the plurality of upper body directional feature data stored in the upper body feature data storage device in conjunction with the directional data and using each of the plurality of upper body directional feature data;

a facial feature data storage device configured to store facial directional feature data indicative of a face of the person in conjunction with the directional data;

a facial image detection device configured to obtain the upper body image data in conjunction with the directional data corresponding to the upper body directional feature data used when the upper body image data is extracted, configured to read the facial directional feature data corresponding to the obtained directional data from the facial feature data storage device, and configured to extract facial image data indicative of the face of the person from the upper body image data using the read facial directional feature data; and

a facial image storage device configured to store the facial image data detected by the facial image detection device.

13. An image processing method, comprising:

storing upper body directional feature data, which indicates an upper body including a head and shoulders of a person and indicates that the upper body is facing a specific direction, for each of a plurality of directions in conjunction with directional data indicative of the direction of the upper body directional feature data;

extracting upper body image data indicative of the upper body of the person from image data by using each of the plurality of the upper body directional feature data;

storing facial directional feature data indicative of face of the person in conjunction with the directional data;

obtaining the upper body image data in conjunction with the directional data corresponding to the upper body directional feature data used when the upper body image data is extracted;

reading the facial directional feature data corresponding to the obtained directional data from the facial feature data storage device;

extracting facial image data indicative of the face of the person from the upper body image data using the read facial directional feature data; and

storing the facial image data detected by the facial image detection device.

14. A non-transitory computer-readable recording medium for storing a program that makes a computer execute:

storing upper body directional feature data, which indicates an upper body including a head and shoulders of a person and indicates that the upper body is facing a specific direction, for each of a plurality of directions in conjunction with directional data indicative of the direction of the upper body directional feature data;

extracting upper body image data indicative of the upper body of the person from an image data by using each of the plurality of the upper body directional feature data;

storing facial directional feature data indicative of a face of the person in conjunction with the directional data;

obtaining the upper body image data in conjunction with the directional data corresponding to the upper body directional feature data used when the upper body image data is extracted;

reading the facial directional feature data corresponding to the obtained directional data from the facial feature data storage device;

extracting facial image data indicative of the face of the person from the upper body image data using the read facial directional feature data; and

storing the facial image data detected by the facial image detection device.