Display arrangement including face detection

• US 7,643,658 B2
• Filed: 01/21/2005
• Issued: 01/05/2010
• Est. Priority Date: 01/23/2004
• Status: Active Grant

• Current Assignee: Sony Europe (Amsterdam) BV (Sony Group Corp.)
• Original Assignee: Sony United Kingdom Limited (Sony Group Corp.)
• Inventors: Jennings, Victoria Sophia, Rambaruth, Ratna, Kilner, Andrew Roger, Paszkowicz, Sebastian Aleksander, Gillard, Clive Henry, Porter, Robert Mark Stefan
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): ZARKA, DAVID PETER
• Application Number:
  US11/040,587
• Publication Number:
  US 20050197923A1
• Time in Patent Office: 1,810 Days
• Field of Search: 382/118
• US Class Current: 382/118
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition
• CPC Class Codes: G09F 27/00 Combined visual and audible advertising or displaying, e.g. for public address

Abstract

A display arrangement comprises an image display device having a first camera directed towards positions adopted by users viewing the display; one or more further cameras directed towards other respective locations; a face detector for detecting human faces in images captured by the cameras; and means for detecting those faces which appear in images captured by both the first camera and at least one of the further cameras.

Claims

1. A display arrangement, comprising:

an image display device to display at least one image representing advertising material for goods or services for sale in a shop, the image display device including a first camera directed towards locations adopted by viewers viewing the image display device, and the first camera being configured to capture images of faces of the viewers as the viewers view the image display device;

one or more further cameras directed towards other respective locations, at least one of the further cameras being directed towards a checkout of the shop to capture images of purchasers;

a face detector for detecting human faces of the purchasers and the viewers in images captured by the cameras, the face detector being arranged to detect a total dwell time of each of the faces detected by the first camera, and the dwell time is a time that each of the faces is directed towards the image display device; and

a correlation detector to correlate faces which appear in images captured by both the first camera and at least one of the further cameras by detecting whether a purchaser whose face is detected at the checkout was also detected as a viewer of the image display device.

2. The arrangement according to claim 1, further comprising one or more further image display devices,

and at least one of the further cameras is directed towards positions adopted by viewers viewing one of the further image display devices.

3. The arrangement according to claim 1, wherein:

at least one of the further cameras is directed towards a position the purchaser would adopt in selecting goods for sale, and

at least one image displayed on the image display device represents advertising material for the goods for sale.

4. The arrangement according to claim 1, further comprising:

logic to detect whether goods or services being purchased by the purchaser include goods or services viewed on the image display device by the purchaser.

5. The arrangement according to claim 1, further comprising:

logic to generate data indicating a number of faces detected at each camera.

6. The arrangement according to claim 1, further comprising:

logic to generate data indicating a cumulative dwell time of the faces of the purchasers and the viewers.

7. The arrangement according to claim 1, further comprising:

logic to detect an order in which the face of a viewer is detected at two or more of the cameras.

8. The arrangement according to claim 1, further comprising:

a reader for a portable storage device carried by the viewer, the portable storage device storing at least a viewer identifier; and

a store for storing data correlating the viewer identifiers and detected faces.

9. The arrangement according to claim 1, wherein the face detector detects a cumulative dwell time representing a total time spent viewing the image display device by each of the faces detected by the first camera within a given time period or by a predetermined category of the detected faces.

10. The arrangement according to claim 1, further comprising:

a statistical analysis module configured to maintain cumulative statistical data on the faces of the viewers detected locally by the first camera, including a headcount of distinct faces, gender data, and age category data.

11. The arrangement according to claim 10, wherein the statistical analysis module mains cumulative statistical data on an ethnic origin, specific physical characteristics and categories of emotional expressions on the faces of the viewers.

12. The arrangement according to claim 1, wherein the image display device displays the advertising material according to an age of a viewer.

13. A display method, comprising:

using a first camera to capture images of viewers viewing an image display device at a first location, at least one image displayed on the image display device representing advertising material for goods or services for sale in a shop;

using one or more further cameras to capture images of the viewers at one or more other respective locations, at least one of the further cameras being directed towards a checkout of the shop to capture images of purchasers;

detecting human faces in the captured images;

detecting a total dwell time of each of the faces detected by the first camera, the dwell time being a time that each of the faces is directed towards the image display device; and

correlating the faces which appear in the images captured by the first camera and the at least one of the further cameras by detecting whether a purchaser whose face is detected at the checkout was also detected as a viewer of the image display device.

14. A computer readable storage medium including computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform the method according to claim 13.

15. The method according to claim 13, further comprising:

using an eigenface method of face recognition in the detecting human faces.