METHOD AND APPARATUS FOR IMAGE DISPLAY CONTROL ACCORDING TO VIEWER FACTORS AND RESPONSES

• US 20100328492A1
• Filed: 06/30/2009
• Published: 12/30/2010
• Est. Priority Date: 06/30/2009
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Fedorovskaya, Elena A., Neustaedter, Carman G.
• Granted Patent: US 8,154,615 B2
• Time in Patent Office: Days
• US Class Current: 348/231.2
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

A device and system and a method for selecting and displaying images in accordance with attention responses of an image viewer. An image capture device captures a viewer image of a spatial region potentially containing a viewer of the image display device and a viewer detection mechanism analyzes the viewer image and determines whether a potential viewer is observing the display screen. A gaze pattern_analysis mechanism identifies a viewer response to a currently displayed image by analysis of the viewer'"'"'s emotional response as revealed by the viewer'"'"'s facial expression and other sensory inputs, and the selection of an image to be displayed is then modified in accordance with the indicated viewer'"'"'s attention response.

Claims

1. An image display mechanism for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, comprising:

a display control mechanism for selecting at least one image from an image library for display by an image display device, and

an image control mechanism for determining a viewer'"'"'s attentiveness to a currently displayed image and providing a response output according to at least one of the viewer'"'"'s facially expressed and behavioral response to a currently displayed image.

2. The image display mechanism of claim 1 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, wherein the image control mechanism comprises:

an image capture device for capturing at least one viewer image in a spatial region potentially containing a viewer of the image display device,

a viewer detection mechanism for analyzing the at least one viewer image and determining whether a potential viewer of the image display screen is present in the spatial region,

a face identification mechanism for determining whether the spatial region includes a region identifiable as a face,

a line of vision process for determining whether a viewer present in the spatial region is gazing at the image display screen.

3. The image display mechanism of claim 2 wherein the image control mechanism further comprises a gaze pattern analysis mechanism for providing a response output identifying the viewer'"'"'s attentiveness to a currently displayed image.

4. The image display mechanism of claim 1 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, wherein:

the display control mechanism is responsive to the identification of a viewer'"'"'s response for modifying an image or the selection of images from the image library in accordance with the viewer response.

5. The image display mechanism of claim 1 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, wherein:

the image library contains at least one set of images, and

the display control mechanism is responsive to the identification of a viewer'"'"'s attentiveness for modifying an image selection or the set of images in accordance with the viewer response to the currently displayed image.

6. The image display mechanism of claim 1 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, wherein:

the image library contains a plurality of images, and

the display control mechanism is responsive to the identification of a viewer'"'"'s attentiveness according to solely behavioral changes of the viewer to the current image.

7. The image display mechanism of claim 4 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising:

a viewer facial library for storing facial features of at least one identified potential viewer of the displayed image,

a viewer identification mechanism for extracting viewer facial features from the viewer image and comparing the viewer facial features from the viewer image with the at least one identified potential viewer of the displayed image,

when the viewer identification mechanism identifies a correspondence between the viewer facial features extracted from the viewer image and an identified potential viewer of the displayed image, and

selecting at least one next image to be displayed from an image set containing at least one image wherein the image set corresponds to the viewer identified as corresponding to the viewer facial features from the viewer image.

8. The image display mechanism of claim 7 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer wherein the display control mechanism selects an image from the image set containing subjects besides the image viewer, and which subjects correspond in a predefined manner with the image viewer.

9. The image display mechanism of claim 8 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, wherein the display control mechanism is responsive to the identification of a viewer response for modifying the set of images corresponding to the viewer identified as corresponding to the viewer facial features from the viewer image in accordance with the viewer'"'"'s attentiveness to the current image.

10. A method for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, comprising the steps of:

selecting at least one image from an image library of a data storage medium for display by an image display device,

determining a viewer'"'"'s response to a currently displayed image by analysis of a facial expression and gaze pattern of the image viewer, and

providing a response output based upon the viewer'"'"'s response to a currently displayed image.

11. The method of claim 10 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer comprising the further step of determining a viewer'"'"'s response to a currently displayed image further and analyzing image properties to determine the image viewer'"'"'s desires and intent.

12. The method of claim 10 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising the steps of:

capturing at least one viewer image of a spatial region potentially containing a viewer of the image display device,

analyzing the at least one viewer image and determining whether a potential viewer of the image display screen is present in the spatial region,

determining whether the spatial region includes a region identifiable as a face,

determining whether a viewer present in the spatial region is observing the image display screen, and

analyzing the gaze pattern to determine a gaze fixation point of the viewer present in the spatial region and observing the image display screen and providing a corresponding response output modified according to the determined gaze fixation point.

13. The method of claim 10 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising the step of:

modifying the selection of images from the image library in accordance with the viewer'"'"'s gaze pattern response.

14. The method of claim 12 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising the step of determining according to predefined data stored in the data storage medium if the viewer'"'"'s gaze is a function of exogenous or endogenous attention.

15. The method of claim 10 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising the step of:

modifying at least one set of images in the image library in accordance with the viewer'"'"'s gaze pattern response to the current image.

16. The method of claim 10 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising the step of:

selecting an image having a predetermined relationship to the current image in accordance with the identification of a viewer'"'"'s response.

17. The method of claim 10 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising the steps of:

storing facial features of at least one identified potential viewer of the displayed image in a facial library,

extracting viewer facial features from the viewer image and comparing the viewer facial features from the viewer image with at least one stored set of facial features of an identified potential viewer, and

selecting at least one next image corresponding to a viewer identified as corresponding to the viewer facial features from the viewer image.

18. The method of claim 17 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising the steps of selecting images to be displayed from an image set corresponding to image subjects having a predetermined relationship with the image viewer and having a predetermined association with the image viewer.

19. The method of claim 17 for selecting images to be displayed in accordance with facially expressed and behavioral responses of an image viewer, further comprising the steps of:

modifying a set of images corresponding to the viewer identified as corresponding to the viewer facial features from the viewer image in accordance with the viewer'"'"'s response and gaze pattern to the current image.

20. An image display device, comprising:

an image library for storing at least one image in at least one set of images,

an image display,

a display control mechanism for selecting from the image library at least one image for display by the image display, and

an image control mechanism for determining a viewer response to a currently displayed image and providing a response output according to a viewer response to a currently displayed image.

21. The image display device of claim 20 for selecting images to be displayed in accordance with facially expressed responses of an image viewer, wherein the image control mechanism comprises:

an image capture device for capturing at least one viewer image of a spatial region potentially containing a viewer of the image display device,

a viewer detection mechanism for analyzing the at least one viewer image and determining whether a potential viewer of the image display screen is present in the spatial region,

a face identification mechanism for determining whether the spatial region includes a region identifiable as a face,

a line of vision process for determining whether a viewer present in the spatial region is viewing the image display screen, and

a facial expression analysis mechanism for providing a response output identifying a viewer response to a currently displayed image.

22. The image display mechanism of claim 21 wherein the image control mechanism further comprises a gaze pattern analysis mechanism for providing the response output according to the viewer'"'"'s attentiveness to a currently displayed image

23. The image display device of claim 20 for selecting images to be displayed in accordance with facially expressed responses of an image viewer, wherein:

the display control mechanism is responsive to the identification of a viewer response for modifying the selection of images from the image library in accordance with the viewer response.

24. The image display device of claim 20 for selecting images to be displayed in accordance with facially expressed responses of an image viewer, wherein:

the image library contains at least one set of images, and

the display control mechanism is responsive to the identification of a viewer response for modifying the set of images in accordance with the viewer response to the current image.

25. The image display device of claim 20 for selecting images to be displayed in accordance with facially expressed responses of an image viewer, wherein:

the image library contains a plurality of images, and

the display control mechanism is responsive to the identification of a viewer response to the current image for selecting an image similar to the current image.

26. The image display device of claim 22 for selecting images to be displayed in accordance with facially expressed responses of an image viewer, further comprising:

a viewer facial library for storing facial features of at least one identified potential viewer of the displayed image,

a viewer identification mechanism for extracting viewer facial features from the viewer image and comparing the viewer facial features from the viewer image with the at least one identified potential viewer of the displayed image,

when the viewer identification mechanism identifies a correspondence between the viewer facial features extracted from the viewer image and an identified potential viewer of the displayed image, and

selecting at least one next image to be displayed from an image set containing at least one image wherein the image set corresponds to the viewer identified as corresponding to the viewer facial features from the viewer image.

27. The image display device of claim 26 for selecting images to be displayed in accordance with facially expressed responses of an image viewer, wherein the image control mechanism selects a subsequent image from an image set corresponding to image subjects besides the viewer and having a predetermined association with the image viewer.

28. The image display device of claim 27 for selecting images to be displayed in accordance with facially expressed responses of an image viewer, wherein the display control mechanism is responsive to the identification of a viewer response for modifying the set of images corresponding to the viewer identified as corresponding to the viewer facial features from the viewer image in accordance with the viewer response to the current image.

29. The image display device of claim 20 for selecting images to be displayed in accordance with behavioral expressed responses of an image viewer, wherein the display control mechanism is responsive to the identification of viewer responsive behavioral expressions besides gaze patterns and facial expressions, for modifying the selection of images from the image library in accordance with the viewer response.