Display device content selection through viewer identification and affinity prediction

• US 9,210,313 B1
• Filed: 02/11/2010
• Issued: 12/08/2015
• Est. Priority Date: 02/17/2009
• Status: Active Grant

• Current Assignee: Ikorongo Technology, LLC (Concert Technology Corporation)
• Original Assignee: Ikorongo Technology, LLC (Concert Technology Corporation)
• Inventors: Svendsen, Hugh Blake
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): Quiett, Carramah J
• Application Number:
  US12/703,846
• Time in Patent Office: 2,126 Days
• Field of Search: 348/231.2, 348/231.3, 348/231.6, 348/231.9, 348/333.02, 348/333.05, 348/333.11, 382/115, 382/118
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/179 metadata assisted face recognition
• CPC Class Codes: G11B 27/19 by using information detectable on the record carrier
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H04N 9/8715 : involving the mixing of the reproduced video signal with a non-recorded signal, e.g. a text signal

Abstract

Method and device are described for customizing the content selection to present on a display device based on identifying the viewer of the device. In one embodiment, the present disclosure relates to selecting from among a group of digital images, those images that are most likely to be of interest to the viewer. In general, many of the images available to the display device will be comprised of images containing subject faces. Using the relationship information to predict subject affinity, the display device computes an image affinity for each image by accumulating the individual subject affinity predictions between the viewer and each subject identified in each image. The image affinities are used to select images for presentation on the display device.

Claims

1. A computer implemented method comprising:

obtaining information identifying a reference person;

obtaining information identifying a collection of images associated with the reference person;

obtaining information identifying a social relationship between the reference person and one of a plurality of other subjects that appear in the collection of images;

obtaining a subject affinity defining a subject affinity score between the reference person and the one of the plurality of other subjects based on the identified social relationship between the reference person and the one of the plurality of other subjects, the plurality of other subjects corresponding to subjects other than the reference person;

determining an image affinity for each of a plurality of images in the collection of images based on the subject affinity for the one of the plurality of other subjects, each image affinity defining an image affinity score between the reference person and a corresponding one of the plurality of images based on the one of the plurality of other subjects appearing in the image;

selecting at least one image from the plurality of images based on the image affinities of the plurality of images, the at least one image comprising the one of the plurality of other subjects; and

effecting presentation of the at least one image on a display of a display device.

2. The method of claim 1, wherein the reference person is a viewer of the display device that has been matched to a subject appearing in at least one image of the collection of images.

3. The method of claim 2, wherein the display device is equipped with a digital camera that is used to capture a viewer image.

4. The method of claim 3, wherein the display device is equipped with a motion detector that is used to trigger the digital camera to capture the viewer image when motion is detected in front of the display device.

5. The method of claim 1, wherein the at least one image is stored at the display device.

6. The method of claim 1, wherein the at least one image is obtained from a central system over the Internet.

7. The method of claim 1, wherein the at least one image is obtained from from a secondary collection sharing device over a local area network.

8. The method of claim 1, wherein the reference person and the one of the plurality of other subjects are members of a social network capable of storing information expressing friend relationships built through an invitation process, and the subject affinity is determined based on social distance, the social distance defining a number of hops in a social graph between the reference person and the one of the plurality of other subjects.

9. The method of claim 8, wherein other subjects with a lower social distance to the reference person are awarded a higher subject affinity.

10. The method of claim 8, wherein the subject affinity is refined based on location history as tracked by an device associated with a other subjects reporting time-stamped GPS locations of the respective device to the social network, with other subjects of closer average geographical proximity to the reference person receiving a higher subject affinity estimate.

11. The method of claim 8, wherein the subject affinity is refined based on relationship qualifiers between the reference person and the one of the plurality of other subjects.

12. The method of claim 8, wherein the subject affinity is refined based on a duration of the friend relationship between the reference person and the one of the plurality of other subjects.

13. The method of claim 8, wherein the subject affinity is refined based on a frequency of communication occurring between the reference person and the one of the plurality of other subjects.

14. The method of claim 8, wherein the subject affinity is refined based on a type of communication occurring between the reference person and the one of the plurality of other subjects.

15. The method of claim 8, wherein the subject affinity is non-symmetric between the reference person and the one of the plurality of other subjects.

16. The method of claim 8, wherein the subject affinity is refined based on friend counts of the reference person and the one of the plurality of other subjects.

17. The method of claim 2, wherein the social relationship is inferred from subject face co-occurrence between the reference person and the other subjects in the collection of images, and wherein the subject affinity is determined based on identifying face co-occurrence instances between the reference person and one of the other subjects appearing in a same image in the collection of images.

18. The method of claim 2, wherein the subject affinity is refined based on a proximity between a face of the reference person and faces of the other subjects in a same image of the collection of images.

19. The method of claim 18, wherein the proximity between a face of the reference person and faces of the other subjects in the same image is determined based on dividing a distance between centers of circles enclosing the face of the reference person and the other subject faces by a diagonal distance of the same image.

20. The method of claim 1, wherein obtaining the information identifying the subject affinity further comprises:

effecting presentation on the display of the display device, information identifying at least one other subject appearing as a subject face in the collection of images;

effecting presentation, in an area adjacent to the information identifying the at least one other subject, a control operable to receive viewer input reflecting the subject affinity between the viewer and the at least one other subject;

receiving user input based on the control; and

based on the user input, adjusting the subject affinity for the at least one other subject.

21. The method of claim 2, wherein the subject affinity is refined based on viewer response to the other subjects in the image of the plurality of images, and modifying the viewers subject affinity for other subjects based on a response of the viewer.

22. The method of claim 2, wherein the image affinity is determined by resolving the subjects occurring in an image of the plurality of images, and accumulating a contribution from the subjects according to the subject affinity between the viewer and the subjects in the image.

23. The method of claim 1, wherein the images of the plurality of images are indexed by subject face before being made available to a display device.

24. The method of claim 1, wherein the image affinity is refined by dividing the image into zones, and modifying an image affinity score contribution of subjects according to the zone in which faces of the subjects appear.

25. The method of claim 1, wherein the image affinity is refined based on determining a size of a subject face in an image of the plurality of images, and modifying an image affinity score contribution of the corresponding subject face according to the size.

26. The method of claim 25, wherein the size of the subject face in the image of the plurality of images is determined based on an area of a circle enclosing the subject face.

27. The method of claim 1, wherein ones of the images in the collection of images contain no subjects.

28. The method of claim 1, wherein ones of the images in the collection of images contain multiple subjects.

29. A device comprising:

a communication interface communicatively coupling the device to a central system; and

a control system comprising at least one processor and a memory storing program codes operable to;

obtain information identifying a reference person;

obtain information identifying a collection of images associated with the reference person;

obtain information identifying a social relationship between the reference person and one of a plurality of other subjects that appear in the collection of images;

obtain a subject affinity defining a subject affinity score between the reference person and the one of the plurality of other subjects based on the identified social relationship between the reference person and the one of the plurality of other subjects, the plurality of other subjects corresponding to subjects other than the reference person;

determine an image affinity for each of a plurality of images in the collection of images based on the subject affinity for the one of the plurality of other subjects, each image affinity defining an image affinity score between the reference person and a corresponding one of the plurality of images based on the one of the plurality of other subjects appearing in the image;

select at least one image from the plurality of images based on the image affinities of the plurality of images, the at least one image comprising the one of the plurality of other subjects; and

effect presentation of the at least one image on a display of a display device.

30. A computer implemented method comprising:

using at least a processor and memory for;

obtaining information identifying a reference person;

obtaining information identifying a plurality of images owned by the reference person;

obtaining a subject affinity for an other subject that appears in the plurality of images, the subject affinity defining a subject affinity score between the reference person the other subject based on the identity of the other subject;

determining an image affinity for each of the plurality of images based on the subject affinity, each image affinity defining an image affinity score between the reference person and a corresponding one of the plurality of images; and

selecting at least one image from the plurality of images based on the image affinities of the plurality of images, the at least one image comprising the other subject and the reference person.