Providing subject information regarding upcoming images on a display

• US 10,084,964 B1
• Filed: 07/25/2016
• Issued: 09/25/2018
• Est. Priority Date: 02/17/2009
• Status: Active Grant

• Current Assignee: Ikorongo Technology, LLC (Concert Technology Corporation)
• Original Assignee: Ikorongo Technology, LLC (Concert Technology Corporation)
• Inventors: Svendsen, Hugh Blake
• Primary Examiner(s): Vu, Toan
• Application Number:
  US15/218,222
• Time in Patent Office: 792 Days
• Field of Search: None
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/179 metadata assisted face recognition
• CPC Class Codes: G11B 27/19 by using information detectable on the record carrier
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H04N 9/8715 : involving the mixing of the reproduced video signal with a non-recorded signal, e.g. a text signal

Abstract

Methods are described for presenting in an user interface information regarding the subject faces that will appear in upcoming images. In general, many of the images available to display will be comprised of images containing subject faces. Based on a subject affinity score between the viewer and the subjects, an image affinity is computed. Based on the image affinity scores, images are selected for presentation on the display. As images are displayed, the system analyzes one or more upcoming images to determine information to display. As each image is displayed, subject information comprising the subject'"'"'s face is presented in an area of the display adjacent to the current image.

Claims

1. A device comprising:

electronic capture circuitry operable to;

capture information identifying a viewer; and

a control system comprising a processor and a memory coupled to the electronic capture circuitry, the memory storing program codes operable to instruct the processor to;

obtain an image affinity for each image in a plurality of images based on a subject affinity for a plurality of other subjects, the image affinity defining an image affinity score between the viewer and each image in the plurality of images based on the other subjects appearing in the plurality of images, the subject affinity defining a subject affinity score between the viewer and a corresponding one of the plurality of other subjects, the other subjects corresponding to subjects other than the viewer;

select, by the device, based on the image affinity for each of the plurality of images a sequence of images from the plurality of images;

select, by the device, one or more images from the sequence of images that are to be presented subsequent to a current image to form one or more analysis window images; and

determine updated subject information from the one or more analysis window images, the updated subject information comprising subject representations of those subjects appearing in the one or more analysis window images; and

electronic video circuitry, coupled to the control system, operable to;

effect presentation of the current image from the sequence of images in a first area of a display, wherein only the current image from the sequence of images is presented during a presentation period; and

effect presentation of the updated subject information in a second area of the display.

2. The device of claim 1 further comprising:

the control system further operable to;

determine, based on timing information, that it is time to present a next image from the sequence of images;

select, by the device, the next image in the sequence of images as a new current image; and

effect presentation of the new current image.

3. The device of claim 1 where in order to detect the viewer in proximity to the display, further comprising:

motion sensing electronic circuitry, coupled to the control system, operable to;

detect a presence of the viewer in proximity to the device.

4. The device of claim 3 wherein the subject representations comprise likenesses of those subjects appearing in the one or more analysis window images.

5. The device of claim 1 where in order to effect the presentation of the current image, further comprising:

the control system further operable to;

adjust a time duration the current image is presented according to its image affinity.

6. The device of claim 1 further comprising:

the control system further operable to;

sort the subject representations by their corresponding subject affinity; and

the electronic video circuitry further operable to;

effect presentation of subject status indicator information in the second area of the display.

7. The device of claim 1 wherein the viewer and the plurality of other subjects are members of a social network capable of storing information expressing social relationships in a social graph built through an invitation process, the device further comprising:

the control system further operable to;

adjust the subject affinity based on a social distance, the social distance defining a number of hops in the social graph between the viewer and each of the plurality of other subjects, wherein a lower social distance between the viewer and an other subject is indicative of a closer relationship and a higher subject affinity.

8. The device of claim 1 wherein the viewer and the plurality of other subjects appear in the plurality of images, and wherein a social relationship is inferred from subject face co-occurrence between the viewer and the plurality of other subjects in the plurality of images based on identifying face co-occurrence instances between the viewer and the plurality of other subjects appearing in a same image in the plurality of images, wherein a higher number of co-occurrences between the viewer and an other subject is indicative of a closer social relationship and a higher subject affinity.

9. The device of claim 1 further comprising:

the electronic video circuitry further operable to;

effect presentation on the display, information identifying an other subject appearing as a subject face in the plurality of images; and

effect presentation, in an area adjacent to the information identifying the other subject, a control operable to receive viewer input reflecting the subject affinity between the viewer and the other subject; and

the control system further operable to;

receive user input based on the control; and

based on the user input, adjust the subject affinity for the other subject.

10. The device of claim 1 wherein the device is one of a digital picture frame, cellular phone, personal digital assistant (PDA), and a media player.

11. The device of claim 1 wherein the analysis window images are not shown on the display contemporaneously with the current image.

12. The device of claim 1, wherein the updated subject information comprises next subject indicator information, further comprising:

the control system further operable to;

determine next subject indicator information for the subject representations, the next subject indicator information indicating which subjects will appear in the next image from the sequence of images; and

the electronic video circuitry further operable to;

effect the presentation of the next subject indicator information in the second area of the display.

13. A device comprising:

electronic capture circuitry operable to;

capture information identifying a viewer; and

a control system comprising a processor and a memory coupled to the electronic capture circuitry, the memory storing program codes operable to instruct the processor to;

select, by the device, a sequence of images based on the viewer;

select, by the device, one or more images from the sequence of images that are to be presented subsequent to a current image to form one or more analysis window images;

determine updated subject information from the one or more analysis window images, the updated subject information comprising;

subject representations of those subjects appearing in the one or more analysis window images, and

subject occurrence counts, each subject occurrence count defining a number of times a subject corresponding to a subject representation appears in the one or more analysis window images; and

sort the subject representations by their corresponding subject occurrence counts; and

electronic video circuitry, coupled to the control system, operable to;

effect presentation of the current image from the sequence of images in a first area of a display, wherein only the current image from the sequence of images is presented during a presentation period; and

effect presentation of the updated subject information in a second area of the display.

14. The device of claim 13, wherein the updated subject information comprises next subject indicator information, further comprising:

the control system further operable to;

determine next subject indicator information for the subject representations, the next subject indicator information indicating which subjects will appear in the next image from the sequence of images; and

the electronic video circuitry further operable to;

effect the presentation of the next subject indicator information in the second area of the display.

15. The device of claim 13 wherein the subject representations comprise likenesses of those subjects appearing in the one or more analysis window images.

16. A device comprising:

electronic capture circuitry operable to;

capture information identifying a viewer; and

a control system comprising a processor and a memory coupled to the electronic capture circuitry, the memory storing program codes operable to instruct the processor to;

select, by the device, a sequence of images based on the viewer;

select, by the device, one or more images from the sequence of images that are to be presented subsequent to a current image to form one or more analysis window images;

determine updated subject information from the one or more analysis window images, the updated subject information comprising;

subject representations of those subjects appearing in the one or more analysis window images, and

image countdown counts, each image countdown count defining a number of images to be presented before a subject corresponding to a subject representation appears in the current image; and

sort the subject representations by their corresponding image countdown counts; and

electronic video circuitry, coupled to the control system, operable to;

effect presentation of the current image from the sequence of images in a first area of a display, wherein only the current image from the sequence of images is presented during a presentation period; and

effect presentation of the updated subject information in a second area of the display.

17. The device of claim 16, wherein the updated subject information comprises next subject indicator information, further comprising:

the control system further operable to;

determine next subject indicator information for the subject representations, the next subject indicator information indicating which subjects will appear in the next image from the sequence of images; and

the electronic video circuitry further operable to;

effect the presentation of the next subject indicator information in the second area of the display.

18. The device of claim 16 wherein the subject representations comprise likenesses of those subjects appearing in the one or more analysis window images.

19. A device comprising:

electronic capture circuitry operable to;

capture information identifying a viewer; and

a control system comprising a processor and a memory coupled to the electronic capture circuitry, the memory storing program codes operable to instruct the processor to;

select, by the device, a sequence of images based on the viewer;

select, by the device, one or more images from the sequence of images that are to be presented subsequent to a current image to form one or more analysis window images;

determine updated subject information from the one or more analysis window images, the updated subject information comprising;

subject representations of those subjects appearing in the one or more analysis window images, and

time countdown counts, each time countdown count defining a time duration before a subject corresponding to a subject representation appears in the current image; and

sort the subject representations by their corresponding time countdown counts; and

electronic video circuitry, coupled to the control system, operable to;

effect presentation of the current image from the sequence of images in a first area of a display, wherein only the current image from the sequence of images is presented during a presentation period; and

effect presentation of the updated subject information in a second area of the display.

20. The device of claim 19, wherein the updated subject information comprises next subject indicator information, further comprising:

the control system further operable to;

determine next subject indicator information for the subject representations, the next subject indicator information indicating which subjects will appear in the next image from the sequence of images; and

the electronic video circuitry further operable to;

effect the presentation of the next subject indicator information in the second area of the display.

21. The device of claim 19 wherein the subject representations comprise likenesses of those subjects appearing in the one or more analysis window images.

22. A device comprising:

electronic capture circuitry operable to;

capture information identifying a viewer; and

a control system comprising a processor and a memory coupled to the electronic capture circuitry, the memory storing program codes operable to instruct the processor to;

select, by the device, a sequence of images based on the viewer;

select, by the device, one or more images from the sequence of images that are to be presented subsequent to a current image to form one or more analysis window images;

determine updated subject information from the one or more analysis window images, the updated subject information comprising;

subject representations of those subjects appearing in the one or more analysis window images, and

determine subject prominence scores, each subject prominence score determined as a function of a face size ratio value and a zone value for a subject corresponding to a subject representation in the one or more analysis window images, wherein the subject prominence scores are determined based on one or more analysis window images; and

sort the subject representations by their corresponding subject prominence scores; and

electronic video circuitry, coupled to the control system, operable to;

effect presentation of the current image from the sequence of images in a first area of a display, wherein only the current image from the sequence of images is presented during a presentation period; and

effect presentation of the updated subject information in a second area of the display.

23. The device of claim 22, wherein the updated subject information comprises next subject indicator information, further comprising:

the control system further operable to;

determine next subject indicator information for the subject representations, the next subject indicator information indicating which subjects will appear in the next image from the sequence of images; and

the electronic video circuitry further operable to;

effect the presentation of the next subject indicator information in the second area of the display.

24. The device of claim 22 wherein the subject representations comprise likenesses of those subjects appearing in the one or more analysis window images.