Information processing apparatus, information processing method, and program

• US 9,179,191 B2
• Filed: 12/23/2009
• Issued: 11/03/2015
• Est. Priority Date: 12/26/2008
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Fukuchi, Masaki, Yoshigahara, Takayuki
• Primary Examiner(s): FLYNN, RANDY A
• Application Number:
  US12/646,214
• Publication Number:
  US 20100169905A1
• Time in Patent Office: 2,141 Days
• Field of Search: 725/10, 725/12, 725/9, 725 44- 47
• US Class Current: 1/1
• CPC Class Codes: H04H 60/33 Arrangements for monitoring the users' behaviour or opinions
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44213 Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/4668 for recommending content, e.g. movies
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program

Abstract

An information processing apparatus includes a sight line detecting unit configured to detect a sight line of a viewer who is viewing content data and output sight line information which is a result of the detection, a viewing condition estimating unit configured to estimate a viewing condition of the viewer based on the output sight line information and visual attention information relating to the content data, and a content suggesting unit configured to make various suggestions with respect to the content data for the viewer based on the estimated viewing condition.

Claims

1. An information processing apparatus comprising:

a sight line detecting means arranged to detect a sight line of a viewer who is viewing a first content data and output detected sight line information;

a viewing condition estimating means arranged to;

estimate a viewing condition of the viewer by comparing the detected sight line information with visual attention information relating to the first content data, the visual attention information being determined prior to detecting the sight line of the viewer, and denoting a degree of sight line concentration associated with each region of each image included in the first content data; and

estimate a viewing state based on a result of the viewing condition estimation;

a content suggesting means arranged to suggest a second content data for the viewer based on the estimated viewing condition,

wherein the content suggesting means suggests the second content data corresponding to the estimated viewing state; and

a visual attention information updating means configured to update the visual attention information denoting the degree of sight line concentration associated with each region of each image included in the first content data using the detected sight line information.

2. The information processing apparatus according to claim 1,

wherein a viewing condition variable indicating the viewing condition of the viewer is defined and at least one predetermined type of viewing state is associated with each of at least two possible ranges of the viewing condition variable, and

wherein the viewing condition estimating means estimates the viewing condition by calculating a value of the viewing condition variable, and estimates the viewing state by determining a viewing state of a type associated with a range to which the value of the viewing condition variable belongs of the at least two possible ranges.

3. The information processing apparatus according to claim 2,

wherein at least one predetermined type of suggested action is associated with each of at least two types of the viewing states, and

wherein the content suggesting means makes a suggestion for the viewer based on a suggested action of a type associated with the viewing state of a type determined through the viewing condition estimating means.

4. The information processing apparatus according to claim 2,

wherein a score is variable based on a magnitude of the visual attention information about a region where a sight line of the viewer stays, and

wherein the viewing condition estimating means calculates the score based on the detected sight line information output from the sight line detecting means immediately before and the visual attention information relating to the first content data in a cycle of a predetermined time, and calculates a value of a viewing condition variable based on the score obtained on each of at least one occasion, which is obtained by a current period.

5. The information processing apparatus according to claim 1, wherein the visual attention information updating means is further configured to update the visual attention information using sight line information detected by a plurality of sight line detecting means associated with a plurality of viewers who are viewing the same first content data.

6. An information processing method comprising the steps of:

detecting a sight line of a viewer who is viewing a first content data and outputting detected sight line information;

estimating a viewing condition of the viewer by comparing the detected sight line information with visual attention information relating to the first content data, the visual attention information being determined prior to detecting the sight line of the viewer, and denoting a degree of sight line concentration associated with each region of each image included in the first content data;

estimating a viewing state based on a result of the viewing condition estimation;

suggesting a second content data for the viewer based on the estimated viewing condition,

wherein the second content data corresponds to the estimated viewing state; and

updating the visual attention information denoting the degree of sight line concentration associated with each region of each image included in the first content data using the detected sight line information.

7. The information processing method according to claim 6, further comprising:

defining a viewing condition variable indicating the viewing condition of the viewer; and

associating at least one predetermined type of viewing state with each of at least two possible ranges of the viewing condition variable,

wherein estimating the viewing condition further comprises calculating a value of the viewing condition variable, and

wherein estimating the viewing state further comprises determining a viewing state of a type associated with a range to which the value of the viewing condition variable belongs of the at least two possible ranges.

8. The information processing method according to claim 7, further comprising:

associating at least one predetermined type of suggested action with each of at least two types of the viewing states,

wherein suggesting the second content data for the viewer comprises suggesting the second content data based on a suggested action of a type associated with a type of the viewing state.

9. The information processing method according to claim 7,

wherein a score is variable based on a magnitude of the visual attention information about a region where a sight line of the viewer stays, and

the method further comprising:

calculating the score based on the detected sight line information immediately before and the visual attention information relating to the first content data in a cycle of a predetermined time; and

calculating a value of a viewing condition variable based on the score obtained on each of at least one occasion, which is obtained by a current period.

10. The information processing method according to claim 6, wherein updating the visual attention information further comprises updating the visual attention information using sight line information detected from a plurality of viewers who are viewing the same first content data.

11. A non-transitory, computer-readable storage medium storing a program that, when executed by a computer, causes the computer to execute control processing including the steps of:

detecting a sight line of a viewer who is viewing a first content data and outputting detected sight line information;

estimating a viewing condition of the viewer by comparing the detected sight line information with visual attention information relating to the first content data, the visual attention information being determined prior to detecting the sight line of the viewer, and denoting a degree of sight line concentration associated with each region of each image included in the first content data;

estimating a viewing state based on a result of the viewing condition estimation;

suggesting a second content data for the viewer based on the estimated viewing condition,

wherein the second content data corresponds to the estimated viewing state; and

updating the visual attention information denoting the degree of sight line concentration associated with each region of each image included in the first content data using the detected sight line information.

12. The non-transitory, computer-readable storage medium according to claim 11, wherein the control processing further comprises:

defining a viewing condition variable indicating the viewing condition of the viewer; and

associating at least one predetermined type of viewing state with each of at least two possible ranges of the viewing condition variable,

wherein estimating the viewing condition further comprises calculating a value of the viewing condition variable, and

wherein estimating the viewing state further comprises determining a viewing state of a type associated with a range to which the value of the viewing condition variable belongs of the at least two possible ranges.

13. The non-transitory, computer-readable storage medium according to claim 12, further comprising:

associating at least one predetermined type of suggested action with each of at least two types of the viewing states,

wherein suggesting the second content data for the viewer comprises suggesting the second content data based on a suggested action of a type associated with a type of the viewing state.

14. The non-transitory, computer-readable storage medium according to claim 12,

wherein a score is variable based on a magnitude of the visual attention information about a region where a sight line of the viewer stays, and

the method further comprising:

calculating the score based on the detected sight line information immediately before and the visual attention information relating to the first content data in a cycle of a predetermined time; and

calculating a value of a viewing condition variable based on the score obtained on each of at least one occasion, which is obtained by a current period.

15. The non-transitory, computer-readable storage medium according to claim 11, wherein updating the visual attention information further comprises updating the visual attention information using sight line information detected from a plurality of viewers who are viewing the same first content data.

16. An information processing apparatus comprising:

a sight line detecting unit configured to detect a sight line of a viewer who is viewing a first content data and output detected sight line information;

a viewing condition estimating unit configured to;

estimate a viewing condition of the viewer by comparing the detected sight line information with visual attention information relating to the first content data, the visual attention information being determined prior to detecting the sight line of the viewer, and denoting a degree of sight line concentration associated with each region of each image included in the first content data; and

estimate a viewing state based on a result of the viewing condition estimation;

a content suggesting unit configured to suggest a second content data for the viewer based on the estimated viewing condition,

wherein the content suggesting unit suggests the second content data corresponding to the estimated viewing state; and

a visual attention information updating unit configured to update the visual attention information denoting the degree of sight line concentration associated with each region of each image included in the first content data using the detected sight line information.

17. The information processing apparatus according to claim 16 wherein at least one of the first or second content data comprises a program provided to present information related to at least one of character, drawing, color, voice, action, or video of a movie, music, a play, a literature, a photograph, a cartoon, animation, or a computer game.

18. The information processing apparatus according to claim 16,

wherein a viewing condition variable indicating the viewing condition of the viewer is defined and at least one predetermined type of viewing state is associated with each of at least two possible ranges of the viewing condition variable, and

wherein the viewing condition estimating unit estimates the viewing condition by calculating a value of the viewing condition variable, and estimates the viewing state by determining a viewing state of a type associated with a range to which the value of the viewing condition variable belongs of the at least two possible ranges.

19. The information processing apparatus according to claim 18,

wherein at least one predetermined type of suggested action is associated with each of at least two types of the viewing states, and

wherein the content suggesting unit makes a suggestion for the viewer based on a suggested action of a type associated with the viewing state of a type determined through the viewing condition estimating unit.

20. The information processing apparatus according to claim 18,

wherein a score is variable based on a magnitude of the visual attention information about a region where a sight line of the viewer stays, and

wherein the viewing condition estimating unit calculates the score based on the detected sight line information output from the sight line detecting unit immediately before and the visual attention information relating to the first content data in a cycle of a predetermined time, and calculates a value of a viewing condition variable based on the score obtained on each of at least one occasion, which is obtained by a current period.

21. The information processing apparatus according to claim 16, wherein the visual attention information updating unit is further configured to update the visual attention information using sight line information detected by a plurality of sight line detecting units associated with a plurality of viewers who are viewing the same first content data.