Using affect within a gaming context

• US 9,247,903 B2
• Filed: 02/06/2012
• Issued: 02/02/2016
• Est. Priority Date: 06/07/2010
• Status: Active Grant

• Current Assignee: Affectiva, Inc. (Smart Eye AB)
• Original Assignee: Affectiva, Inc. (Smart Eye AB)
• Inventors: Bender, Daniel, Picard, Rosalind Wright, Sadowsky, Richard Scott, Turcot, Panu James, Wilder-Smith, Oliver Orion, el Kaliouby, Rana
• Primary Examiner(s): Srivastava, Vivek
• Assistant Examiner(s): BUKHARI, SIBTE H
• Application Number:
  US13/366,648
• Publication Number:
  US 20120135804A1
• Time in Patent Office: 1,457 Days
• Field of Search: 709/202, 463/36
• US Class Current: 1/1
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 40/67 for remote operation

Abstract

Mental state data is collected as a person interacts with a game machine. Analysis is performed on this data and mental state information and affect are shared across a social network. The affect of a person can be represented to the social network or gaming community in the form of an avatar. Recommendations can be based on the affect of the person. Mental states can be analyzed by web services which may, in turn, modify the game.

Claims

1. A computer implemented method for gaming comprising:

collecting mental state data, wherein the mental state data includes facial data and physiological data, captured by a webcam, of an individual while the individual is involved in a game;

analyzing, using a web services server, the mental state data to produce mental state information; and

modifying the game based on the mental state information, wherein the modifying the game includes;

changing tasks with which the individual is presented based on a threshold, wherein the threshold comprises a divider in the mental state information that provides for dividing the respondents into those respondents that had higher reactions from those with lower reactions; and

modifying an avatar that represents the individual, wherein the avatar is animated based on the mental state information.

2. The method of claim 1 wherein the collecting mental state data further comprises collecting actigraphy data.

3. The method of claim 1 further comprising inferring mental states with regard to the game based on the mental state data which was collected wherein the mental states include one or more of frustration, confusion, disappointment, hesitation, cognitive overload, focusing, engagement, attention, boredom, exploration, confidence, trust, delight, disgust, skepticism, doubt, satisfaction, excitement, laughter, calmness, stress, and curiosity.

4. The method of claim 1 wherein the game is within an electronic gaming environment.

5. The method of claim 1 wherein the changing tasks includes making the game harder.

6. The method of claim 5 wherein the mental state information includes an indication of boredom.

7. The method of claim 1 wherein the changing tasks includes making the game easier.

8. The method of claim 7 wherein the mental state information includes an indication of frustration.

9. The method of claim 1 wherein the game is a multiplayer game.

10. The method of claim 9 wherein the modifying the game includes modifying an avatar that represents a group of people who are playing the multiplayer game.

11. The method of claim 10 wherein the avatar represents a collective mental state for the group of people.

12. The method of claim 9 wherein the multiplayer game includes requiring players to imitate a face.

13. The method of claim 9 wherein the multiplayer game includes an objective of achieving a collective mental state.

14. The method of claim 9 wherein the multiplayer game includes an objective of avoiding a collective mental state.

15. The method of claim 1 wherein the game includes an objective of achieving a mental state by the individual.

16. The method of claim 1 wherein the game includes an objective of avoiding a mental state by the individual.

17. The method of claim 1 further comprising developing a mental state for a computer generated player.

18. The method of claim 17 wherein the computer generated player is competing against the individual.

19. The method of claim 17 wherein the computer generated player is on a team with the individual.

22. The method of claim 1 wherein the webcam comprises a video camera, a still camera, a thermal imager, a CCD device, a phone camera, or a three-dimensional camera.

23. The method of claim 9 further comprising sharing the mental state information across a social network.

24. The method of claim 23 wherein the sharing is used in rating for games and where the sharing includes information on affect that indicates a level of engagement.

25. The method of claim 23 wherein the sharing is real time and a gaming experience is modified in real time.

26. The method of claim 23 wherein the sharing further comprises sharing modified mental state information which is more positive than inferred mental states as analyzed.

27. The method of claim 9 wherein the mental state information includes mental state information from the other players of the multiplayer game.

28. The method of claim 27 further comprising aggregating the mental state information on a plurality of people who play the multiplayer game.

29. The method of claim 28 wherein the aggregated mental state information is rendered on a display.

20. A computer program product stored on a non-transitory computer-readable medium for gaming, the computer program product comprising:

code for collecting facial data and physiological data using a webcam;

code for collecting mental state data, wherein the mental state data includes the facial data of an individual while the individual is involved in a game;

code for analyzing, using a web services server, the mental state data to produce mental state information; and

code for modifying the game based on the mental state information, wherein the modifying the game includes;

changing tasks with which the individual is presented based on a threshold, wherein the threshold comprises a divider in the mental state information that provides for dividing the respondents into those respondents that had higher reactions from those with lower reactions; and

modifying an avatar that represents the individual, wherein the avatar is animated based on the mental state information.

21. A computer system for gaming comprising:

a memory for storing instructions;

one or more processors attached to the memory wherein the one or more processors are configured to;

collect facial data and physiological data using a webcam;

collect mental state data, wherein the mental state data includes the facial data of an individual while the individual is involved in a game;

analyze, using a web services server, the mental state data to produce mental state information; and

modify the game based on the mental state information;

wherein modification of the game includes;

changing tasks with which the individual is presented based on a threshold, wherein the threshold comprises a divider in the mental state information that provides for dividing the respondents into those respondents that had higher reactions from those with lower reactions; and

modifying an avatar that represents the individual, wherein the avatar is animated based on the mental state information.