Avatar-based augmented reality engagement

• US 10,585,470 B2
• Filed: 04/07/2017
• Issued: 03/10/2020
• Est. Priority Date: 04/07/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bastide, Paul R., Broomhall, Matthew E., Loredo, Robert E., Srinivasan, Sathyanarayanan
• Primary Examiner(s): Lhymn, Sarah
• Application Number:
  US15/481,915
• Publication Number:
  US 20180292887A1
• Time in Patent Office: 1,068 Days
• Field of Search: 345619, 345632, 345633
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

A method, computer system, and computer program product for determining an engagement level of an individual is provided. The present invention may include capturing a plurality of image data depicting a relative location of a user. The present invention may also include identifying an individual within the captured image data. The present invention may further include gathering a plurality of engagement level indicator data associated with the identified individual. The present invention may also include calculating a current engagement level of the identified individual using the plurality of gathered engagement level indicator data.

Claims

1. A processor-implemented method for determining an engagement level of an individual, the method comprising:

detecting, by a processor, a user intent to engage with another individual based on a user interaction with an augmented reality device;

receiving a user-designated topic;

capturing, by a computer, a plurality of image data depicting a relative location of a user;

identifying an individual within the captured image data using facial recognition, a social media location check-in, and a plurality of livestreaming information;

generating a profile for each identified individual;

storing the generated profile in a repository;

gathering a plurality of engagement level indicator data associated with the identified individual, wherein the plurality of engagement level indicator data associated with the identified individual comprises of a plurality of physical attribute information and a plurality of social media information, and wherein the plurality of physical attribute comprises a plurality of facial expressions, a posture of the identified individual, a laughter volume of the identified individual, a plurality of emoted speech of the identified individual, an eye elongation of the identified individual, and an iris elongation of the identified individual;

calculating a current engagement level of the identified individual using the plurality of gathered engagement level indicator data and a variability of conversation of the identified individual with one or more other individuals, wherein a variability of conversation is categorical or physical, and wherein a categorical variability of conversation establishes a percentage chance with which a conversation varies from the user-designated topic, and wherein a physical variability of conversation weighs a plurality of physical criteria to determine a likelihood the user will enter into a conversation with the identified individual; and

generating a relative engagement model showing an engagement of the identified individual compared to one or more individuals within the captured plurality of image data using the calculated current engagement level, wherein the generated relative engagement model is an analytical model that shows an engagement of the identified individual compared to one or more other individuals within the plurality of captured image data; and

modifying a device display using an avatar corresponding to the calculated current engagement level, wherein the avatar is unique to the calculated current engagement level based on an openness of the identified individual to engage with the user.

2. The method of claim 1, further comprising:

detecting a user intent to engage with another individual based on a user interaction with an augmented reality device.

3. The method of claim 2, wherein the user interaction is selected from a group consisting of a plurality of audible phrases into a microphone, a button push, an activation of a menu, a plurality of user gestures captured by an image capturing device associated with the augmented reality device, and a user attention to a specific individual.

4. The method of claim 1, wherein the current engagement level is represented as a value on a sliding scale.

5. A computer system for determining an engagement level of an individual, the computer system comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising;

detecting a user intent to engage with another individual based on a user interaction with an augmented reality device;

receiving a user-designated topic;

capturing, by a computer, a plurality of image data depicting a relative location of a user;

identifying an individual within the captured image data using facial recognition, a social media location check-in, and a plurality of livestreaming information;

generating a profile for each identified individual;

storing the generated profile in a repository;

gathering a plurality of engagement level indicator data associated with the identified individual, wherein the plurality of engagement level indicator data associated with the identified individual comprises of a plurality of physical attribute information and a plurality of social media information, and wherein the plurality of physical attribute comprises a plurality of facial expressions, a posture of the identified individual, a laughter volume of the identified individual, a plurality of emoted speech of the identified individual, an eye elongation of the identified individual, and an iris elongation of the identified individual;

calculating a current engagement level of the identified individual using the plurality of gathered engagement level indicator data and a variability of conversation of the identified individual with one or more other individuals, wherein a variability of conversation is categorical or physical, and wherein a categorical variability of conversation establishes a percentage chance with which a conversation varies from the user-designated topic, and wherein a physical variability of conversation weighs a plurality of physical criteria to determine a likelihood the user will enter into a conversation with the identified individual; and

generating a relative engagement model showing an engagement of the identified individual compared to one or more individuals within the captured plurality of image data using the calculated current engagement level, wherein the generated relative engagement model is an analytical model that shows an engagement of the identified individual compared to one or more other individuals within the plurality of captured image data; and

modifying a device display using an avatar corresponding to the calculated current engagement level, wherein the avatar is unique to the calculated current engagement level based on an openness of the identified individual to engage with the user.

6. The computer system of claim 5, further comprising:

detecting a user intent to engage with another individual based on a user interaction with an augmented reality device.

7. The computer system of claim 6, wherein the user interaction is selected from a group consisting of a plurality of audible phrases into a microphone, a button push, an activation of a menu, a plurality of user gestures captured by an image capturing device associated with the augmented reality device, and a user attention to a specific individual.

8. The computer system of claim 5, wherein the current engagement level is represented as a value on a sliding scale.

9. A computer program product for determining an engagement level of an individual, the computer program product comprising:

one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor, the program instructions comprising;

program instruction to detect a user intent to engage with another individual based on a user interaction with an augmented reality device;

program instructions to receive a user-designated topic;

program instructions to capture a plurality of image data depicting a relative location of a user;

program instructions to identify an individual within the captured image data using facial recognition, a social media location check-in, and a plurality of livestreaming information;

program instruction to generate a profile for each identified individual;

program instructions to store the generated profile in a repository;

program instructions to gather a plurality of engagement level indicator data associated with the identified individual, wherein the plurality of engagement level indicator data associated with the identified individual comprises of a plurality of physical attribute information and a plurality of social media information, and wherein the plurality of physical attribute comprises a plurality of facial expressions, a posture of the identified individual, a laughter volume of the identified individual, a plurality of emoted speech of the identified individual, an eye elongation of the identified individual, and an iris elongation of the identified individual;

program instructions to calculate a current engagement level of the identified individual using the plurality of gathered engagement level indicator data and a variability of conversation of the identified individual with one or more other individuals, wherein a variability of conversation is categorical or physical, and wherein a categorical variability of conversation establishes a percentage chance with which a conversation varies from the user-designated topic, and wherein a physical variability of conversation weighs a plurality of physical criteria to determine a likelihood the user will enter into a conversation with the identified individual; and

program instructions to generate a relative engagement model showing an engagement of the identified individual compared to one or more individuals within the captured plurality of image data using the calculated current engagement level, wherein the generated relative engagement model is an analytical model that shows an engagement of the identified individual compared to one or more other individuals within the plurality of captured image data; and

program instructions to modify a device display using an avatar corresponding to the calculated current engagement level, wherein the avatar is unique to the calculated current engagement level based on an openness of the identified individual to engage with the user.

10. The computer program product of claim 9, further comprising:

program instructions to detect a user intent to engage with another individual based on a user interaction with an augmented reality device.

11. The computer program product of claim 9, wherein the current engagement level is represented as a value on a sliding scale.