Real-time analytics to identify visual objects of interest

• US 9,471,837 B2
• Filed: 10/15/2014
• Issued: 10/18/2016
• Est. Priority Date: 08/19/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Anderson, Brian M., Johnson, Randy S., Katzman, Scott B., Kelley, John F., Schneider, Jacob C., Walton, Kaleb D.
• Primary Examiner(s): Caschera, Antonio A
• Application Number:
  US14/515,125
• Publication Number:
  US 20160055378A1
• Time in Patent Office: 734 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A method of identifying and suggesting objects for a user within a user'"'"'s field of vision of a personal imaging system. A user'"'"'s field of vision within the system is monitored. The personal imaging system captures biometric data of the user regarding objects that are focused on within the user'"'"'s field of vision. The biometric data and any identified emotions regarding the objects being focused on from the user is associated with the user'"'"'s profile which includes at least demographics. A real-time correlation of the user'"'"'s profile and demographics with other users is performed to provide a correlation interest score. The interest score may be used to recommend and direct the user'"'"'s attention to other objects within the user'"'"'s field of vision that the user may be interested in.

Claims

1. A method of identifying and suggesting objects for a user within a user'"'"'s field of vision of a personal imaging system comprising a computer coupled to a plurality of sensors, comprising the steps of:

the computer monitoring the user'"'"'s field of vision of the personal imaging system through the plurality of sensors;

the computer identifies at least one object within the user'"'"'s field of vision of the personal imaging system, the computer;

determining that the user is focusing on an object;

capturing biometric levels of the user regarding at least one identified object in the user'"'"'s field of vision through the personal imaging system;

if the biometric levels of the user exceed threshold levels of biometrics of the user, capturing facial expressions of the user regarding the at least one identified object in the user'"'"'s field of vision through the personal imaging system; and

associating at least the biometric levels from the user with the user'"'"'s profile comprising demographics and storing the feedback in a repository.

2. The method of claim 1, further comprising the steps of:

identifying emotions of the user based on the biometric levels and facial expressions captured regarding the at least one identified object in the user'"'"'s field of vision through the personal imaging system; and

associating at least the received identified emotions and biometric levels from the user with the user'"'"'s profile comprising demographics and storing the feedback in a repository.

3. The method of claim 2, further comprising the steps of:

if the computer determines that information regarding the object is present in the repository, the computer performing a real-time correlation of the user'"'"'s profile, associated emotions from the biometric levels, and associated demographics with other user'"'"'s demographics to provide a correlation interest score; and

if the correlation interest score exceeds a predefined threshold, the computer highlighting the object in the user'"'"'s field of vision through the personal imaging system.

4. The method of claim 3, further comprising the step of displaying characteristics of the identified object to the user in the user'"'"'s field of vision.

5. The method of claim 3, wherein the highlighting of the object in the user'"'"'s field of vision through the personal imaging system indicates whether the object is recommended based on demographics of the user, biometric levels, associated emotions, and correlation interest score of the object.

6. The method of claim 1, prior to the step of the computer monitoring the user'"'"'s field of vision, the method further comprising the steps of:

the computer receiving identification information regarding the user of the personal imaging system;

the computer receiving profile information from the user regarding the user'"'"'s interests;

the computer performing analytics to match the profile information of the user to predefined demographics;

the computer providing demographics to the user for verification;

based on the verification received from the user, storing demographics with the profile and identification of the user in the repository;

the computer obtaining a baseline of biometric data of the user through the personal imaging system;

the computer selecting a biometric threshold level for the user based on the baseline of biometric data received; and

the computer storing the baseline of the biometric data of the user and the selected biometric threshold level for the user with identification of the user and demographics of the user.

7. The method of claim 1, wherein the biometric levels of the user are captured through sensors in communication with the personal imaging system monitoring physiological characteristics of the user.

8. A method of identifying and suggesting objects for a user within a user'"'"'s field of vision of a personal imaging system comprising a computer coupled to a plurality of sensors, comprising the steps of:

the computer monitoring the user'"'"'s field of vision of the personal imaging system through the plurality of sensors;

the computer identifies at least one object within the user'"'"'s field of vision of the personal imaging system, the computer;

capturing biometric levels of the user regarding at least one identified object in the user'"'"'s field of vision through the personal imaging system;

if the biometric levels of the user exceed threshold levels of biometrics of the user, capturing facial expressions of the user regarding the at least one identified object in the user'"'"'s field of vision through the personal imaging system;

identifying emotions of the user based on the biometric levels and facial expressions captured regarding the at least one identified object in the user'"'"'s field of vision through the personal imaging system; and

associating at least the received identified emotions and biometric levels from the user with the user'"'"'s profile comprising demographics and storing the feedback in a repository;

if the computer determines that information regarding the object is present in the repository, the computer performing a real-time correlation of the user'"'"'s profile, associated emotions from the biometric levels, and associated demographics with other user'"'"'s demographics to provide a correlation interest score; and

if the correlation interest score exceeds a predefined threshold, the computer highlighting the object in the user'"'"'s field of vision through the personal imaging system.

9. The method of claim 8, prior to the step of the computer monitoring the user'"'"'s field of vision, the method further comprising the steps of:

the computer receiving identification information regarding the user of the personal imaging system;

the computer receiving profile information from the user regarding the user'"'"'s interests;

the computer performing analytics to match the profile information of the user to predefined demographics;

the computer providing demographics to the user for verification;

based on the verification received from the user, storing demographics with the profile and identification of the user in the repository;

the computer obtaining a baseline of biometric data of the user through the personal imaging system;

the computer selecting a biometric threshold level for the user based on the baseline of biometric data received; and

the computer storing the baseline of the biometric data of the user and the selected biometric threshold level for the user with identification of the user and demographics of the user.

10. The method of claim 8, wherein the highlighting of the object in the user'"'"'s field of vision through the personal imaging system indicates whether the object is recommended based on demographics of the user, biometric levels, associated emotions, and correlation interest score of the object.

11. The method of claim 8, further comprising the step of displaying characteristics of the object identified to the user in the user'"'"'s field of vision.

12. The method of claim 8, wherein the biometric levels of the user are captured through sensors in communication with the personal imaging system monitoring physiological characteristics of the user.

13. A method of identifying and suggesting objects for a user within a user'"'"'s field of vision of a personal imaging system comprising a computer coupled to a plurality of sensors, comprising the steps of:

the computer monitoring the user'"'"'s field of vision of the personal imaging system through the plurality of sensors;

the computer identifies at least one object within the user'"'"'s field of vision of the personal imaging system, the computer;

determining that the user is focusing on an object;

requesting feedback from a user regarding at least one object identified in the user'"'"'s field of vision through the personal imaging system, comprising the steps of;

capturing a user gesture in regards to the at least one object identified in the user'"'"'s field of vision;

capturing biometric levels of the user regarding the at least one identified object in the user'"'"'s field of vision through the personal imaging system;

if the biometric levels of the user exceed threshold levels of biometrics of the user, capturing facial expressions of the user regarding the at least one identified object in the user'"'"'s field of vision through the personal imaging system;

identifying emotions of the user based on the biometric levels and facial expressions captured regarding the at least one identified object in the user'"'"'s field of vision through the personal imaging system; and

associating at least the received identified emotions and biometric levels from the user with the user'"'"'s profile comprising demographics and storing the feedback in a repository; and

associating received feedback from the user with the user'"'"'s profile and associated demographics and storing the feedback in a repository.

14. The method of claim 13, further comprising the steps of:

if the computer determines that information regarding the object is present in the repository, the computer performing a real-time correlation of the user'"'"'s profile, associated emotions from the biometric levels, and associated demographics with other user'"'"'s demographics to provide a correlation interest score; and

if the correlation interest score exceeds a predefined threshold, the computer highlighting the object in the user'"'"'s field of vision through the personal imaging system.

15. The method of claim 14, further comprising the step of displaying characteristics of the identified object to the user in the user'"'"'s field of vision.

16. The method of claim 14, wherein the highlighting of the object in the user'"'"'s field of vision through the personal imaging system indicates whether the object is recommended based on demographics of the user, biometric levels, associated emotions, and correlation interest score of the object.

17. The method of claim 13, prior to the step of the computer monitoring the user'"'"'s field of vision, the method further comprising the steps of:

the computer receiving identification information regarding the user of the personal imaging system;

the computer receiving profile information from the user regarding the user'"'"'s interests;

the computer performing analytics to match the profile information of the user to predefined demographics;

the computer providing demographics to the user for verification;

based on the verification received from the user, storing demographics with the profile and identification of the user in the repository;

the computer obtaining a baseline of biometric data of the user through the personal imaging system;

the computer selecting a biometric threshold level for the user based on the baseline of biometric data received; and

the computer storing the baseline of the biometric data of the user and the selected biometric threshold level for the user with identification of the user and demographics of the user.

18. The method of claim 13, wherein the biometric levels of the user are captured through sensors in communication with the personal imaging system monitoring physiological characteristics of the user.