CONTEXTUAL HELP BASED ON FACIAL RECOGNITION

• US 20120162443A1
• Filed: 12/22/2010
• Published: 06/28/2012
• Est. Priority Date: 12/22/2010
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Allen, Corville O.
• Application Number:
  US12/976,900
• Publication Number:
  US 20120162443A1
• Time in Patent Office: Days
• US Class Current: 348/207.1
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A computer program product includes a computer readable storage medium to store a computer readable program, wherein the computer readable program, when executed on a computer, causes the computer to perform operations for providing contextual help based on a user facial expression. The operations include: capturing a user facial expression using a camera device connected to a computing device; categorizing the user facial expression into a facial expression category; collecting an application context from the computing device in conjunction with an application, wherein the application context includes a recently performed task and a current application state, wherein the current application state comprises information on a current performance of an application in which the user is operating; determining a set of available tasks relating to the application context; and automatically executing one of the set of available tasks based on the facial expression category and the application context.

Claims

1. A computer program product, comprising:

a computer readable storage medium to store a computer readable program, wherein the computer readable program, when executed by a processor within a computer, causes the computer to perform operations for providing contextual help based on a user facial expression, the operations comprising;

capturing the user facial expression using a camera device connected to a computing device;

categorizing the user facial expression into a facial expression category;

collecting an application context from the computing device, wherein the application context comprises a recently performed task and a current application state, wherein the current application state comprises information on a current performance of an application in which the user is operating;

determining a set of available tasks relating to the application context; and

automatically executing one of the set of available tasks based on the facial expression category and the application context.

2. The computer program product of claim 1, wherein the set of available tasks comprises creating a shortcut for the recently performed task.

3. The computer program product of claim 1, wherein the computer program product, when executed on the computer, causes the computer to perform additional operations, comprising:

capturing a plurality of facial expressions;

collecting a plurality of application contexts for at least one application; and

automatically executing one of the set of available tasks based on a combination of the plurality of facial expressions and the plurality of application contexts.

4. The computer program product of claim 1, wherein the computer program product, when executed on the computer, causes the computer to perform additional operations, comprising:

detecting an error in the recently performed task, wherein automatically executing one of the set of available tasks comprises presenting a help display to a user.

5. The computer program product of claim 1, wherein automatically executing one of the set of available tasks is further based on an input location of the recently performed task, wherein the set of available tasks comprises a task with an input location proximate the input location of the recently performed task.

6. The computer program product of claim 1, wherein automatically executing one of the set of available tasks comprises undoing the recently performed task.

7. The computer program product of claim 1, wherein automatically executing one of the set of available tasks comprises determining a subsequent logical task, wherein the set of available tasks comprises high frequency tasks.

8. A method for providing contextual help based on a user facial expression, the method comprising:

capturing the user facial expression using a camera device connected to a computing device;

categorizing the user facial expression into a facial expression category;

collecting an application context from the computing device, wherein the application context comprises a recently performed task and a current application state, wherein the current application state comprises information on a current performance of an application in which the user is operating;

determining a set of available tasks relating to the application context; and

automatically executing one of the set of available tasks based on the facial expression category and the application context.

9. The method of claim 8, wherein the set of available tasks comprises creating a shortcut for the recently performed task.

10. The method of claim 8, further comprising:

capturing a plurality of facial expressions;

collecting a plurality of application contexts for at least one application; and

automatically executing one of the set of available tasks based on a combination of the plurality of facial expressions and the plurality of application contexts.

11. The method of claim 8, further comprising:

detecting an error in the recently performed task, wherein automatically executing one of the set of available tasks comprises presenting a help display to a user.

12. The method of claim 8, wherein automatically executing one of the set of available tasks is further based on an input location of the recently performed task, wherein the set of available tasks comprises a task with an input location proximate the input location of the recently performed task.

13. The method of claim 8, wherein automatically executing one of the set of available tasks comprises undoing the recently performed task.

14. The method of claim 8, wherein automatically executing one of the set of available tasks comprises determining a subsequent logical task, wherein the set of available tasks comprises high frequency tasks.

15. A contextual help system, comprising:

a camera device connected to a computing device to capture a facial expression of a user;

a facial recognition analyzer to categorize the facial expression into a facial expression category;

a context analyzer to collect an application context from the computing device, wherein the application context comprises a recently performed task and a current application state, wherein the current application state comprises information on a current performance of an application in which the user is operating; and

a help interface to determine a set of available tasks relating to the application context and automatically execute one of the set of available tasks based on the facial expression category and the application context.

16. The system of claim 15, wherein the set of available tasks comprises creating a shortcut for the recently performed task.

17. The system of claim 15, wherein the camera device is further configured to capture a plurality of facial expressions, the context analyzer is further configured to collect a plurality of application contexts, and the help interface is further configured to execute one of the set of available tasks based on a combination of the plurality of facial expressions and the plurality of application contexts.

18. The system of claim 15, wherein the help interface is further configured to detect an error in the recently performed task, wherein automatically executing one of the set of available tasks comprises presenting a help display to a user.

19. The system of claim 15, wherein automatically executing one of the set of available tasks is further based on an input location of the recently performed task, wherein the set of available tasks comprises a task with an input location proximate the input location of the recently performed task.

20. The system of claim 15, wherein automatically executing one of the set of available tasks comprises determining a subsequent logical task, wherein the set of available tasks comprises high frequency tasks mapped to at least one facial expression category.