Providing suggested content based on user detected gesture

• US 10,599,915 B2
• Filed: 01/19/2017
• Issued: 03/24/2020
• Est. Priority Date: 01/19/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Goyal, Munish, Leung, Wing L., Rakshit, Sarbajit K., Starks, Kimberly G.
• Primary Examiner(s): Dasgupta, Shourjo
• Application Number:
  US15/409,846
• Publication Number:
  US 20180203947A1
• Time in Patent Office: 1,160 Days
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 9/4451 User profiles; Roaming
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

A computer-implemented method comprises: monitoring, by a user device, user gesture and user input activity by a user of the user device; detecting, by the user device, that the user has entered a thinking state based on the monitoring; determining, by the user device, subject content based on content displayed on a display of the user device based on the detecting; determining, by the user device, relevant content associated with the subject content; and presenting, by the user device, the relevant content to the user via the display.

Claims

1. A computer-implemented method comprising:

monitoring, by a user device, user gesture and user input activity by a user of the user device;

detecting, by the user device, that the user has entered a thinking state based on the monitoring;

determining, by the user device, subject content based on content displayed on a display of the user device based on the detecting;

determining, by the user device, relevant content associated with the subject content; and

presenting, by the user device, the relevant content to the user via the display,

wherein the monitoring comprises receiving data from an accessory device relating to a position and orientation of the accessory device,

the accessory device is a stylus device, and

the position and orientation of the accessory device is in a mouth of the user.

2. The method of claim 1, wherein the monitoring comprises receiving video data from a camera implemented by the user device.

3. The method of claim 1, wherein the detecting comprises detecting that the user activity has paused for a threshold period of time.

4. The method of claim 1, wherein the determining the relevant content is further based on at least one selected from a group consisting of:

user profile information; and

content previously authored by an individual having prior experience with the subject content wherein the individual having the prior experience is identified from at least one selected from a group consisting of;

a social media network; and

a database identifying subject matter experts having the prior experience.

5. The method of claim 4, wherein the user profile information comprises at least one selected from the group consisting of:

user interests;

user expertise information;

user writing style;

historical user written content; and

historical user browsing activity.

6. The method of claim 1, wherein the subject content includes content being viewed by the user.

7. The method of claim 6, further comprising determining a user viewing direction, wherein the content being viewed by the user is determined based on the user viewing direction.

8. The method of claim 1, wherein a service provider at least one of creates, maintains, deploys and supports the user device.

9. The method of claim 1, wherein the monitoring, the detecting, the determining the subject content, the determining the relevant content, and the presenting the relevant content are provided by a service provider on a subscription, advertising, and/or fee basis.

10. The method of claim 1, wherein the user device includes software provided as a service in a cloud environment.

11. The method of claim 1, further comprising deploying a system for detecting that the user has entered the thinking state and providing the relevant content based on the detecting that the user has entered the thinking state, comprising providing a computer infrastructure operable to perform the monitoring, the detecting, the determining the subject content, the determining the relevant content, and the presenting the relevant content.

12. A computer program product for detecting that the user has entered a thinking state while using a user device and providing relevant content, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

monitor user input activity by the user of the user device;

detect that the user has entered a thinking state based on the monitoring;

determine subject content based on content displayed on a display of the user device based on the detecting;

determine relevant content associated with the subject content; and

present the relevant content to the user via the display,

wherein the monitoring comprises receiving data from an accessory device relating to a position and orientation of the accessory device,

the accessory device is a stylus device, and

the position and orientation of the accessory device is in a mouth of the user.

13. The computer program product of claim 12, wherein the monitoring comprises receiving video data from a camera implemented by the user device.

14. The computer program product of claim 12, wherein the user input activity comprises receiving handwritten input to the user device via a stylus device.

15. The computer program product of claim 12, wherein the determining the relevant content is further based on user profile information.

16. A system comprising:

a CPU, a computer readable memory and a computer readable storage medium associated with a user device;

program instructions to monitor user input to the user device via an accessory device;

program instructions to detect a pause in the user input based on the monitoring;

program instructions to provide relevant content to the user via the user device based on the detecting the pause,

wherein the monitoring comprises receiving data from the accessory device relating to a position and orientation of the accessory device,

the accessory device is a stylus device,

the position and orientation of the accessory device is in a mouth of the user, and

the program instructions are stored on the computer readable storage medium for execution by the CPU via the computer readable memory.

17. The system of claim 16, further comprising program instruction to determine the relevant content based on:

the user'"'"'s viewing direction and content presented on a display of the user device corresponding to the user'"'"'s viewing direction; and

user profile information.