Correlating user reaction with at least an aspect associated with an augmentation of an augmented view

• US 10,254,830 B2
• Filed: 05/02/2017
• Issued: 04/09/2019
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, Jr., John D., Tegreene, Clarence T.
• Primary Examiner(s): Pringle-Parker, Jason A
• Application Number:
  US15/584,333
• Publication Number:
  US 20170315613A1
• Time in Patent Office: 707 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Computationally implemented methods and systems include detecting one or more user reactions of a user in response to a display to the user of an augmented view of an actual scene from a real environment, the augmented view that was displayed including one or more augmentations, and correlating the detected one or more user reactions with at least one or more aspects associated with the one or more augmentations that were included in the augmented view that was presented. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

Claims

1. A computationally-implemented method, comprising:

generating an augmented view including displaying one or more augmentations at a first location relative to an actual scene of a real environment;

detecting a pupil size and dwell path of a user during display of the augmented view of the actual scene of the real environment;

determining low interest in the one or more augmentations at the first location based at least partly on the pupil size and dwell path of the user indicating focus off of the one or more augmentations within a specified time of initial focus; and

moving the one or more augmentations away from the first location to a peripheral location in the augmented view based at least in part on the indication of low interest.

2. The computationally-implemented method of claim 1, further comprising:

detecting one or more physiological characteristics of the user proximate to display of the augmented view.

3. The computationally-implemented method of claim 1, further comprising:

detecting at least one of a facial expression, a head position, or a head tilt of the user proximate to display of the augmented view.

4. The computationally-implemented method of claim 1, further comprising:

correlating one or more physiological characteristics of the user sensed during and/or following display of the augmented view with one or more aspects of the one or more augmentations.

5. The computationally-implemented method of claim 1, further comprising:

correlating at least one of a facial expression, a head position, or a head tilt of the user detected during and/or following display of the augmented view with one or more aspects of the one or more augmentations.

6. The computationally-implemented method of claim 1, further comprising:

correlating one or more user reactions with one or more aspects of one or more non-augmentation scene elements included in the augmented view.

7. The computationally-implemented method of claim 1, further comprising:

correlating one or more user reactions with one or more non-augmentation scene elements that have been obfuscated in the augmented view by the one or more augmentations.

8. The computationally-implemented method of claim 1, further comprising:

generating a second augmented view of a second actual scene at a later time.

9. The computationally-implemented method of claim 8, wherein the second actual scene includes one or more similar environmental conditions as the actual scene.

10. The computationally-implemented method of claim 1, further comprising:

changing a format of the one or more augmentations.

11. The computationally-implemented method of claim 1, further comprising:

detecting one or more anchor visual patterns or items within the actual scene.

12. The computationally-implemented method of claim 1, further comprising:

placing the one or more augmentations in the augmented view along one or more detected dwell paths.

14. The computationally-implemented method of claim 1, wherein the operations are performed using one or more computer processors of an augmented reality device.

15. The computationally-implemented method of claim 1, wherein the generating an augmented view including displaying one or more augmentations at a first location relative to an actual scene of a real environment comprises:

generating an augmented view on at least one of the following devices;

smartphone, tablet, or eyewear.

16. The computationally-implemented method of claim 1, further comprising:

changing a shape or dimension of the one or more augmentations.

17. The computationally-implemented method of claim 1, wherein the determining low interest in the one or more augmentations at the first location based at least partly on the pupil size and dwell path of the user indicating focus off of the one or more augmentations within a specified time of initial focus comprises:

determining low interest in the one or more augmentations at the first location based at least partly on the pupil size and dwell path of the user indicating focus away from the one or more augmentations in less than a specified time.

18. The computationally-implemented method of claim 1, wherein the generating an augmented view including displaying one or more augmentations at a first location relative to an actual scene of a real environment comprises:

generating an augmented view including displaying one or more augmentations at a first central location relative to an actual scene of a real environment.

19. The computationally-implemented method of claim 1, wherein the detecting a pupil size and dwell path of a user during display of the augmented view of the actual scene of the real environment comprises:

detecting a pupil size and eye path of a user during display of the augmented view of the actual scene of the real environment.

20. The computationally-implemented method of claim 1, further comprising:

performing facial recognition with respect to one or more aspects of the real environment.

21. The computationally-implemented method of claim 1, further comprising:

performing voice recognition with respect to one or more aspects of the real environment.

22. The computationally-implemented method of claim 1, further comprising:

obtaining the one or more augmentations from a remote network location.

23. The computationally-implemented method of claim 1, further comprising:

providing output of at least one social networking application.

13. The computationally-implemented method of cam 1, when the moving the one or more augmentations away from the first location to a peripheral location in the augmented view based at least in part on the indication of low interest comprises:

moving the one or more augmentations away from the first location to a corner location in the augmented view based at east in part on the indication of low interest.

24. A computationally-implemented system, comprising:

means for generating an augmented view including displaying one or more augmentations at a first location relative to an actual scene of a real environment means for detecting a pupil size and dwell path of a user during display of the augmented view of the actual scene of the real environment;

means for determining low interest in the one or more augmentations at the first location based at least partly on the pupil size and dwell path of the user indicating focus off of the one or more augmentations within a specified time of initial focus; and

means for moving the one or more augmentations away from the first location to a peripheral location in the augmented view based at least in part on the indication of low interest.

25. A system comprising:

circuitry configured for generating an augmented view including displaying one or more augmentations at a first location relative to an actual scene of a real environment;

circuitry configured for detecting a pupil size and dwell path of a user during display of the augmented view of the actual scene of the real environment;

circuitry configured for determining low interest in the one or more augmentations at the first location based at least partly on the pupil size and dwell path of the user indicating focus off of the one or more augmentations within a specified time of initial focus; and

circuitry configured for moving the one or more augmentations away from the first location to a peripheral location in the augmented view based at least in part on the indication of low interest.