Displaying in response to detecting one or more user behaviors one or more second augmentations that are based on one or more registered first augmentations

• US 10,665,017 B2
• Filed: 02/19/2013
• Issued: 05/26/2020
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, Jr., John D., Tegreene, Clarence T.
• Primary Examiner(s): Chen, Yu
• Application Number:
  US13/770,053
• Publication Number:
  US 20140098137A1
• Time in Patent Office: 2,653 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Computationally implemented methods and systems include registering one or more first augmentations that were shown to a user through a first augmented view of a first actual scene, the one or more first augmentations having been shown at least at end of a segment of time, detecting, following the showing of the one or more first augmentations up to the end of the segment of time, one or more user behaviors of the user that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations; and displaying, in response at least in part to said detecting, one or more second augmentations through a second augmented view of the first actual scene or of a second actual scene, the one or more second augmentations to be displayed being based, at least in part, on the registering of the one or more first augmentations. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

Claims

1. A system, comprising:

circuitry for registering one or more first augmentations shown to a user through a first augmented view of a first actual scene;

circuitry for detecting one or more user behaviors of the user that, when detected, infer at least some user interest in seeing the one or more first augmentations;

circuitry for correlating the detected one or more user behaviors with one or more aspects associated with the one or more first augmentations;

circuitry for transmitting one or more results of the correlating;

circuitry for receiving, in response to the transmitting, one or more second augmentations for displaying in a second augmented view of at least one of the first actual scene or a second actual scene;

circuitry for displaying the one or more second augmentations through the second augmented view of the at least one of the first actual scene or the second actual scene, the one or more second augmentations including at least one aspect directing the user to relocate to one or more physical locations associated with the first actual scene in order to review the one or more first augmentations; and

circuitry for displaying one or more third augmentations to aid the user in reviewing an aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, the displaying occurring upon detection of the user having relocated, responsive to the one or more second augmentations, to the one or more physical locations associated with the first actual scene in order to review the one or more first augmentations.

2. The system of claim 1, wherein circuitry for registering one or more first augmentations shown to a user through a first augmented view of a first actual scene comprises:

circuitry for registering one or more aspects of the one or more first augmentations.

3. The system of claim 1, wherein circuitry for registering one or more first augmentations shown to a user through a first augmented view of a first actual scene comprises:

circuitry for registering one or more dynamic augmentations that change in response to one or more physical movements of the user detected by one or more locational sensors.

4. The system of claim 3, wherein circuitry for registering one or more dynamic augmentations that change in response to one or more physical movements of the user detected by one or more locational sensors comprises:

circuitry for registering one or more dynamic augmentations that change in response to one or more physical movements of the user detected by one or more locational sensors which determine the one or more physical movements of the user via one or more cellular tower signals.

5. The system of claim 1, wherein circuitry for registering one or more first augmentations shown to a user through a first augmented view of a first actual scene comprises:

circuitry for registering one or more dynamic augmentations that change in response to one or more atmospheric conditions including at least one dynamic augmentation that provides at least one indication of current air temperature including at least an image of a mercury level of a thermometer moving at least one of up or down in accordance with one or more temperature sensor readings.

6. The system of claim 1, wherein circuitry for detecting one or more user behaviors of the user that, when detected, infer at least some user interest in seeing the one or more first augmentations comprises:

circuitry for sensing one or more physiological characteristics of the user proximate to the one or more first augmentations being shown to the user.

7. The system of claim 6, wherein circuitry for sensing one or more physiological characteristics of the user proximate to the one or more first augmentations being shown to the user comprises:

circuitry for sensing, via one or more wristband-embedded sensors configured for physiological characteristic detection, the one or more physiological characteristics of the user at least one of during or following the one or more first augmentations being shown to the user via a head-mounted display worn by the user in wireless communication with a wristband worn by the user that includes the one or more wristband-embedded sensors.

8. The system of claim 6, wherein circuitry for sensing one or more physiological characteristics of the user proximate to the one or more first augmentations being shown to the user comprises:

circuitry for sensing at least one of one or more ocular characteristics, one or more galvanic skin responses, one or more blood pressure measurements, or one or more pulse rate measurements of the user proximate to the one or more first augmentations being shown to the user.

9. The system of claim 1, wherein circuitry for displaying the one or more second augmentations through the second augmented view of the at least one of the first actual scene or the second actual scene, the one or more second augmentations including at least one aspect directing the user to relocate to one or more physical locations associated with the first actual scene in order to review the one or more first augmentations comprises:

circuitry for displaying the one or more second augmentations, including at least displaying at least one of one or more symbolic augmentations or one or more textual augmentations associated with directing the user to physically move in a cardinal direction in order to relocate to the one or more physical locations associated with the first actual scene in order to observe for a second time an aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred.

10. The system of claim 1, wherein circuitry for displaying one or more third augmentations to aid the user in reviewing an aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, the displaying occurring upon detection of the user having relocated, responsive to the one or more second augmentations, to the one or more physical locations associated with the first actual scene in order to review the one or more first augmentations comprises:

circuitry for receiving an indication of the user having relocated to the one or more physical locations associated with the first actual scene; and

circuitry for displaying the one or more third augmentations.

11. The system of claim 1, wherein circuitry for displaying one or more third augmentations to aid the user in reviewing an aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, the displaying occurring upon detection of the user having relocated, responsive to the one or more second augmentations, to the one or more physical locations associated with the first actual scene in order to review the one or more first augmentations comprises:

circuitry for displaying a bull'"'"'s eye that directs the user to look at the aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, the bull'"'"'s eye incorporated in the one or more third augmentations.

12. The system of claim 1, wherein circuitry for displaying one or more third augmentations to aid the user in reviewing an aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, the displaying occurring upon detection of the user having relocated, responsive to the one or more second augmentations, to the one or more physical locations associated with the first actual scene in order to review the one or more first augmentations comprises:

circuitry for displaying at least one of one or more symbolic augmentations or one or more textual augmentations associated with directing the user to visually focus for a second time at the aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, including at least increasing a size of the aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred to aid the user in observing for a second time the aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred.

13. The system of claim 1, wherein circuitry for displaying one or more third augmentations to aid the user in reviewing an aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, the displaying occurring upon detection of the user having relocated, responsive to the one or more second augmentations, to the one or more physical locations associated with the first actual scene in order to review the one or more first augmentations comprises:

circuitry for displaying at least one of one or more symbolic augmentations or one or more textual augmentations associated with directing the user to visually focus in one or more visual directions in order to observe for a second time the aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred.

14. The system of claim 1, wherein circuitry for displaying the one or more second augmentations through the second augmented view of the at least one of the first actual scene or the second actual scene, the one or more second augmentations including at least one aspect directing the user to relocate to one or more physical locations associated with the first actual scene in order to review the one or more first augmentations comprises:

circuitry for displaying a modified version of the one or more first augmentations as the one or more second augmentations.

15. The system of claim 14, wherein circuitry for displaying a modified version of the one or more first augmentations as the one or more second augmentations comprises:

circuitry for displaying a modified version of the one or more first augmentations including at least one modified visual format relative to the one or more first augmentations.

16. The system of claim 1, wherein circuitry for displaying the one or more second augmentations through the second augmented view of the at least one of the first actual scene or the second actual scene, the one or more second augmentations including at least one aspect directing the user to relocate to one or more physical locations associated with the first actual scene in order to review the one or more first augmentations comprises:

circuitry for displaying one or more second augmentations, including one or more augmentations which dynamically change based, at least in part, on the circuitry for detecting the one or more user behaviors of the user, through a second augmented view of at least one of the first actual scene or a second actual scene.

17. The system of claim 16, wherein circuitry for displaying one or more second augmentations, including one or more augmentations which dynamically change based, at least in part, on the circuitry for detecting the one or more user behaviors of the user, through a second augmented view of at least one of the first actual scene or a second actual scene comprises:

circuitry for displaying one or more second augmentations, including one or more augmentations which dynamically change based on at least one user heart rate detected by the circuitry for detecting the one or more user behaviors of the user, through a second augmented view of at least one of the first actual scene or a second actual scene.

18. The system of claim 1, wherein circuitry for displaying the one or more second augmentations through the second augmented view of the at least one of the first actual scene or the second actual scene, the one or more second augmentations including at least one aspect directing the user to relocate to one or more physical locations associated with the first actual scene in order to review the one or more first augmentations comprises:

circuitry for determining a purpose of the user based, at least in part, on the circuitry for detecting the one or more user behaviors of the user; and

circuitry for displaying one or more second augmentations through a second augmented view of at least one of the first actual scene or a second actual scene based, at least in part, on the circuitry for determining the purpose of the user.

19. The system of claim 18, wherein circuitry for determining a purpose of the user based, at least in part, on the circuitry for detecting the one or more user behaviors of the user comprises:

circuitry for determining that a user is riding a bicycle on a bike trail for exercise purposes rather than for leisure purposes based, at least in part, on the circuitry for detecting the one or more user behaviors of the user, the bike trail having been used by the user in the past for leisure purposes.

20. The system of claim 19, wherein circuitry for determining that a user is riding a bicycle on a bike trail for exercise purposes rather than for leisure purposes based, at least in part, on the circuitry for detecting the one or more user behaviors of the user, the bike trail having been used by the user in the past for leisure purposes comprises:

circuitry for determining that a user is riding a bicycle for exercise purposes rather than for leisure purposes based, at least in part, on at least one elevated heart rate of the user detected by the circuitry for detecting the one or more user behaviors of the user compared to at least one heart rate detected while the user was using the bike trail in the past for leisure purposes.

21. The system of claim 1, wherein circuitry for detecting one or more user behaviors of the user that, when detected, infer at least some user interest in seeing the one or more first augmentations comprises:

circuitry for detecting one or more user behaviors of the user, the circuitry including at least one blood pressure sensor integrated with at least one augmented reality device on which the one or more first augmentations are shown to the user.

22. The system of claim 21, wherein circuitry for detecting one or more user behaviors of the user, the circuitry including at least one blood pressure sensor integrated with at least one augmented reality device on which the one or more first augmentations are shown to the user comprises:

circuitry for detecting one or more user physiological characteristics as the one or more user behaviors of the user that, when detected, infer one or more of at least one user mental state or at least one user emotional state in response to seeing the one or more first augmentations, including at least detecting, via the at least one blood pressure sensor integrated with at least one augmented reality device on which the one or more first augmentations are shown to the user, at least one change in blood pressure indicative of the one or more of at least one user mental state or at least one user emotional state in response to seeing the one or more first augmentations.

23. The system of claim 22, wherein circuitry for detecting one or more user physiological characteristics as the one or more user behaviors of the user that, when detected, infer one or more of at least one user mental state or at least one user emotional state in response to seeing the one or more first augmentations, including at least detecting, via the at least one blood pressure sensor integrated with at least one augmented reality device on which the one or more first augmentations are shown to the user, at least one change in blood pressure indicative of the one or more of at least one user mental state or at least one user emotional state in response to seeing the one or more first augmentations comprises:

circuitry for detecting, via the at least one blood pressure sensor integrated with at least one augmented reality device on which the one or more first augmentations are shown to the user, at least one change in blood pressure indicative of at least one user mood in response to seeing the one or more first augmentations.

24. The system of claim 1, wherein circuitry for correlating the detected one or more user behaviors with one or more aspects associated with the one or more first augmentations comprises:

circuitry for correlating one or more user physiological characteristics detected as the one or more user behaviors of the user with the one or more aspects associated with the one or more first augmentations.

25. The system of claim 24, wherein circuitry for correlating one or more user physiological characteristics detected as the one or more user behaviors of the user with the one or more aspects associated with the one or more first augmentations comprises:

circuitry for correlating one or more user physiological characteristics sensed at least one of during or following the one or more first augmentations being shown to the user with the one or more aspects associated with the one or more first augmentations.

26. The system of claim 24, wherein circuitry for correlating one or more user physiological characteristics detected as the one or more user behaviors of the user with the one or more aspects associated with the one or more first augmentations comprises:

circuitry for correlating at least one head tilt of the user sensed at least one of during or following the one or more first augmentations being shown to the user with the one or more aspects associated with the one or more first augmentations.

27. The system of claim 26, wherein circuitry for correlating at least one head tilt of the user sensed at least one of during or following the one or more first augmentations being shown to the user with the one or more aspects associated with the one or more first augmentations comprises:

circuitry for correlating at least one head tilt of the user sensed via at least one tilt sensor integrated with the at least one augmented reality device on which the one or more first augmentations are shown to the user at least one of during or following the one or more first augmentations being shown to the user with the one or more aspects associated with the one or more first augmentations.

28. The system of claim 27, wherein circuitry for correlating at least one head tilt of the user sensed via at least one tilt sensor integrated with the at least one augmented reality device on which the one or more first augmentations are shown to the user at least one of during or following the one or more first augmentations being shown to the user with the one or more aspects associated with the one or more first augmentations comprises:

circuitry for inferring which of the one or more aspects associated with the one or more first augmentations caused the at least one head tilt of the user based, at least in part, on at least one orientation of a head of the user relative to the one or more aspects associated with the one or more first augmentations at least one of during or following the one or more first augmentations being shown to the user.

29. The system of claim 1, wherein circuitry for correlating the detected one or more user behaviors with one or more aspects associated with the one or more first augmentations comprises:

circuitry for correlating the detected one or more user behaviors with one or more non-augmentation scene elements that have been obfuscated in the first augmented view by the one or more first augmentations.

30. The system of claim 1, wherein circuitry for transmitting one or more results of the correlating comprises:

circuitry for transmitting the one or more results of the correlating to one or more network servers associated with one or more independent parties.

31. The system of claim 1, wherein circuitry for transmitting one or more results of the correlating comprises:

circuitry for transmitting the one or more results of the correlating by transmitting one or more defined relationships between the detected one or more user behaviors and the one or more aspects associated with the one or more first augmentations.

32. The system of claim 1, wherein circuitry for receiving, in response to the transmitting, one or more second augmentations for displaying in a second augmented view of at least one of the first actual scene or a second actual scene comprises:

circuitry for receiving one or more second augmentations having one or more formats that were selected based, at least in part, on the one or more results of the correlating that were transmitted.

33. The system of claim 32, wherein circuitry for receiving one or more second augmentations having one or more formats that were selected based, at least in part, on the one or more results of the correlating that were transmitted comprises:

circuitry for receiving one or more second augmentations that are designed to be placed at one or more particular locations in the second augmented view, the placement or placements of the one or more second augmentations in the second augmented view being based, at least in part, on the one or more results of the correlating that were transmitted.

34. The system of claim 32, wherein circuitry for receiving one or more second augmentations having one or more formats that were selected based, at least in part, on the one or more results of the correlating that were transmitted comprises:

circuitry for receiving one or more second augmentations that are designed to have one or more particular dimensions or shapes, the one or more particular dimensions or shapes of the one or more second augmentations based, at least in part, on the one or more results of the correlating that were transmitted.

35. The system of claim 1, wherein circuitry for receiving, in response to the transmitting, one or more second augmentations for displaying in a second augmented view of at least one of the first actual scene or a second actual scene comprises:

circuitry for receiving one or more visibility rules for displaying the one or more second augmentations that are based, at least in part, on the one or more results of the correlating that were transmitted.

36. A system, comprising:

at least one processing device; and

one or more instructions that, when executed by the at least one processing device, configure the at least one processing device to perform one or more operations including at least;

registering one or more first augmentations shown to a user through a first augmented view of a first actual scene;

detecting one or more user behaviors of the user that, when detected, infer at least some user interest in seeing the one or more first augmentations;

correlating the detected one or more user behaviors with one or more aspects associated with the one or more first augmentations;

transmitting one or more results of the correlating;

receiving, in response to the transmitting, one or more second augmentations for displaying in a second augmented view of at least one of the first actual scene or a second actual scene;

displaying the one or more second augmentations through the second augmented view of the at least one of the first actual scene or the second actual scene, the one or more second augmentations including at least one aspect directing the user to relocate to one or more physical locations associated with the first actual scene in order to review the one or more first augmentations; and

displaying one or more third augmentations to aid the user in reviewing an aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, the displaying occurring upon detection of the user having relocated, responsive to the one or more second augmentations, to the one or more physical locations associated with the first actual scene in order to review the one or more first augmentations.

37. A method, comprising:

registering one or more first augmentations shown to a user through a first augmented view of a first actual scene;

detecting one or more user behaviors of the user that, when detected, infer at least some user interest in seeing the one or more first augmentations;

correlating the detected one or more user behaviors with one or more aspects associated with the one or more first augmentations;

transmitting one or more results of the correlating;

receiving, in response to the transmitting, one or more second augmentations for displaying in a second augmented view of at least one of the first actual scene or a second actual scene;

displaying the one or more second augmentations through the second augmented view of the at least one of the first actual scene or the second actual scene, the one or more second augmentations including at least one aspect directing the user to relocate to one or more physical locations associated with the first actual scene in order to review the one or more first augmentations; and

displaying one or more third augmentations to aid the user in reviewing an aspect of the first augmented view of the first actual scene in which the at least some user interest was inferred, the displaying occurring upon detection of the user having relocated, responsive to the one or more second augmentations, to the one or more physical locations associated with the first actual scene in order to review the one or more first augmentations,

wherein at least one of the registering, detecting, correlating, transmitting, receiving, or displaying is at least partially implemented using at least one processing device.