Dynamically preserving scene elements in augmented reality systems

• US 9,639,964 B2
• Filed: 03/15/2013
• Issued: 05/02/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, Jr., John D., Tegreene, Clarence T.
• Primary Examiner(s): Zhai, Kyle
• Application Number:
  US13/840,742
• Publication Number:
  US 20140267409A1
• Time in Patent Office: 1,509 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

Methods, apparatuses, computer program products, devices and systems are described that carry out accepting a user request associated with at least one of an item, an aspect, or an element of a field of view of an augmented reality device; determining that a first presentation of the at least one item, aspect, or element has a limited period of viability for user interaction relative to the field of view of the augmented reality device; and at least one of maintaining the first presentation or providing a substantially similar second presentation in response to determining that a first presentation of the at least one item, aspect, or element has a limited period of viability for interaction relative to the field of view of the augmented reality device.

Claims

1. A system comprising:

circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

circuitry configured for determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by;

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will remain within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will remain within the field of view of the augmented reality device to a threshold time interval; and

circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

2. The system of claim 1, wherein the circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device comprises:

circuitry configured for accepting at least one of touch input, finger pointing input, voice input, hand gesture input, finger gesture input, combined touch input of a touch sensitive portion and pointing input, or eye movement input.

3. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving a displayed indicia of at least one of a person, an object, an animal, a sign, a building, a vehicle, a display, a perspective, or a condition on the augmented reality device.

4. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device based at least partly on at least one of a user task, a system task, a context, a known user interest, or a known user preference.

5. The system of claim 1, wherein the circuitry configured for determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for determining that a moving object has a limited period of visibility within the field of view of the augmented reality device.

6. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for freezing a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

7. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving at least one displayed simulated representation of the real-world object.

8. The system of claim 1, further comprising:

circuitry configured for performing image recognition to identify the real-world object based on at least of;

a context of the augmented reality device, a device in proximity to the augmented reality device, an operation running on the augmented reality device, a task associated with the augmented reality device, a known user interest, or a known user preference.

9. The system of claim 1, further comprising:

circuitry configured for performing image recognition to identify the real-world object; and

circuitry configured for displaying at least one menu item or command available for the real-world object responsive to recognition of the real world object.

10. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for highlighting a displayed view of the real-world object.

11. The system of claim 1, wherein the circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device comprises:

circuitry configured for detecting a user action at a location within a field of view of the augmented reality device corresponding to the real-world object.

12. The system of claim 1, further comprising:

circuitry configured for dragging the indicia of the real-world object within a displayed view of the field of view of the augmented reality device responsive to a user action.

13. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving at least one of a displayed image, cartoon, frozen scene portion, or simulation of the real-world object on the augmented reality device.

14. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving a displayed indicia of the real-world object on at least one of a smartphone, tablet, camera, augmented reality eyeglasses, or smartwatch.

15. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving a displayed indicia of the real-world object at a location within a displayed view of the field of view of the augmented reality device corresponding to at least one of a touch input, finger pointing input, voice input, hand gesture input, finger gesture input, or eye movement input.

16. The system of claim 1, wherein the circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device comprises:

circuitry configured for preserving at least one of a photo, symbol, pixel, or drop destination corresponding to the real-world object.

17. The system of claim 1, further comprising:

circuitry configured for determining the displayed indicia of the real-world object based at least partly on a geographic location of the augmented reality device.

18. The system of claim 1, further comprising:

circuitry configured for performing at least one of the following specified operations associated with the displayed indicia of the real-world object;

playing media, implementing a check-out, queuing media, posting to a social media site, completing a transaction, collecting an item, presenting content for viewing, or enabling game play.

19. The system of claim 1, further comprising:

circuitry configured for transferring data associated with the real-world object to a computing device in the field of view of the augmented reality device at least partly in response to movement of the displayed indicia of the real-world object to a position that corresponds to the computing device in the field of view of the augmented reality device.

20. The system of claim 1, wherein the determining a time interval that the real-world object will remain within the field of view of the augmented reality device comprises:

determining at least one of a speed, acceleration, or trajectory of the real-world object relative to the field of view of the augmented reality device.

21. The system of claim 1, wherein the determining a time interval that the real-world object will remain within the field of view of the augmented reality device comprises:

determining a time interval until the real-world object will move behind an obstacle.

25. The system of claim 1, wherein the circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device comprises:

circuitry configured for detecting, with a camera of the augmented reality device, at least one of a hand gesture, a finger gesture or an eye movement corresponding to the real-world object.

26. The system of claim 1, further comprising:

circuitry configured for releasing the displayed indicia of the real-world object such that displayed indicia of the real-world object transitions through animation to a live view.

22. A computer-implemented method comprising:

receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by;

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will remain within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will remain within the field of view of the augmented reality device to a threshold time interval; and

preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

23. A system comprising:

an augmented reality device including at least;

at least one sensor operable to at least detect one or more user inputs;

a display operable to display an augmented reality scene that includes one or more real-world objects and one or more augmented reality representations; and

one or more wireless communication units operable to transmit or receive one or more wireless communications; and

one or more instructions on at least one media that when executed on a computing device associated with the augmented reality device program the computing device for;

receiving, via the augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by;

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will remain within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will remain within the field of view of the augmented reality device to a threshold time interval; and

preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

24. A system comprising:

means for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

means for determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by;

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will remain within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will remain within the field of view of the augmented reality device to a threshold time interval; and

means for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.

27. A system comprising:

circuitry configured for receiving, via an augmented reality device, a user input indicating a request to interact with a real-world object within a field of view of the augmented reality device;

circuitry configured for determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device by;

detecting movement of the real-world object relative to the field of view of the augmented reality device;

determining a time interval that the real-world object will have at least a threshold viewable size within the field of view of the augmented reality device; and

comparing the time interval that the real-world object will have at least the threshold viewable size within the field of view of the augmented reality device to a threshold time interval; and

circuitry configured for preserving a displayed indicia of the real-world object on the augmented reality device responsive to the determining that the real-world object has a limited period of visibility within the field of view of the augmented reality device.