Temporal element restoration in augmented reality systems

• US 10,109,075 B2
• Filed: 03/15/2013
• Issued: 10/23/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, Jr., John D., Tegreene, Clarence T.
• Primary Examiner(s): Zhai, Kyle
• Application Number:
  US13/840,767
• Publication Number:
  US 20140267410A1
• Time in Patent Office: 2,048 Days
• Field of Search: None
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06Q 30/0641 Shopping interfaces
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

Methods, apparatuses, computer program products, devices and systems are described that carry out accepting a request associated with at least one of an item, an aspect, or an element that is not present in a field of view of a user'"'"'s augmented reality device; presenting in a display of the augmented reality device at least one augmented reality representation related to the at least one item, aspect, or element in response to accepting a request associated with at least one item, aspect, or element that is not present in a field of view of an augmented reality device; and processing the request and any related interaction of the user via the at least one augmented reality representation.

Claims

1. A system comprising:

circuitry configured for accepting input associated with an element that is moving in a field of view of a device;

circuitry configured for determining that the element will leave the field of view of the device in less than a specified time period based at least partly on a vector of the element; and

circuitry configured for maintaining, on a display of the device at a position corresponding to the input, at least one augmented reality representation of the element responsive to a determination that the element will leave the field of view of the device in less than the specified time period.

2. The system of claim 1, wherein the circuitry configured for accepting input associated with the element that is moving in the field of view of the device comprises:

circuitry configured for accepting at least one of the following types of inputs associated with the element that is moving in the field of view of the device;

touch, gesture, or voice.

3. The system of claim 1, further comprising:

circuitry configured for receiving at least one system state of the device.

4. The system of claim 3, wherein the circuitry configured for receiving the at least one system state of the device comprises:

circuitry configured for receiving the at least one system state of at least one of an application or a process.

5. The system of claim 1, wherein the circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period comprises:

circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation as a substitute for the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period.

6. The system of claim 1, further comprising:

circuitry configured for generating a proxy representation of the element as the at least one augmented reality representation.

7. The system of claim 1, wherein the circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period comprises:

circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element including at least one graphical distinction responsive to the determination that the element will leave the field of view of the device in less than the specified time period.

8. The system of claim 1, wherein the circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period comprises:

circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element with a context specific attribute responsive to the determination that the element will leave the field of view of the device in less than the specified time period.

9. The system of claim 8, wherein the circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element with a context specific attribute responsive to the determination that the element will leave the field of view of the device in less than the specified time period comprises:

circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element with the context specific attribute responsive to the determination that the element will leave the field of view of the device in less than the specified time period, the context specific attribute including one or more of a size, color, or brightness.

10. The system of claim 1, further comprising:

circuitry configured for processing a user interaction with the at least one augmented reality representation including providing a menu of one or more options near the at least one augmented reality representation.

11. The system of claim 1, further comprising:

circuitry configured for processing a user interaction with the at least one augmented reality representation including providing a dialog interface near the at least one augmented reality representation.

12. The system of claim 1, further comprising:

circuitry configured for processing a user interaction with the at least one augmented reality representation including establishing a communication channel in response to the user interaction.

13. The system of claim 1, further comprising:

circuitry configured for releasing the at least one augmented reality representation in response to completion of a user interaction.

14. The system of claim 1, wherein the device includes at least one of an augmented reality device, a personal digital assistant, a personal entertainment device, a mobile phone, a laptop computer, a tablet computer, and/or a desktop computer.

15. The system of claim 1, wherein the circuitry configured for accepting input associated with the element that is moving in the field of view of the device comprises:

circuitry configured for accepting input associated with a person or an object that is moving in the field of view of the device.

16. The system of claim 1, wherein the circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period comprises:

circuitry configured for maintaining, on the display of the device at the position corresponding to the input and prior to the element leaving the field of view, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period.

17. The system of claim 1, further comprising:

circuitry configured for determining the vector of the element within the field of view.

18. The system of claim 1, wherein the circuitry configured for determining that the element will leave the field of view of the device in less than the specified time period based at least partly on the vector of the element comprises:

circuitry configured for determining that the element will leave the field of view of the device in less than the specified time period based at least partly on the vector of the element, the vector including at least a speed and a trajectory of the element.

19. The system of claim 1, wherein the circuitry configured for determining that the element will leave the field of view of the device in less than the specified time period based at least partly on the vector of the element comprises:

circuitry configured for determining that the element will leave the field of view of the device by one or more of moving off a screen or becoming too small for interaction.

20. The system of claim 1, wherein the circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period comprises:

circuitry configured for providing a slower moving augmented reality representation of the element on the display of the device beginning at the position corresponding to the input, responsive to the determination that the element will leave the field of view of the device in less than the specified time period.

21. The system of claim 1, wherein the circuitry configured for accepting input associated with the element that is moving in the field of view of the device comprises:

circuitry configured for accepting input associated with the element that is moving in a camera field of view of the device.

22. The system of claim 1, wherein the circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period comprises:

circuitry configured for locking, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period.

23. The system of claim 1, further comprising:

circuitry configured for animating the augmented reality representation off the display in a direction that the element was previously moving, at least partly in response to a release of the augmented reality representation.

24. The system of claim 1, wherein the circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element responsive to the determination that the element will leave the field of view of the device in less than the specified time period comprises:

circuitry configured for maintaining, on the display of the device at the position corresponding to the input, the at least one augmented reality representation of the element independent of one or more other moving scene elements, responsive to the determination that the element will leave the field of view of the device in less than the specified time period.

25. The system of claim 1, wherein the circuitry configured for determining that the element will leave the field of view of the device in less than the specified time period based at least partly on the vector of the element comprises:

circuitry configured for determining that the element will leave the field of view of the device in less than the specified time period based at least partly on direction of movement of the element.

26. The system of claim 1, wherein the circuitry configured for determining that the element will leave the field of view of the device in less than the specified time period based at least partly on the vector of the element comprises:

circuitry configured for determining that the element will leave the field of view of the device in less than the specified time period based at least partly on speed of movement of the element.

27. The system of claim 1, further comprising:

circuitry configured for providing augmented information that is at least partly based on recognition of the element.

28. A computer-implemented method comprising:

accepting input associated with an element that is moving in a field of view of a device;

determining that the element will leave the field of view of the device in less than a specified time period based at least partly on a vector of the element; and

maintaining, on a display of the device at a position corresponding to the input, at least one augmented reality representation of the element responsive to a determination that the element will leave the field of view of the device in less than the specified time period.

29. A computer processor programmed by one or more instructions that configure the computer processor to execute operations comprising:

accepting input associated with an element that is moving in a field of view of a device;

determining that the element will leave the field of view of the device in less than a specified time period based at least partly on a vector of the element; and

maintaining, on a display of the device at a position corresponding to the input, at least one augmented reality representation of the element responsive to a determination that the element will leave the field of view of the device in less than the specified time period.

30. An augmented reality device comprising:

a display;

at least one computing device; and

one or more instructions that program the at least one computing device to execute operations including at least;

accepting input associated with an element that is moving in a field of view of the augmented reality device;

determining that the element will leave the field of view of the augmented reality device in less than a specified time period based at least partly on a vector of the element; and

maintaining, on the display of the augmented reality device at a position corresponding to the input, at least one augmented reality representation of the element responsive to a determination that the element will leave the field of view of the augmented reality device in less than the specified time period.

31. A system comprising:

means for accepting input associated with an element that is moving in a field of view of a device;

means for determining that the element will leave the field of view of the device in less than a specified time period based at least partly on a vector of the element; and

means for maintaining, on a display of the device at a position corresponding to the input, at least one augmented reality representation of the element responsive to a determination that the element will leave the field of view of the device in less than the specified time period.