Extramissive spatial imaging digital eye glass apparatuses, methods, and systems for virtual or augmediated vision, manipulation, creation, or interaction with objects, materials, or other entities

• US 10,540,014 B2
• Filed: 10/10/2018
• Issued: 01/21/2020
• Est. Priority Date: 01/03/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Meta View, Inc.
• Inventors: Gribetz, Meron, Mann, W. Steve G.
• Primary Examiner(s): Sasinowski, Andrew
• Application Number:
  US16/156,798
• Publication Number:
  US 20190278379A1
• Time in Patent Office: 468 Days
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A sensing and display apparatus, comprising: a first phenomenon interface configured to operatively interface with a first augmediated-reality space, and a second phenomenon interface configured to operatively interface with a second augmediated-reality space, is implemented as an extramissive spatial imaging digital eye glass.

Claims

1. An apparatus, comprising:

at least one spatial imaging device configured to sense a real-world environment of the apparatus;

at least one head-mounted output display;

at least one computational device communicatively coupled to the at least one spatial imaging device and at least one head mounted output display, wherein the at least one computational device is configured to execute program instructions that cause the at least one computational device to;

determine a spatial coordinate system corresponding to data generated by the at least one spatial imaging device from the real-world environment;

process data generated by the at least one imaging device to determine spatial coordinates corresponding to movement of a real-world object within the real-world environment indicating a location of the real-world object within the spatial coordinate system;

generate an interactive, virtual N-dimensional manifold corresponding to a collection of points derived from the spatial coordinates corresponding to the real-world object, the interactive, virtual N-dimensional manifold having one or more associated virtual coordinates mapped by the spatial coordinate system indicating a user interactive space within the spatial coordinate system; and

display one or more virtual images or objects by the at least one head-mounted output display corresponding to the virtual coordinates associated with the interactive, virtual N-dimensional manifold.

2. The apparatus of claim 1, wherein the interactive, virtual N-dimensional manifold is generated from synthesizing a time-integral of the determined spatial coordinates associated with movement of the real world object through the spatial coordinate system.

3. The apparatus of claim 1, wherein the display of the one or more virtual images are perceived by a user of the apparatus as a visual indication of a location of the interactive, virtual N-dimensional manifold in a virtual or augmented reality field of view of the user of the apparatus.

4. The apparatus of claim 1, wherein the at least one computational device is configured to execute further program instructions that cause the at least one computational device to:

process data generated by the at least one imaging device to determine spatial coordinates corresponding to a user interaction with the interactive, virtual N-dimensional manifold;

determine a particular user interaction from the associated virtual coordinates and the determined spatial coordinates corresponding to the user interaction with the interactive, virtual N-dimensional manifold; and

display one or more virtual images or objects by the at least one head-mounted output display in response to the determined user interaction.

5. The apparatus of claim 4, wherein the interactive, virtual N-dimensional manifold is an interactive virtual user interface configured to receive user input and generate an interaction with the apparatus.

6. The apparatus of claim 4, wherein the determined user interaction edits, shapes, or sculpts the virtual coordinates associated with the interactive virtual N-dimensional manifold.

7. The apparatus of claim 1, wherein the at least one computational device is configured to execute further program instructions that cause the at least one computational device to:

process data generated by the at least one imaging device to determine spatial coordinates corresponding to a user interaction with the interactive, virtual N-dimensional manifold;

determine a particular user interaction from the associated virtual coordinates and the determined spatial coordinates corresponding to the user interaction using a dimensional hierarchy of user interaction with the interactive, virtual N-dimensional manifold; and

display one or more virtual images or objects by the at least one head-mounted output display in response to the determined user interaction.

8. The apparatus of claim 1, wherein the interactive, virtual N-dimensional manifold is an interactive virtual user interface configured to receive user input and generate an interaction with the apparatus.

9. The apparatus of claim 1, wherein the interactive, virtual N-dimensional manifold is one of a 1-dimensional manifold, 2-dimensional manifold, and 3-dimensional manifold.

10. The apparatus of claim 1, further comprising one or more communication interfaces, wherein the at least one computational device is configured to execute further program instructions that cause the at least one computational device to:

transmit, via at least one communications interface, data used generate the interactive, virtual N-dimensional manifold corresponding to the collection of points derived from the spatial coordinates.

11. The apparatus of claim 1, further comprising one or more communication interfaces wherein the at least one computational device is configured to execute further program instructions that cause the at least one computational device to:

receive, via at least one communication interface, data indicating input of user interaction with the interactive, virtual N-dimensional manifold by another apparatus, and

display one or more virtual images or objects by the at least one head-mounted output display in response to the received data.

12. The apparatus of claim 1, further comprising one or more communication interfaces wherein the at least one computational device is configured to execute further program instructions that cause the at least one computational device to:

receive, via at least one communication interface, data indicating an interactive, virtual N-dimensional manifold corresponding to a collection of points derived from spatial coordinates determined by another apparatus, and

display one or more virtual images or objects by the at least one head-mounted output display in response to the received data.

13. The apparatus of claim 12, wherein the at least one computational device is configured to execute further program instructions that cause the at least one computational device to:

process data generated by the at least one imaging device to determine spatial coordinates corresponding to a user interaction with the interactive, virtual N-dimensional manifold from another apparatus;

determine a particular user interaction from the associated virtual coordinates and the determined spatial coordinates corresponding to the user interaction using a dimensional hierarchy of user interaction with the interactive, virtual N-dimensional manifold from another apparatus; and

display one or more virtual images or objects by the at least one head-mounted output display in response to the determined user interaction.

14. The apparatus of claim 1, wherein the virtual spatial coordinates are mapped to the real-world object and at least one of the one or more virtual images or objects when displayed is perceived by the user as appearing on or in place of the real-world object.

15. The apparatus of claim 14, wherein the real-world object is one of a wand, a stylus, a physical component located in the environment, and a personal computer including:

a cell phone, a smart phone, a camera phone, or a tablet computer.

16. The apparatus of claim 1, wherein the real-world object is a user body part.

17. An apparatus, comprising:

at least one spatial imaging device;

at least one head-mounted output display;

at least one computational device communicatively coupled to the at least one spatial imaging device and at least one head mounted output display, wherein the at least one computational device is configured to execute program instructions that cause the at least one computational device to;

determine a spatial coordinate system corresponding to data generated by the at least one spatial imaging device;

determine one or more interactive, virtual N-dimensional manifolds, each manifold having one or more associated virtual coordinates mapped by the spatial coordinate system;

process data generated by the at least one imaging device to determine spatial coordinates corresponding to a user interaction with at least one virtual N-dimensional manifold;

determine a particular user interaction from the associated virtual coordinates and the determined spatial coordinates using a dimensional hierarchy associated with possible user interaction with the at least one N-dimensional manifold; and

display one or more virtual images or objects by the at least one head-mounted output display in response to the determined user interaction.

18. The apparatus of claim 17, wherein the at least one computational device is configured to execute further program instructions that cause the at least one computational device to:

generate a first taction signal associated with a first dimension from the determined spatial coordinates; and

generate a second taction signal associated with a different dimension from the determined spatial coordinates,

wherein determining a user interaction includes determining a level of taction based on the first and second taction signals.

19. The apparatus of claim 18, wherein the one or more interactive, virtual N-dimensional manifolds include a plurality of manifolds, and determining a user interaction includes selecting one of the plurality of manifolds based on the determined level of taction.

20. The apparatus of claim 17, wherein at least one virtual image or object is associated with the at least one interactive, virtual N-dimensional manifold and the at least on virtual image is displayed based on the one or more associated virtual coordinates of the at least one interactive, virtual N-dimensional manifold.

21. The apparatus of claim 17, wherein the shape or configuration of the at least one interactive, virtual N-dimensional manifold is edited or sculpted based on the determined user interaction.

22. The apparatus of claim 17, wherein determining a particular user interaction includes determining a user gesture from the determined spatial coordinates and wherein a same user gesture results in one of two different user interactions based on the dimensional hierarchy of user interaction with the at least one interactive, virtual N-dimensional manifold.

23. The apparatus of claim 17, further comprising one or more communication interfaces wherein the at least one computational device is configured to execute further program instructions that cause the at least one computational device to:

receive, via at least one communication interface, data indicating input of user interaction with the interactive, virtual N-dimensional manifold by another apparatus, and

display one or more virtual images or objects by the at least one head-mounted output display in response to the received data.