Technique for more efficiently displaying text in virtual image generation system

• US 9,978,182 B2
• Filed: 06/30/2016
• Issued: 05/22/2018
• Est. Priority Date: 06/30/2015
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Yeoh, Ivan Li Chuen, Miller, Samuel A., Hand, Randall E., Edwin, Lionel Ernest, O'Connor, Philip, Schwab, Brian David, Lindsay, Spencer
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): Lhymn, Sarah
• Application Number:
  US15/198,983
• Publication Number:
  US 20170032575A1
• Time in Patent Office: 691 Days
• Field of Search: 345619, 345632, 345633
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A virtual image generation system and method of operating same is provided. An end user is allowed to visualize the object of interest in a three-dimensional scene. A text region is spatially associated with the object of interest. A textual message that identifies at least one characteristic of the object of interest is generated. A textual message is streamed within the text region.

Claims

1. A method of operating a virtual image generation system, the method comprising:

allowing an end user to visualize an object of interest in a three-dimensional scene;

spatially associating a text region within a field of view of the user, wherein the text region is spatially associated with the object of interest;

generating a gesture reference associated with the object of interest;

generating a textual message that identifies at least one characteristic of the object of interest;

streaming the textual message within the text region;

sensing gestural commands from the end user by detecting an angular position of an anatomical part of the end user relative to a plurality of different regions of the gesture reference; and

controlling the streaming of the textual message in response to the sensed gestural commands, wherein the gesture reference is an annular ring surrounding the object of interest, and

wherein a first side of the annular ring forms one of the different regions, and

a second side of the annular ring opposite of the first side of the annular ring forms another one of the different regions.

2. The method of claim 1, further comprising displaying the gesture reference as a gesture reference object adjacent the object of interest.

3. The method of claim 1, wherein the anatomical part of the end user is a head of the end user.

4. The method of claim 3, wherein the anatomical part of the end user is a finger or hand of the end user.

5. The method of claim 1, wherein the gesture reference is separate and distinct from the object of interest.

6. The method of claim 1, wherein the annular ring comprises a plurality of concentric rings, and an interface between two adjacent ones of the concentric rings forms one of the different regions.

7. The method of claim 6, wherein an inner or outer edge of the annular ring forms another one of the different regions.

8. The method of claim 1, wherein the gesture reference is the object of interest.

9. The method of claim 1, wherein the streaming of the textual message is controlled in response to the sensed gestural commands by initiating the streaming of the textual message when the anatomical part of the end user is pointed at the one region of the gesture reference, and terminating the streaming of the textual message when the anatomical part of the end user is pointed at another different region of the gesture reference.

10. The method of claim 1, wherein the streaming of the textual message is controlled in response to the sensed gestural commands by displaying at least one word of the textual message when the anatomical part of the end user is pointed at the one region of the gesture reference, and displaying at least another word of the textual message when the anatomical part of the end user is pointed at another different region of the gesture reference.

11. The method of claim 1, wherein the gestural commands are sensed from the end user as the anatomical part of the end user is scanned across the gesture reference.

12. The method of claim 1, wherein the one or more sensors are configured for sending the gestural commands from the end user as the anatomical part of the end user is scanned across the gesture reference.

13. A virtual image generation system for use by an end user, comprising:

a display system configured for allowing the end user to visualize an object of interest in a three dimensional scene;

a control system configured for spatially associating a text region with a field of view of the end user, wherein the text region is spatially associated with the object of interest,

generating a gesture reference associated with the object of interest,

generating a textual message that identifies at least one characteristic of the object of interest, and

instructing the display system to stream the textual message within the text region; and

one or more sensors configured for sensing gestural commands from the end user by detecting an angular position of an anatomical part of the end user relative to a plurality of different regions of the gesture reference, wherein the control system is further configured for controlling the streaming of the textual message in response to the sensed gestural commands,

wherein the gesture reference is an annular ring surrounding the object of interest, and

wherein a first side of the annular ring forms one of the different regions, and

a second side of the annular ring opposite of the first side of the annular ring forms another one of the different regions.

14. The virtual image generation system of claim 13, wherein the control system is further configured for instructing the display system to display the gesture reference as a gesture reference object adjacent the object of interest.

15. The virtual image generation system of claim 13, wherein the anatomical part of the end user is a head of the end user.

16. The virtual image generation system of claim 15, wherein the anatomical part of the end user is a finger or hand of the end user.

17. The virtual image generation system of claim 13, wherein the gesture reference is separate and distinct from the object of interest.

18. The virtual image generation system of claim 13, wherein the annular ring comprises a plurality of concentric rings, and an interface between two adjacent ones of the concentric rings forms one of the different regions.

19. The virtual image generation system of claim 18, wherein an inner or outer edge of the annular ring forms another one of the different regions.

20. The virtual image generation system of claim 19, wherein the gesture reference is the object of interest.

21. The virtual image generation system of claim 13, wherein the control system is configured for controlling the streaming of the textual message in

response to the sensed gestural commands by instructing the display system to initiate the streaming of the textual message when the anatomical part of the end user is pointed at the one region of the gesture reference, and terminate the streaming of the textual message when the anatomical part of the end user is pointed at another different region of the gesture reference.

22. The virtual image generation system of claim 13, wherein the control system is configured for controlling the streaming of the textual message in response to the sensed gestural commands by instructing the display system to display at least one word of the textual message when the anatomical part of the end user is pointed at the one region of the gesture reference, and display at least another word of the textual message when the anatomical part of the end user is pointed at another different region of the gesture reference.