ECLIPSE CURSOR FOR MIXED REALITY DISPLAYS

• US 20200135141A1
• Filed: 12/19/2019
• Published: 04/30/2020
• Est. Priority Date: 01/30/2018
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Day, John Austin, Pazmino, Lorena, Petty, James Cameron, Hoover, Paul Armistead, Sorrell, Chris, Powderly, James M., Niles, Savannah
• Granted Patent: US 10,885,874 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/08 Cursor circuits
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence

Abstract

Systems and methods for displaying a cursor and a focus indicator associated with real or virtual objects in a virtual, augmented, or mixed reality environment by a wearable display device are disclosed. The system can determine a spatial relationship between a user-movable cursor and a target object within the environment. The system may render a focus indicator (e.g., a halo, shading, or highlighting) around or adjacent objects that are near the cursor. The focus indicator may be emphasized in directions closer to the cursor and deemphasized in directions farther from the cursor. When the cursor overlaps with a target object, the system can render the object in front of the cursor (or not render the cursor at all), so the object is not occluded by the cursor. The cursor and focus indicator can provide the user with positional feedback and help the user navigate among objects in the environment.

Claims

1-40. -40. (canceled)

41. A wearable display system comprising:

a display configured to be positioned in front of an eye of a user, the display configured to project virtual content toward the eye of the user;

a user input device configured to receive user input data associated with movement of a virtual cursor within an environment of the user; and

a hardware processor in communication with the display and the user input device, the hardware processor programmed to;

identify a cursor location of the virtual cursor within the environment;

identify an object location of an object within the environment;

determine a distance between the cursor location and the object location; and

based at least in part on the distance satisfying a distance threshold, cause the display to display a focus indicator associated with the object.

42. The wearable display system of claim 41, wherein the user input device comprises an inertial measurement unit configured to detect a head pose of the user.

43. The wearable display system of claim 41, wherein the user input device comprises an inward-facing imaging system configured to detect an eye pose of the user.

44. The wearable display system of claim 41, wherein the user input device comprises at least one of a handheld totem comprising a touch sensitive surface or an outward-facing imaging system configured to detect a user gesture.

45. The wearable display system of claim 41, wherein the hardware processor is further programmed to determine a relative orientation between the virtual cursor and the object, wherein to cause the display to display the focus indicator, the hardware processor is further programmed to cause the display to display the focus indicator based at least in part on the relative orientation between the virtual cursor and the object.

46. The wearable display system of claim 45, wherein the hardware processor is programmed to direct the display to render the focus indicator preferentially toward the relative orientation of the virtual cursor.

47. The wearable display system of claim 41, wherein the focus indicator comprises at least one of a size change of at least a portion of the object, a depth change of at least a portion of the object, a glow at least partially surrounding the object, a halo at least partially surrounding the object, or graphical highlighting at least partially surrounding the object.

48. The wearable display system of claim 41, wherein to cause the display to display the focus indicator, the hardware processor is programmed to:

perform a plurality of render passes, wherein the plurality of render passes comprises;

a first render pass to render a cursor glow to a first buffer, wherein a location of the cursor glow is based at least in part on the location of the virtual cursor;

a second render pass to render a shape mask representation of the object to a second buffer; and

a third render pass to render a glow mask associated with the object to a third buffer.

49. The wearable display system of claim 41, wherein to cause the display to display the focus indicator, the hardware processor is programmed to:

perform a first render pass to render a cursor glow to a first buffer, a location of the cursor glow based at least in part on the location of the virtual cursor;

perform a second render pass to render a shape mask representation of the object to a second buffer;

perform a third render pass to render a glow mask associated with the object to a third buffer; and

perform a fourth render pass configured to combine at least the first buffer, the second buffer, and the third buffer for presentation to the user.

50. The wearable display system of claim 49, wherein to perform the fourth render pass, the hardware processor is programmed to combine at least the first buffer, the second buffer, and the third buffer with a fourth buffer comprising virtual scene content.

51. The wearable display system of claim 41, wherein the hardware processor is programmed to, based at least in part on the distance satisfying the distance threshold, render the object in front of the virtual cursor.

52. The wearable display system of claim 41, wherein the hardware processor is programmed to, based at least in part on the distance satisfying the distance threshold, cease rendering the virtual cursor.

53. The wearable display system of claim 41, wherein the distance satisfies the distance threshold when the object and the virtual cursor overlap.

54. A method of rendering a focus indicator associated with an object, the method comprising:

under control of a hardware processor;

identifying a cursor location of a virtual cursor within a mixed reality environment associated with a display positioned in front of an eye of a user, the display configured to project virtual content toward the eye of the user;

identifying an object location of an object within the mixed reality environment;

determining a distance between the cursor location and the object location; and

based at least in part on the distance satisfying a distance threshold, causing the display to display a focus indicator associated with the object.

55. The method of claim 54, further comprising:

receiving user input data associated with movement of the virtual cursor via a user input device, wherein the user input device comprises at least one of an inertial measurement unit configured to detect a head pose of the user, an inward-facing imaging system configured to detect an eye pose of the user, a touch sensitive surface configured to detect a user gesture, or an outward-facing imaging system configured to detect the user gesture.

56. The method of claim 54, further comprising:

determining a relative orientation between the virtual cursor and the object, wherein said causing the display to display the focus indicator is further based at least in part on the relative orientation between the virtual cursor and the object.

57. The method of claim 54, wherein the focus indicator comprises at least one of a size change of at least a portion of the object, a depth change of at least a portion of the object, a glow at least partially surrounding the object, a halo at least partially surrounding the object, or graphical highlighting at least partially surrounding the object.

58. The method of claim 54, wherein causing the display to display the focus indicator comprises:

performing a first render pass to render a cursor glow to a first buffer, a location of the cursor glow based at least in part on the location of the virtual cursor;

performing a second render pass to render a shape mask representation of the object to a second buffer;

performing a third render pass to render a glow mask associated with the object to a third buffer; and

performing a fourth render pass configured to combine at least the first buffer, the second buffer, and the third buffer for presentation to the user.

59. The method of claim 54, further comprising rendering the object in front of the virtual cursor based at least in part on the distance satisfying the distance threshold.

60. The method of claim 54, further comprising ceasing to render the virtual cursor based at least in part on the distance satisfying the distance threshold.