LIGHT-EMITTING USER INPUT DEVICE

• US 20180314416A1
• Filed: 04/27/2018
• Published: 11/01/2018
• Est. Priority Date: 04/27/2017
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Powderly, James M., Niles, Savannah, Nesladek, Christopher David, Azu, Isioma Osagbemwenorue, Fontaine, Marshal Ainsworth, Awad, Haney, Wheeler, William, Schwab, Brian David, Bucknor, Brian
• Granted Patent: US 11,112,932 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 2203/04103 Manufacturing, i.e. details related to manufacturing processes specially suited for touch sensitive devices
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0308 comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 : by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/145 : Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: G09G 2320/0626 : for control of overall brightness
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 3/32 : semiconductive, e.g. using light-emitting diodes [LED]
• CPC Class Codes: H04W 4/80 : Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

A light emitting user input device can include a touch sensitive portion configured to accept user input (e.g., from a user'"'"'s thumb) and a light emitting portion configured to output a light pattern. The light pattern can be used to assist the user in interacting with the user input device. Examples include emulating a multi-degree-of-freedom controller, indicating scrolling or swiping actions, indicating presence of objects nearby the device, indicating receipt of notifications, assisting pairing the user input device with another device, or assisting calibrating the user input device. The light emitting user input device can be used to provide user input to a wearable device, such as, e.g., a head mounted display device.

Claims

1. A system comprising:

a light emitting assembly of a user input device, wherein the light emitting assembly is configured to illuminate a plurality of light patterns for providing information of an object in an environment;

a hardware processor communicatively coupled to the light emitting assembly and programmed to;

identify an object in an environment of a user;

access contextual information associated with the object;

determine characteristics of a light pattern to be illuminated by the light emitting assembly based at least partly on the contextual information; and

instruct the light emitting assembly to illuminate in accordance with the light pattern.

2. The system of claim 1, wherein the object comprises at least one of:

a physical object or a virtual object.

3. The system of claim 2, wherein the status of the virtual object comprises at least one of:

a current interaction with the virtual object by the user, whether the virtual object has received new information, whether the virtual object is idle, or whether the virtual object is in an error state.

4. The system of claim 1, wherein the characteristics of the light pattern comprises at least one of:

a brightness, a position, a shape, a size, or a color.

5. The system of claim 1, wherein the contextual information associated with the object comprises at least one of:

a location of the object, an urgency of the object, a type of the object, a property of the object, a volume of information associated with the object, or a preference of the user.

6. The system of claim 1, wherein the system further comprises a wearable display device, and the object is invisible from a user'"'"'s view or is outside via the wearable display device, and the hardware processor is programmed to determine at least one of a size, shape, or color of the light pattern to provide a cue to the user for locating the object.

7. The system of claim 1, wherein object is a component of a wearable system for presenting virtual content to a user, and the light pattern indicates a status of the component of the wearable system.

8. The system of claim 7, wherein the component comprises at least one of:

the user input device, a wearable display device, or a battery pack.

9. The system of claim 1, wherein the status comprises at least one of:

a battery status, a power charging status, a wireless pairing status between the wearable display device and the user input device, a status of a calibration process of the user input device, or a status of the wearable display device.

10. The system of claim 1, wherein the light pattern encodes an alert or information associated with the object.

11. The system of claim 1, wherein the status comprises at least one of:

a current interaction with the object by the user,

whether the object has received new information,

whether the object is idle, or

whether the object is in an error state.

12. The system of claim 1, wherein the characteristics of the light patterns is configurable by a user via an application programming interface.

13. A light-emitting user input device comprising:

a touch component configured to accept a user input;

a light emitting assembly configured to output a plurality of light patterns, the light emitting assembly at least partially surrounding the touch component; and

a hardware processor communicatively coupled to the touch component and the light emitting assembly, and programmed to;

identify a user interface operation supported by the touch component based on contextual information;

determine a first light pattern associated with the user interface operation;

generate and transmit instructions to the light emitting assembly to display a halo having the first light pattern;

receive a user input on the touch component; and

update the halo with a second light pattern to reflect the user input.

14. The light-emitting user input device of claim 13, wherein the contextual information comprises at least one of:

an environment of the user, types of inputs supported by the light-emitting user input device,

information associated with objects that the handheld user input device is configured to interact, or

characteristics of a wearable device associated with the handheld user input device.

15. The light-emitting user input device of claim 13, wherein the light emitting assembly comprises a light guide and a ring of LEDs.

16. The light-emitting user input device of claim 13, wherein the light emitting assembly is configured to accept a plurality of user inputs and wherein the hardware processor is further programmed to display the halo based at least partly on the plurality of user inputs supported by actuating the light guide.

17. The light-emitting user input device of claim 13, wherein the user input comprises at least one of:

a swipe, a tap, a press, or a touch gesture.

18. The light-emitting user input device of claim 13, wherein the light-emitting user input device comprises at least one of a totem, a smartwatch, or a smartphone.

19. The light-emitting user input device of claim 13, wherein the first light pattern provides a cue to a user that the user interface operation is supported by the light-emitting user input device.

20. The light-emitting user input device of claim 13, wherein the hardware processor is further programmed to:

determine whether the received user input is improper based on the contextual information; and

wherein the second light pattern provides a visual alert that the received user input is improper in response to a determination that the received user input is improper.

21. The light-emitting user input device of claim 13, wherein at least a portion of the halo appears to be brighter or larger in the second light pattern as compared to the first light pattern.

22. The light-emitting user input device of claim 13, wherein characteristics of the plurality of light patterns is configurable by a user via an application programming interface.

23. The light-emitting user input device of claim 22, wherein the characteristics comprise at least one of:

a placement or movement pattern, a color, a brightness, a shape, or a size of an arc.

24. The light-emitting user input device of claim 13, wherein the second light pattern indicates, at least one of the following in response to the user input:

a battery status, a power charging status, a wireless pairing status between the handheld user input device and another computing device, or whether the handheld user input device is idle.

25. A method comprising:

under control of a hardware processor;

identifying a type of user input supported by a light-emitting user input device based on contextual information, wherein the light-emitting user input device comprises a first element for illuminating a plurality of light patterns and a second element for receiving user inputs;

determining a first light pattern associated with the type of user input supported;

generating and transmitting instructions to the first element to illuminate a halo having the first light pattern;

determining a second light pattern based on a user input on the light-emitting user input device; and

updating the halo to the second light pattern in response to the user input.

26. The method of claim 25, wherein the first element is further configured to receive another user inputs, and wherein the type of user input for determining the first light pattern is associated with the other user inputs supported by the first element.

27. The method of claim 25, wherein the contextual information comprises at least one of:

an environment of the user, types of inputs supported by the light-emitting user input device,

information associated with objects that the handheld user input device is configured to interact, or

characteristics of a wearable device associated with the handheld user input device.

28. The method of claim 25, wherein the type of user input comprises at least one of:

a swipe, a tap, a press, or a touch input.

29. The method of claim 25, wherein the first light pattern provides a cue to a user that the type of user input is supported by the light-emitting user input device.

30. The method of claim 25, further comprising:

determining whether the user input received by the light-emitting user input device is improper based on the contextual information; and

wherein the second light pattern provides a visual alert that the user input is improper in response to a determination that the user input is improper.

31. The method of claim 25, wherein at least a portion of the halo appears to be brighter or larger in the second light pattern as compared to the first light pattern.

32. A light-emitting user input device comprising:

a plurality of interactable regions configured to receive user inputs, wherein at least one interactable region of the plurality of interactable regions comprises is part of a light emitting assembly of the light-emitting user input device; and

a hardware processor programmed to;

detect a user'"'"'s actuation of the light-emitting user input device;

determine an interactable region among the plurality of interactable regions corresponding to the user'"'"'s actuation;

translate the user'"'"'s actuation into a user input for performing a user interface operation based at least on a type of the actuation and the interaction region associated with the actuation; and

instruct the light emitting assembly to illuminate a light pattern in response to the user input.

33. The light-emitting user input device of claim 32, wherein the light-emitting user input device further comprise a touch surface and the light emitting assembly comprises a light guide, and wherein the plurality of interactable regions comprises a first interactable region associated with the light guide and a second interactable region associated with the touch surface.

34. The light-emitting user input device of claim 33, wherein in response to a detection that the user actuated the first interactable region, the hardware processor is programmed to cause the light emitting assembly to illuminate a first light pattern associated with the user input in the first interactable region, and wherein in response to a detection that the user actuated the second interactable region, the hardware processor is programmed to cause the light emitting assembly to illuminate a second light pattern associated with the user input in the second interactable region.

35. The light-emitting user input device of claim 32, wherein the light pattern is determined based at least partly on contextual information associated with the light-emitting user input device.

36. The light-emitting user input device of claim 32, wherein the user'"'"'s actuation comprises at least one of:

a swipe, a tap, a press, or a touch gesture.

37. The light-emitting user input device of claim 32, wherein the light pattern comprises an arcuate region having a color, an arcuate length, or a visual effect.

38. The light-emitting user input device of claim 32, wherein the hardware processor is further programmed to communicate with a wearable device causing the wearable device to perform a user interface operation based on the user input.

39. A method comprising:

under control of a hardware processor;

detecting a user'"'"'s actuation of a light-emitting user input device, wherein the light-emitting user input device comprises a plurality of interactable regions;

determining an interactable region among the plurality of interactable regions corresponding to the user'"'"'s actuation;

translating the user'"'"'s actuation into a user input for performing a user interface operation based at least on a type of the actuation and the interaction region associated with the actuation; and

causing a light emitting assembly of a light-emitting user input device to illuminate a light pattern in response to the user input.

40. The method of claim 39, wherein the plurality of interactable regions comprises a first interactable region supporting a first type of user input and a second interactable region supporting a second type of user input.

41. The method of claim 40, wherein in response to a detection that the user actuated the first interactable region, the hardware processor is programmed to cause the light emitting assembly to illuminate a first light pattern associated with the user input in the first interactable region, and wherein in response to a detection that the user actuated the second interactable region, the hardware processor is programmed to cause the light emitting assembly to illuminate a second light pattern associated with the user input in the second interactable region.

42. The method of claim 40, wherein at least one of the first type of user input or the second type of user input is dependent on contextual information associated with a user'"'"'s interactions with a wearable system, and wherein the contextual information comprises at least one of a type of application with which a user is interacting, available user inputs supported by the plurality of interactable regions, or a virtual environment of the user.

43. The method of claim 39, wherein the light-emitting user input device comprises a touch surface which is divided into one or more interactable regions.

44. The method of claim 39, wherein the light pattern is determined based at least partly on contextual information associated with the light-emitting user input device.

45. The method of claim 39, wherein the plurality of interactable region comprises the light emitting assembly of the light-emitting user input device.

46. The method of claim 39, wherein the user'"'"'s actuation comprises at least one of:

a swipe, a tap, a press, or a touch gesture.

47. The method of claim 39, wherein the light pattern comprises an arcuate region having a color, an arcuate length, or a visual effect.

48. A light-emitting user input device comprising:

a touchpad assembly configured to receive user inputs, wherein the touchpad assembly comprises;

a touch surface,

a touch sensor coupled to the touch surface, where at least a portion of the touch sensor is underneath the touch surface and is configured to detect an actuation of the touch surface,

a light-emitting assembly comprising an optical diffusive element and a plurality of light illuminating elements, wherein a light-emitting assembly is coupled to the touch sensor and the touch surface and is configured to display a plurality of light patterns, and

a printed circuit board coupled to the light-emitting assembly and the touch sensor; and

a body comprising for supporting the touchpad assembly.

49. The light-emitting user input device of claim 48, wherein at least a portion of the optical diffusive element is overlaid on the touch sensor and the touch sensor is further configured to detect an actuation of the optical diffusive element.

50. The light-emitting user input device of claim 48, wherein the optical diffusive element comprises a light guide.

51. The light-emitting user input device of claim 48, wherein the optical diffusive element substantially surround the touch surface.

52. The light-emitting user input device of claim 48, wherein the touchpad assembly further comprises an armature for holding the light-emitting assembly, the touch surface, and the touch sensor.

53. The light-emitting user input device of claim 48, wherein the plurality of light illuminating elements comprises light emitting diodes.

54. The light-emitting user input device of claim 48, further comprising a connection interface configured to establish a wireless connection with a wearable device configured to present mixed reality content;

55. The light-emitting user input device of claim 48, wherein the body has an upper portion for supporting the touchpad assembly and a bottom portion configured to be removably mounted to a base.

56. The light-emitting user input device of claim 48, wherein the body further comprises at least one of a trigger, bumper, or a home button for user interactions.

57. The light-emitting user input device of claim 48 further comprises a hardware processor programmed to control illumination of the plurality of light illuminating elements.