External user interface for head worn computing

• US 9,939,934 B2
• Filed: 01/17/2014
• Issued: 04/10/2018
• Est. Priority Date: 01/17/2014
• Status: Active Grant

• Current Assignee: Mentor Acquisition One, LLC (Magic Leap, Inc.)
• Original Assignee: Osterhout Design Group
• Inventors: Nortrup, Edward H., Border, John N., Osterhout, Ralph F.
• Primary Examiner(s): Yang, Nan-Ying
• Application Number:
  US14/158,198
• Publication Number:
  US 20150205385A1
• Time in Patent Office: 1,544 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G04G 17/08 Housings
• CPC Class Codes: G04G 17/083 Watches distributed over several housings
• CPC Class Codes: G04G 21/00 Input or output devices integrated in time-pieces
• CPC Class Codes: G04G 21/025 for measuring physiological data
• CPC Class Codes: G04G 21/08 Touch switches specially adapted for time-pieces
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0386 for light pen
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1454 : involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 2370/06 : Consumer Electronics Control, i.e. control of another device by a display or vice versa
• CPC Class Codes: G09G 2370/22 : Detection of presence or absence of input display information or of connection or disconnection of a corresponding information source
• CPC Class Codes: G09G 5/12 : Synchronisation between the display unit and other units, e.g. other display units, video-disc players

Abstract

Aspects of the present invention relate to external user interfaces used in connection with head worn computers (HWC). Embodiments relate to an external user interface that has a physical form intended to be hand held. The hand held user interface may be in the form similar to that of a writing instrument, such as a pen. In embodiments, the hand held user interface includes technologies relating to writing surface tip pressure monitoring, lens configurations setting a predetermined imaging distance, user interface software mode selection, quick software application launching, and other interface technologies.

Claims

1. A method, comprising:

capturing an image from a writing surface end of a hand-held device;

determining, based at least in part on the image, that the writing surface end of the hand-held device is on a writing surface;

monitoring forces exerted on a writing surface end of the hand-held device over a period of time;

identifying a discrete force event during the period of time based on the monitored forces, the discrete force event including a sudden and substantial increase in force;

causing a graphic user interface element to appear in a display of a head-worn computer in the event that the discrete force event substantially matches a predetermined force signature;

wherein the hand-held device includes an inertial movement unit (IMU) to determine motion of the hand-held device, and

wherein the motion is used in coordination with an image of a writing surface to determine a stroke pattern.

2. The method of claim 1, wherein the motion is used to predict a gesture, wherein the gesture is used to control an aspect of a graphical user interface.

3. The method of claim 1, wherein the motion causes a selection of a user interface mode.

4. The method of claim 1, wherein the force is identified using a piezo-electric device.

5. The method of claim 1, wherein the hand-held device is in communication with a head-worn computer (HWC).

6. The method of claim 1, wherein the user interface process is a selection of an item.

7. The method of claim 1, wherein the user interface process produces a menu associated with a right-side click.

8. The method of claim 1, wherein the user interface process produces a result associated with a double click.

9. A hand-held user interface device for a head-worn computer, comprising:

a force monitor adapted to measure forces exerted on a writing surface end of a hand-held device over a period of time;

a camera adapted to capture an image from the writing surface end of the hand-held device; and

a processor adapted to determine that the writing surface end of the hand-held device is on a writing surface based, at least in part, on the image, the processor further adapted to identify a discrete force event during the period of time based on the measured forces, the discrete force event including a sudden and substantial increase in force,

wherein the processor is adapted to communicate with the head-worn computer to cause a graphical user interface element to appear in a display of the head-worn computer in the event the discrete force event exceeds a predetermined threshold; and

wherein the hand-held device includes an inertial movement unit (IMU) to determine motion of the hand-held device, and wherein the motion is used in coordination with an image of a writing surface to determine a stroke pattern.