In-air ultrasound pen gestures

• US 9,720,521 B2
• Filed: 02/19/2015
• Issued: 08/01/2017
• Est. Priority Date: 02/21/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Padovani, Niccolo, Bornstein, Gilad, Altman, Nathan
• Primary Examiner(s): Sharifi-Tafreshi, Koosha
• Application Number:
  US14/626,198
• Publication Number:
  US 20150242002A1
• Time in Patent Office: 894 Days
• Field of Search: 345179, 345173, 345177, 178 1801, 178 1902, 178 1804, 178 1901
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/043 using propagating acoustic waves
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Implementations of the technology described herein provide a method for detecting gesture commands using an ultrasonic pen system. The system has a pen and a user device. Detection of gesture commands is based on two-dimensional gestures relative to the screen of a user device, three-dimensional gestures relative to the screen of the user device, roll/rotation around a longitudinal axis of the pen body, and micro-twisting around the longitudinal axis of the pen body. The user device receives the gestures and translates them into commands such as UNDO and BACK.

Claims

1. A method of detecting gesture commands using an ultrasonic pen and a user device, comprising:

acquiring a plurality of positions of the ultrasonic pen along one or more of an x-axis, a y-axis, and a z-axis of the user device;

recognizing a micro-twisting gesture of an entirety of the ultrasonic pen about a longitudinal axis of the ultrasonic pen, the micro-twisting gesture including a sequence of movements, the recognizing of the sequence of movements including;

recognizing a first movement around the longitudinal axis; and

recognizing a second movement, subsequent to the first movement, around the longitudinal axis in a direction opposite the first movement; and

performing a command based on the recognized sequence of movements included in the micro-twisting gesture.

2. The method of claim 1, wherein acquiring the positions includes:

receiving an ultrasonic signal;

calculating a distance traveled by the ultrasonic signal; and

identifying the plurality of acquired positions based on the calculated distance.

3. The method of claim 1, wherein the plurality of acquired positions are acquired from a top side or a bottom side of the user device.

4. The method of claim 1, wherein the plurality of acquired positions are acquired from a left side or a right side of the user device.

5. An apparatus for detecting gesture commands using an ultrasonic pen and a user device, comprising a processor and a memory, the processor and memory being configured to:

acquire a plurality of positions of the ultrasonic pen along an x-axis, a y-axis, and a z-axis of the user device;

recognize a micro-twisting gesture of an entirety of the ultrasonic pen about a longitudinal axis of the ultrasonic pen, the micro-twisting gesture including a sequence of movements, wherein to recognize the sequence of movements, the processor and memory are further configured to;

recognize a first movement around the longitudinal axis; and

recognize a second movement, subsequent to the first movement, around the longitudinal axis in a direction opposite the first movement; and

perform a command based on the recognized sequence of movements included in the micro-twisting gesture.

6. The apparatus of claim 5, further comprising a microphone array configured to receive an ultrasonic signal, wherein the processor and memory are further configured to:

calculate a distance traveled by the ultrasonic signal; and

identify the plurality of acquired positions based on the calculated distance.

7. The apparatus of claim 5, wherein the first movement is in a first direction along one or more of the x-axis, the y-axis, and the z-axis, and the second movement is in a second direction along one or more of the x-axis, the y-axis, and the z-axis, the second direction being different from the first direction.

8. The apparatus of claim 5, wherein the plurality of acquired positions are acquired from a top side or a bottom side of the user device.

9. A non-transitory computer-readable storage medium including information that, when accessed by a machine, cause the machine to perform operations for detecting gesture commands using an ultrasonic pen and a user device, the operations comprising:

acquiring a plurality of positions of the ultrasonic pen along one or more of an x-axis, a y-axis, and a z-axis of the user device;

recognizing a micro-twisting gesture of an entirety of the ultrasonic pen about a longitudinal axis of the ultrasonic pen, the micro-twisting gesture including a sequence of movements, the recognizing of the sequence of movements including;

recognizing a first movement around the longitudinal axis; and

recognizing a second movement, subsequent to the first movement, around the longitudinal axis in a direction opposite the first movement; and

performing a command based on the recognized sequence of movements included in the micro-twisting gesture.

10. The non-transitory computer-readable storage medium of claim 9, wherein acquiring the positions includes:

receiving an ultrasonic signal;

calculating a distance traveled by the ultrasonic signal; and

identifying the plurality of acquired positions based on the calculated distance.

11. The non-transitory computer-readable storage medium of claim 9, wherein the plurality of acquired positions are acquired from a left side or a right side of the user device.

12. The non-transitory computer-readable storage medium of claim 9, wherein the plurality of acquired positions are acquired from a top side or a bottom side of the user device.