MOTION AND GESTURE INPUT FROM A WEARABLE DEVICE

• US 20180196514A1
• Filed: 03/07/2018
• Published: 07/12/2018
• Est. Priority Date: 09/25/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: ALLEC, Nicholas Paul Joseph, MU, Xiaoyi
• Granted Patent: US 10,503,254 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0308 comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

This disclosure relates to detecting hand gesture input using an electronic device, such as a wearable device strapped to a wrist. The device can have multiple photodiodes, each sensing light at a different position on a surface of the device that faces skin of a user. Examples of the disclosure detect hand gestures by recognizing patterns in sensor data that are characteristic of each hand gesture, as the tissue expands and contracts and anatomical features in the tissue move during the gesture.

Claims

1. A device comprising:

a plurality of light emitters configured to emit light at a tissue of a user;

a plurality of photodiodes configured to sense at least a portion of a reflection of the emitted light and generate sensor data indicative of movement of anatomic features in the tissue of the user;

a plurality of channels, each channel associated with one of the plurality of light emitters and one of the plurality of photodiodes and further associated with a signal pattern including the sensor data from the respective photodiode; and

logic configured to;

receive the signal patterns of the plurality of channels,

determine signal characteristics from the signal patterns, and

determine a hand gesture from the determined signal characteristics.

2. The device of claim 1, wherein the signal patterns from at least some of the plurality of channels are associated with a hand clench of the user, and the signal patterns from others of the plurality of channels are associated with a hand flap of the user.

3. The device of claim 1, wherein the plurality of light emitters includes two light emitters, the plurality of photodiodes includes two photodiodes, and the plurality of channels includes eight channels.

4. The device of claim 3, wherein the determination of the signal characteristics of the signal patterns includes determining a relationship between a peak and a trough of a signal pattern.

5. The device of claim 4, wherein at least some of the signal characteristics include the trough occurring before the peak in the respective signal pattern, and the relationship is identified by a sign indicative of the trough occurring before the peak.

6. The device of claim 4, wherein at least some of the signal characteristics include the peak occurring before the trough in the respective signal pattern, and the relationship is identified by a sign indicative of the peak occurring before the trough.

7. The device of claim 1, wherein the determination of the hand gesture includes determining a difference in the signal characteristics of at least two of the plurality of channels.

8. The device of claim 1, wherein the determination of the signal characteristics of the signal patterns includes determining a time difference between a peak and a trough in a signal pattern.

9. The device of claim 1, wherein the determination of the signal characteristics of the signal patterns includes determining a period between peaks in a signal pattern.

10. The device of claim 1, wherein the determination of the signal characteristics of the signal patterns includes determining a phase between peaks in a signal pattern.

11. The device of claim 1, wherein the sensor data is divided into a plurality of frames and the logic is further configured to:

analyze one or more of the plurality of frames and extract frequency information as additional signal characteristics used in the determination of the hand gesture.

12. The device of claim 1, further comprising:

one or more of a force sensor and an accelerometer, wherein the force sensor is configured to measure a force and generate force sensor data, and the accelerometer is configured to measure an acceleration and generate accelerometer data,

wherein the logic is further configured to;

receive the force sensor data, the accelerometer data, or both, and

further determine the signal characteristics from the received force sensor data, the received accelerometer data, or both.

13. A method of determining a hand gesture, the method comprising:

emitting light from a plurality of light emitters at a tissue of a user;

receiving at least a portion of a reflection of the emitted light using a plurality of photodiodes;

generating sensor data from the received light, the sensor data indicative of movement of anatomic features in the tissue of the user;

associating the plurality of light emitters and plurality of photodiodes with a plurality of channels;

associating signal patterns included in the sensor data to the plurality of channels;

determining signal characteristics from the signal patterns; and

determining a hand gesture from the determined signal characteristics.

14. The method of claim 13, wherein the association of the signal patterns to the plurality of channels includes:

associating the signal patterns from at least some of the plurality of channels with a hand clench of the user; and

associating the signal patterns from others of the plurality of channels with a hand flap of the user.

15. The method of claim 13, wherein the determination of the signal characteristics of the signal patterns includes determining a relationship between a peak and a trough of a signal pattern.

16. The method of claim 15, wherein the relationship includes a sign indicative of a relative order of the peak and the trough in the signal pattern or a time difference between the peak and the trough in the signal pattern.

17. The method of claim 13, wherein the determination of the hand gesture includes determining a difference in the signal characteristics of at least two of the plurality of channels.

18. The method of claim 13, further comprising:

performing one or more of;

measuring a force using a force sensor and generating force sensor data indicative of the measured force, and

measuring an acceleration using an accelerometer and generating accelerometer data indicative of the measured acceleration; and

further determining the signal characteristics from the received force sensor data, the received accelerometer data, or both.

19. The method of claim 13, further comprising:

filtering out information from the sensor data before the determination of the signal characteristics.

20. The method of claim 19, wherein the information includes heart rate frequencies.