Gesture control

• US 9,335,825 B2
• Filed: 01/26/2010
• Issued: 05/10/2016
• Est. Priority Date: 01/26/2010
• Status: Active Grant

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Ollikainen, Jani Petri Juhani, Teikari, Ilari Aleksi, Kaunisto, Risto Heikki Sakari, Hui, Ping, Rautiainen, Terhi Tuulikki
• Primary Examiner(s): Feild, Joseph
• Assistant Examiner(s): ELNAFIA, SAIFELDIN E
• Application Number:
  US12/693,667
• Publication Number:
  US 20110181509A1
• Time in Patent Office: 2,296 Days
• Field of Search: 345156-158
• US Class Current: 1/1
• CPC Class Codes: G01S 13/56 for presence detection presence detection using near field arrangements G01V3/00, e.g. G01V3/08, G01V3/12; burglar, theft or intruder alarms with electrical actuation G08B13/22 - G08B13/26
• CPC Class Codes: G01S 13/88 Radar or analogous systems specially adapted for specific applications electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

An apparatus including: a radio transmitter configured to transmit radio signals that are at least partially reflected by a human body; one or more radio receivers configured to receive the transmitted radio signals after having been at least partially reflected by a human body of a user; a gesture detector configured to detect a predetermined time-varying modulation that is present in the received radio signals compared to the transmitted radio signals; and a controller configured to interpret the predetermined time-varying modulation as a predetermined user input command and change the operation of the apparatus.

Claims

1. An apparatus comprising:

at least one processor; and

at least one memory including computer program code, where the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to at least;

detect that an application is being started on the apparatus;

in response to the application being started on the apparatus, turn on a continuous wave doppler radar at the apparatus and transmit radio signals that comprise the continuous wave doppler radar, wherein the radio signals are at least partially reflected by a human body of a user of the apparatus;

receive the transmitted radio signals after having been at least partially reflected by a gesture by the human body of the user;

detect in the received radio signals a predetermined time-varying modulation caused by the gesture by the human body of the user and that is present in a modulation of the received radio signals as compared to a modulation of the transmitted radio signals, wherein detecting the predetermined time-varying modulation of the received signal comprises detecting a doppler frequency shift in the continuous wave doppler radar of the radio signals transmitted from the apparatus, wherein the doppler frequency shift comprises a frequency modulated continuous wave variation caused by the gesture by the human body of the user;

associate the detected predetermined time-varying modulation with a predetermined user input command; and

based on the associated predetermined user input command control at least one operation of the application on the apparatus.

2. An apparatus as claimed in claim 1, wherein the apparatus is a hand portable apparatus.

3. An apparatus as claimed in claim 1, wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to determine from the predetermined time-varying modulation of the received radio signal one or more kinematic parameters that parameterize the gesture by the human body that caused the predetermined time-varying modulation including at least a speed or velocity parameter.

4. An apparatus as claimed in claim 3, wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to detect the one of more kinematic parameters to resolve at least one of a position and a velocity of the gesture in two or three dimensions.

5. An apparatus as claimed in claim 4, wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to provide slowly varying analogue control when a slowly moving continuous gesture is detected.

6. An apparatus as claimed in claim 3, wherein a separate user actuation in addition to the gesture is required to enable the controlling the at least one operation of the apparatus in response to the gesture.

7. An apparatus as claimed in claim 4, wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to provide binary two-state control when a fast moving gesture is detected.

8. An apparatus as claimed in claim 1, wherein when the time-varying modulation of the received signal comprises the doppler frequency shift, the predetermined time-varying modulation is characterized by a doppler frequency shift between the received signals and the transmitted signals.

9. An apparatus as claimed in claim 1, wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to maintain in a memory of the apparatus a correspondence between a time varying nature of the predetermined user input command and a time varying nature of the predetermined time-varying modulation, and to use the maintained correspondence in the memory to associate the detected predetermined time-varying modulation caused by the gesture by the human body of the user with the predetermined user input command.

10. An apparatus as claimed in claim 1, wherein the application is a music player application, wherein the controlling the at least one operation of the application comprises controlling at least a volume of the music player application.

11. An apparatus as claimed in claim 10, wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus:

for a case the predetermined time-varying modulation is caused by the gesture moving towards a face of the apparatus at a velocity that does not exceed a pre-defined threshold, to reduce the volume of the music player application,

for a case the predetermined time-varying modulation is caused by the gesture moving away from a face of the apparatus at a velocity that does not exceed a pre-defined threshold, to increase the volume of the music player application,

for a case the predetermined time-varying modulation is caused by the gesture moving towards a face of the apparatus at a velocity that exceeds a pre-defined threshold, to mute the volume of the music player application, and

for a case the predetermined time-varying modulation is caused by the gesture moving away from a face of the apparatus at a velocity that exceeds a pre-defined threshold, to exit the music player application.

12. An apparatus as claimed in claim 1, wherein the controlling the at least one operation of the application comprises the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to change any one or more of:

display zoom-in, display zoom-out, display scroll-up, display scroll-down, display scroll-right, display scroll-left, a telephone call state, a camera capture state, an audio output muting, and an application run status.

13. An apparatus as claimed in claim 1, wherein the apparatus has a front face and wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to:

transmit the radio signals at least substantially normally to the front face, and

receive the radio signals that are reflected towards the front face, wherein the gesture by the human body of the user is required to be moving towards the front face of the apparatus to cause the doppler radar frequency modulated continuous wave variation to change the at least one operation of the application on the apparatus.

14. An apparatus as claimed in claim 1, wherein the apparatus comprises a plurality of radio receivers and wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to:

detect, for each of the plurality of receivers, the predetermined time-varying modulation that is present in a modulation of the received radio signals compared to the modulation of the transmitted radio signals, and

interpret a combination of predetermined time-varying modulations associated with the respective radio receivers as the predetermined user input command and control the at least one operation of the apparatus.

15. An apparatus as claimed in claim 1, wherein the at least one memory including the computer program code is configured with the at least one processor to cause the apparatus to operate with transmission diversity.

16. An apparatus as claimed in claim 1, wherein the apparatus is user programmable to predetermine time-varying modulations for detection in the received radio signals.

17. An apparatus comprising:

a controller configured to detect that an application is being started on the apparatus,

and in response to the application being started on the apparatus turn on a continuous wave doppler radar at the apparatus;

a radio receiver configured to receive radio signals that comprise the continuous wave doppler radar after having been at least partially reflected by a gesture by a human body of a user of the apparatus;

a gesture detector configured to detect in the received radio signals a predetermined time-varying modulation caused by the gesture by the human body of the user and that is present in a modulation of the received radio signals as compared to a modulation of the radio signals before the reflection, wherein detecting the predetermined time-varying modulation of the received signal comprises detecting a doppler frequency shift in the transmitted continuous wave doppler radar from the apparatus, wherein the doppler frequency shift comprises a frequency modulated continuous wave variation caused by the gesture by human body of the user;

an interface for providing the detected predetermined time-varying modulation as an output; and

the controller configured to associate the detected predetermined time-varying modulation with a predetermined user input command, and based on the associated predetermined user input command control at least one operation of the application on the apparatus.

18. The apparatus as claimed in claim 17, wherein the gesture detector is configured to determine, with respect to the gesture by the human body that provides the received radio signals, one or more kinematic parameters that parameterize the gesture by the human body that caused the time-varying modulation including at least a speed or velocity parameter.

19. A method comprising:

detecting, by an apparatus, that an application is being started on the apparatus, and in response to the application being started on the apparatus turning on a continuous wave doppler radar at the apparatus;

transmitting, by the apparatus, radio signals that comprise the continuous wave doppler radar, wherein the radio signals are at least partially reflected by a human hand;

receiving, by the apparatus, the transmitted radio signals after having been at least partially reflected by a gesturing human hand;

detecting, by the apparatus, a predetermined time-varying modulation, characterizing the gesture, that is present in a modulation of the received radio signals compared to a modulation of the transmitted radio signals, wherein detecting the predetermined time-varying modulation of the received signal comprises detecting a doppler frequency shift in the transmitted radio signals based on the continuous wave doppler radar, wherein the doppler frequency shift comprises a frequency modulated continuous wave variation caused by the gesturing human hand; and

controlling, by the apparatus, an operation of the application on the apparatus in dependence upon the predetermined time-varying modulation.

20. A method as claimed in claim 19 further comprising:

determining one or more kinematic parameters that parameterize a gesture that causes the predetermined time-varying modulation including at least a speed or velocity parameter.

21. The method as claimed in claim 19, wherein the application is a music player application, wherein the controlling the at least one operation of the application comprises controlling at least a volume of the music player application.

22. The method as claimed in claim 21, comprising:

for a case the predetermined time-varying modulation is caused by the gesture moving towards a face of the apparatus at a velocity that does not exceed a pre-defined threshold, reducing the volume of the music player application, and

for a case the predetermined time-varying modulation is caused by the gesture moving away from a face of the apparatus at a velocity that does not exceed a pre-defined threshold, increasing the volume of the music player application.

23. The method as claimed in claim 21, comprising:

for a case the predetermined time-varying modulation is caused by the gesture moving towards a face of the apparatus at a velocity that exceeds a pre-defined threshold, muting the volume of the music player application, and

for a case the predetermined time-varying modulation is caused by the gesture moving away from a face of the apparatus at a velocity that exceeds a pre-defined threshold, exiting the music player application.