Methods and apparatuses for gesture based remote control

• US 8,150,384 B2
• Filed: 06/16/2010
• Issued: 04/03/2012
• Est. Priority Date: 06/16/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Abifaker, Chadia, Velarde, Ruben M.
• Primary Examiner(s): Chang, Kent
• Assistant Examiner(s): Siddiqui, Kashif
• Application Number:
  US12/816,885
• Publication Number:
  US 20110312311A1
• Time in Patent Office: 657 Days
• Field of Search: 455418-4221, 455434-435, 455/456, 455/566, 725 35- 37, 348154-156, 348/208, 345156-158
• US Class Current: 455/418
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G08C 2201/20 Binding and programming of remote control devices
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 2201/71 Directional beams
• CPC Class Codes: G08C 2201/91 Remote control based on location and proximity
• CPC Class Codes: G08C 2201/93 Remote control using other portable devices, e.g. mobile phone, PDA, laptop
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/21 for social networking applications
• CPC Class Codes: H04W 4/50 Service provisioning or reconfiguring
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

Methods and apparatuses are provided that may be implemented in and/or with a mobile device to allow gesture based remote control of one or more controllable devices.

Claims

1. A method comprising:

estimating a location of a mobile device based, at least in part, on one or more received signals;

associating an orientation of a mobile device with a direction toward a location of a controllable device, said direction being determined based, at least in part, on said estimated location of said mobile device and said orientation of said mobile device;

indicating a selection of said controllable device;

determining one or more commands associated with said selected controllable device based, at least in part, on user input obtained via one or more inertial sensors of said mobile device; and

initiating transmission of said commands to said selected controllable device.

2. The method as recited in claim 1, and further comprising:

determining said orientation based, at least in part, on signals associated with at least one of;

said one or more inertial sensors, a location service receiver, a compass, and/or a magnetometer.

3. The method as recited in claim 2, wherein said one or more inertial sensors comprises at least one at least one of:

an accelerometer, a gyroscope, and/or a barometer.

4. The method as recited in claim 1, wherein said one or more signals are received with a location service receiver.

5. The method as recited in claim 1, wherein said direction comprises a directional region of space and further comprising:

indicating that said location of said controllable device is within said directional region of space.

6. The method as recited in claim 1, and further comprising:

displaying information identifying at least said controllable device; and

obtaining user input associated with said selection of said controllable device via at least one of;

an interactive or touch-screen display, a microphone, a camera, a button or keypad, and/or at least one of said one or more inertial sensors.

7. The method as recited in claim 6, and further comprising:

displaying information identifying at least one other selectable controllable device.

8. The method as recited in claim 6, and further comprising:

authenticating that a user may select and/or control said controllable device based, at least in part, on said user input.

9. The method as recited in claim 1, and further comprising:

indicating said selection of said controllable device via at least one of a visual interface, an audio interface, and/or a tactile interface.

10. The method as recited in claim 1, wherein said one or more commands identify at least one of:

at least one physical act performed by a person, and/or at least one physical gesture performed using said mobile device, as determined based, at least in part, on one or more signals associated with said one or more inertial sensors.

11. The method as recited in claim 10, wherein said at least one physical act results in at least one of an identifiable audio waveform and/or an identifiable visual image.

12. The method as recited in claim 11, wherein said at least one physical act is identified based, at least in part, on signals associated with at least one of:

a microphone and/or a camera of said mobile device.

13. The method as recited in claim 1, and further comprising:

obtaining said one or more commands associated with said selected controllable device from at least one of;

previously stored data, said controllable device, and/or from one or more other devices.

14. The method as recited in claim 1, and further comprising:

obtaining status information associated with said selected controllable device via a network interface; and

presenting feedback to a user based, at least in part, on said status information.

15. The method as recited in claim 1, and further comprising at least one of:

transmitting one or more signals representing said commands to at least one other device via a network interface, and/or to said selected controllable device via a network interface and/or a direct communication link.

39. The mobile device as recited in claim 1, and wherein said at least one processing unit to further determine said direction based, at least in part, on said estimated location of said mobile device and said orientation.

16. A method comprising:

with a mobile device;

obtaining an estimated location of a controllable device;

processing signals from one or more inertial sensors of said mobile device to characterize one or more gestures;

programming said mobile device to initiate transmission of one or more commands to said controllable device based, at least in part, on said characterized one or more gestures; and

programming said mobile device to select said controllable device for receiving said one or more commands responsive to an orientation of said mobile device with a direction toward said estimated location of said controllable device, said direction being determined based, at least in part, on an estimated location of said mobile device and said orientation of said mobile device, wherein said estimated location of said mobile device is based, at least in part, on one or more received signals.

17. The method as recited in claim 16, and further comprising:

receiving signals which characterize at least one additional gesture of said one or more gestures.

18. The method as recited in claim 16, and further comprising:

obtaining at least one of;

operational information associated with said controllable device;

access control information associated with controllable device; and

/or identification information associated with said controllable device.

19. The method as recited in claim 16, and further comprising:

via a camera of said mobile device, obtaining at least one of;

image information associated with said controllable device; and

/or identification information associated with said controllable device.

20. An apparatus comprising:

means for estimating a location of a mobile device based, at least in part, on one or more received signals;

means for associating an orientation of a mobile device with a direction toward a location of a controllable device, said direction being determined based, at least in part, on said estimated location of said mobile device and said orientation of said mobile device;

means for indicating a selection of said controllable device;

means for determining one or more commands associated with said selected controllable device based, at least in part, on user input affecting inertial motion of said mobile device; and

means for initiating transmission of said commands to said selected controllable device.

21. The apparatus as recited in claim 20, and further comprising:

means for determining said orientation.

22. The apparatus as recited in claim 20, wherein said direction comprises a directional region of space and further comprising:

means for indicating that said location of said controllable device is within said directional region of space.

23. The apparatus as recited in claim 20, and further comprising:

means for displaying information identifying at least said controllable device; and

means for obtaining user input associated with said selection of said controllable device.

24. The apparatus as recited in claim 23, and further comprising:

means for displaying information identifying at least one other selectable controllable device.

25. The apparatus as recited in claim 23, and further comprising:

means for authenticating that a user may select and/or control said controllable device.

26. The apparatus as recited in claim 20, and further comprising:

means for indicating said selection of said controllable device.

27. The apparatus as recited in claim 20, wherein said one or more commands identify at least one of:

at least one physical act performed by a person, and/or at least one physical gesture performed using said mobile device.

28. The apparatus as recited in claim 27, wherein said at least one physical act results in at least one of an identifiable audio waveform and/or an identifiable visual image.

29. The apparatus as recited in claim 20, and further comprising:

means for obtaining said one or more commands associated with said selected controllable device from at least one of;

previously stored data, said controllable device, and/or from one or more other devices.

30. The apparatus as recited in claim 20, and further comprising:

means for obtaining status information associated with said selected controllable device via a network interface; and

means for presenting feedback to a user based, at least in part, on said status information.

31. The apparatus as recited in claim 20, and further comprising at least one of:

means for transmitting one or more signals representing said commands to at least one other device, and/or to said selected controllable device via a direct communication link.

32. An apparatus for use in a mobile device comprising:

means for obtaining an estimated location of a controllable device;

means for processing signals from one or more inertial sensors of said mobile device to characterize one or more gestures;

means for programming said mobile device to initiate transmission of one or more commands to said controllable device based, at least in part, on said characterized one or more gestures; and

means for programming said mobile device to select said controllable device for receiving said one or more commands responsive to an orientation of said mobile device with a direction toward said estimated location of said controllable device, said direction being determined based, at least in part, on an estimated location of said mobile device and said orientation of said mobile device, wherein said estimated location of said mobile device is based, at least in part, on one or more received signals.

33. The apparatus as recited in claim 32, and further comprising:

means for receiving signals which characterize at least one additional gesture of said one or more gestures.

34. The apparatus as recited in claim 32, and further comprising:

means for obtaining at least one of;

operational information associated with said controllable device;

access control information associated with controllable device; and

/or identification information associated with said controllable device.

35. The apparatus as recited in claim 32, and further comprising:

means for obtaining at least one of;

image information associated with said controllable device; and

/or identification information associated with said controllable device.

36. A mobile device comprising:

memory;

a network interface;

a location service receiver;

one or more inertial sensors;

at least one user interface; and

at least one processing unit to;

estimate a location of said mobile device based, at least in part, on one or more signals received via at least one of said network interface or said location service receiver;

associate an orientation of said mobile device with a direction toward a location of a controllable device as stored in said memory, said direction being determined based, at least in part, on said estimated location of said mobile device and said orientation of said mobile device;

indicate a selection of said controllable device via said at least one user interface;

determine one or more commands associated with said selected controllable device based, at least in part, on user input obtained via said one or more inertial sensors; and

initiate transmission of said commands to said selected controllable device via said network interface.

37. The mobile device as recited in claim 36, and further comprising:

at least one of;

a compass, and/or a magnetometer; and

wherein said at least one processing unit to further determine said orientation based, at least in part, on signals associated with at least one of;

said one or more inertial sensors, said location service receiver, said compass, and/or said magnetometer.

38. The mobile device as recited in claim 37, wherein said one or more inertial sensors comprises at least one at least one of:

an accelerometer, a gyroscope, and/or a barometer.

40. The mobile device as recited in claim 36, wherein said direction comprises a directional region of space and wherein said at least one processing unit to further determine that said location of said controllable device is within said directional region of space.

41. The mobile device as recited in claim 36, and wherein said at least one processing unit to further:

display information identifying at least said controllable device via said at least one user interface; and

obtain user input associated with said selection of said controllable device via at least one of;

an interactive or touch-screen display, a microphone, a camera, a button or keypad, and/or at least one of said one or more inertial sensors.

42. The mobile device as recited in claim 41, and wherein said at least one processing unit to further display information identifying at least one other selectable controllable device via said at least one user interface.

43. The mobile device as recited in claim 41, and wherein said at least one processing unit to further authenticate that a user may select and/or control said controllable device based, at least in part, on said user input.

44. The mobile device as recited in claim 36, and wherein said at least one processing unit to further indicate said selection of said controllable device via said at least one user interface, said at least one user interface comprising at least one of:

a visual interface;

an audio interface; and

/or a tactile interface.

45. The mobile device as recited in claim 36, wherein said one or more commands identify at least one of:

at least one physical act performed by a person, and/or at least one physical gesture performed using said mobile device, as determined based, at least in part, on one or more signals associated with said one or more inertial sensors.

46. The mobile device as recited in claim 45, wherein said at least one physical act results in information representing at least one of an audio waveform and/or an visual image that is identifiable by said at least one processing unit.

47. The mobile device as recited in claim 46, and further comprising:

at least one of a microphone and/or a camera; and

wherein said at least one physical act is identifiable based, at least in part, on signals associated with at least one of;

said microphone and/or said camera.

48. The mobile device as recited in claim 36, and wherein said at least one processing unit to further obtain said one or more commands associated with said selected controllable device from one or more other devices via said network interface.

49. The mobile device as recited in claim 36, and wherein said at least one processing unit to further:

obtain status information associated with said selected controllable device via said network interface; and

present feedback to a user based, at least in part, on said status information.

50. A mobile device comprising:

one or more inertial sensors; and

at least one processing unit to;

obtain an estimated location of a controllable device;

process signals from said one or more inertial sensors of said mobile device to characterize one or more gestures;

identify one or more commands for use by said controllable device based, at least in part, on said characterized one or more gestures; and

identify said controllable device for receiving said one or more commands responsive to an orientation of said mobile device with a direction toward said estimated location of said controllable device, said direction being determined based, at least in part, on an estimated location of said mobile device and said orientation of said mobile device, wherein said estimated location of said mobile device is based, at least in part, on one or more received signals.

51. The mobile device as recited in claim 50, and further comprising:

a network interface; and

wherein said at least one processing unit to further obtain signals via said network interface which characterize at least one additional gesture of said one or more gestures.

52. The mobile device as recited in claim 50, and further comprising:

a network interface; and

wherein said at least one processing unit to further obtain via said network interface at least one of;

operational information associated with said controllable device;

access control information associated with controllable device;

identification information associated with said controllable device; and

/or said estimated location of said controllable device.

53. The mobile device as recited in claim 50, and further comprising:

a camera; and

wherein said at least one processing unit to further obtain via said camera at least one of;

image information associated with said controllable device; and

/or identification information associated with said controllable device.

54. The mobile device as recited in claim 50, and further comprising:

a location service receiver; and

wherein said at least one processing unit to further determine said estimated location of said controllable device based, at least in part, on one or more signals associated with said location service receiver.

55. An article comprising:

a computer readable medium having computer implementable instructions stored thereon which are executable by at least one processing unit in a mobile device to;

estimate a location of said mobile device based, at least in part, on one or more received signals;

associate an orientation of said mobile device with a direction toward a location of a controllable device, said direction being determined based, at least in part, on said estimated location of said mobile device and said orientation of said mobile device;

indicate a selection of said controllable device;

determine one or more commands associated with said selected controllable device based, at least in part, on user input obtained via said one or more inertial sensors; and

initiate transmission of said commands to said selected controllable device.

56. The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

determine said orientation based, at least in part, on signals associated with at least one of;

said one or more inertial sensors, a location service receiver, a compass, and/or a magnetometer.

57. The article as recited in claim 56, wherein said one or more inertial sensors comprises at least one at least one of:

an accelerometer, a gyroscope, and/or a barometer.

58. The article as recited in claim 55, wherein said one or more signals are received with a location service receiver.

59. The article as recited in claim 55, wherein said direction comprises a directional region of space and wherein said computer implementable instructions are executable by said at least one processing unit to further:

determine that said location of said controllable device is within said directional region of space.

60. The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

initiate display of information identifying at least said controllable device; and

obtain user input associated with said selection of said controllable device via at least one of;

an interactive or touch-screen display, a microphone, a camera, a button or keypad, and/or at least one of said one or more inertial sensors.

61. The article as recited in claim 60, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

initiate display of information identifying at least one other selectable controllable device.

62. The article as recited in claim 60, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

authenticate that a user may select and/or control said controllable device based, at least in part, on said user input.

63. The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

initiate an indication said selection of said controllable device via at least one of a visual interface, an audio interface, and/or a tactile interface.

64. The article as recited in claim 55, wherein said one or more commands identify at least one of:

at least one physical act performed by a person, and/or at least one physical gesture performed using said mobile device, as determined based, at least in part, on one or more signals associated with said one or more inertial sensors of said mobile device.

65. The article as recited in claim 64, wherein said at least one physical act results in at least one of an audio waveform and/or an visual image identifiable by said at least one processing unit executing said computer implementable instructions.

66. The article as recited in claim 65, wherein said at least one physical act is identified based, at least in part, on signals associated with at least one of:

a microphone and/or a camera of said mobile device.

67. The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

obtain said one or more commands associated with said selected controllable device from at least one of;

previously stored data, said selected controllable device, and/or from one or more other devices.

68. The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

obtain status information associated with said selected controllable device via a network interface; and

initiate presentation of feedback to a user based, at least in part, on said status information.

69. The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

initiate transmission of one or more signals representing said commands to at least one other device via a network interface, and/or to said selected controllable device via a network interface and/or a direct communication link.

70. An article comprising:

a computer readable medium having computer implementable instructions stored thereon which are executable by at least one processing unit in a mobile device to;

obtain an estimated location of a controllable device;

process signals from said one or more inertial sensors of said mobile device to characterize one or more gestures;

identify one or more commands for use by said controllable device based, at least in part, on said characterized one or more gestures; and

identify said controllable device for receiving said one or more commands responsive to an orientation of said mobile device with a direction toward said estimated location of said controllable device, said direction being determined based, at least in part, on an estimated location of said mobile device and said orientation of said mobile device, wherein said estimated location of said mobile device is based, at least in part, on one or more received signals.

71. The article as recited in claim 70, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

obtain signals which characterize at least one additional gesture of said one or more gestures.

72. The article as recited in claim 70, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

obtain at least one of;

operational information associated with said controllable device;

access control information associated with controllable device;

identification information associated with said controllable device; and

/or said estimated location of said controllable device.

73. The article as recited in claim 70, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

obtain via a camera of said mobile device at least one of;

image information associated with said controllable device; and

/or identification information associated with said controllable device.

74. The article as recited in claim 70, and wherein said computer implementable instructions are executable by said at least one processing unit to further:

determine said estimated location of said controllable device based, at least in part, on one or more signals associated with a location service receiver.