Projector and ultrasonic gesture-controlled communicator

• US 8,471,868 B1
• Filed: 11/28/2007
• Issued: 06/25/2013
• Est. Priority Date: 11/28/2007
• Status: Active Grant

• Current Assignee: T-Mobile Innovations LLC (Deutsche Telekom AG)
• Original Assignee: Sprint Communications Company LP (Deutsche Telekom AG)
• Inventors: Wilson, Thomas Henry, Persson, Carl, Heitland, Ken W.
• Primary Examiner(s): Shalwala, Bipin
• Assistant Examiner(s): Ketema, Benyam
• Application Number:
  US11/946,095
• Time in Patent Office: 2,036 Days
• Field of Search: 345/157, 345/177, 715/863
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

System, methods, and computer-readable media for providing a gesture-controlled user interface in interacting with a communication device are provided. Embodiments of the present invention include a communication device that recognizes a gesture made by a person that has one or more transducers attached to their hand. The gesture is recognized by performing calculations on the sound waves received from the one or more transducers. In addition, the gesture may be recognized using a digital camera and image recognition techniques. In one embodiment, the gesture is recognized using an echolocation system that transmits ultrasonic waves and captures the waves reflected off the object making the gesture. In another embodiment, the gesture is made and interpreted in relation to a user interface that is projected from the communication device using a laser projector.

Claims

1. One or more non-transitory computer-readable media having computer-executable instructions embodied thereon for performing a method of providing a gesture-controlled user interface with a communication device that includes a microphone array and a laser projector, the method comprising:

receiving one or more microphone-array signals from the microphone array communicating a representation of one or more sound waves received from one or more transducers that are positioned remotely from the communication device and are capable of being removably attached to a user and that are utilized by the user to perform a gesture comprising moving one or more fingers in the outline of a letter or a number;

performing calculations on the one or more microphone-array signals received to generate two or three dimensional coordinates of the one or more fingers making the gesture;

comparing the coordinates to gesture patterns in a data store using a best-fit algorithm; and

identifying the gesture from matching gesture patterns in the data store and retrieving the associated meaning from the data store.

2. The one or more computer-readable media of claim 1, wherein the method further comprises:

controlling a function of the communication device in response to the gesture.

3. The one or more computer-readable media of claim 1, wherein the method further comprises:

projecting a display-type user interface onto a surface using the laser projector.

4. The one or more computer-readable media of claim 3, wherein the method further comprises:

adjusting an appearance characteristic of the display-type user interface using one or more of;

picture data received from a digital camera coupled to the communication device; and

echolocation data received from an echolocation system coupled to the communication device.

5. The one or more computer-readable media of claim 1, wherein the method further comprises:

receiving an echolocation signal from an echolocation subsystem that is coupled to the communication device; and

determining the gesture made by the user using at least the one or more microphone-array signals and the echolocation signal.

6. The one or more computer-readable media of claim 5, wherein the method further comprises:

receiving picture data from a digital camera that is coupled to the communication device; and

identifying the gesture made by the user utilizing at least the one or more microphone-array signals, the echolocation signal, and the picture data.

7. The one or more computer-readable media of claim 1, wherein the method further comprises:

projecting a display-type user interface onto a movable surface using the laser projector; and

wherein the one or more transducers are connected to the movable surface.

8. The one or more computer-readable media of claim 1, wherein the method further comprises:

receiving a voice signal from a microphone embodying words spoken by the user; and

controlling a function on the communication device based on the voice signal.

9. One or more non-transitory computer-readable media having computer-executable instructions embodied thereon for performing a method of controlling a communication device having a laser projector, a microphone array, a digital camera, and an echolocation subsystem using gestures, the method comprising:

projecting a display-type user interface using a laser projector onto a surface that is positioned remotely from the communication device;

receiving an echolocation signal from the echolocation subsystem;

receiving picture data from the digital camera;

receiving one or more microphone-array signals from the microphone array that represent one or more sound waves sent by one or more transducers that are positioned remotely from the communication device and are capable of being removably attached to a user and that are utilized by the user to perform a gesture comprising moving one or more fingers in order to cast a shadow on the display-type user interface and pushing a virtual button or selecting an object on the display-type user interface with the shadow;

calculating the distance between the one or more transducers and the communication device;

calculating the distance between the display-type user interface and the communication device using the echolocation signal;

determining a location of the shadow relative to the user interface and what the user is intending to select; and

interpreting the gesture performed by the user using the one or more microphone-array signals and the picture data.

10. The one or more media of claim 9, wherein the method further comprises:

automatically controlling one or more appearance characteristics of the laser projector using the echolocation signal to determine a distance between the laser projector and the surface.

11. The one or more media of claim 10, wherein the method further comprises:

automatically controlling the one or more appearance characteristics of the laser projector using the picture data.

12. The one or more media of claim 9, wherein the method further comprises communicating one or more of voice and data over a wireless network using a radio coupled to the communication device.

13. The one or more media of claim 9, wherein the method further comprises:

receiving a picture taking gesture that instructs the communication device to take a picture with the digital camera; and

in response to the picture taking gesture, taking a picture with the digital camera.

14. The one or more media of claim 13, wherein the user interface comprises:

a grid that outlines a photograph area, wherein the photograph area represents an area to be captured on a photograph taken by the digital camera with a present zoom setting.

15. The one or more media of claim 9, wherein the method further comprises:

projecting the user interface using the laser projector onto a movable surface.

16. The one or more media of claim 15, wherein the method further comprises:

scrolling through a plurality of layers within a three dimensional representation projected on the user interface in response to moving the movable surface towards or away from the communication device.

17. A communication device with a gesture-controlled user interface, the communication device comprising:

a laser projector that is capable of displaying a user interface on a movable surface that is positioned remotely from the communication device;

a microphone array that is capable of receiving one or more sound waves transmitted from one or more transducers that are positioned remotely from the communication device and are capable of being removably attached to a user; and

a gesture interpretation component that evaluates the one or more sound waves received by the microphone array and determines a gesture made by the user with one or more transducers removably attached to the user, the gesture comprising moving one or more fingers in the outline of a letter or a number or up and down in view of the communication device,

the gesture interpretation component further performs calculations on the one or more microphone-array signals received to generate two or three dimensional coordinates of the one or more fingers making the gesture;

compares the coordinates to gesture patterns in a data store using a best-fit algorithm; and

identifies the gesture from matching gesture patterns in the data store and retrieving the associated meaning from the data store.

18. The communication device of claim 17, further comprising:

an ultrasonic echolocation subsystem that includes an incorporated transducer and an echolocation microphone; and

an ultrasonic echolocation component that determines at least a shape of an object, a size of the object, and a distance the object is from the communication device, using the one or more sound waves transmitted by the incorporated transducer and received by the echolocation.

19. The communication device of claim 17, further comprising:

a digital camera that generates a digital image.

20. The communication device of claim 17, further comprising one or more of the following:

an audio microphone;

a speaker;

an LCD display;

a touch screen;

a keyboard;

a keypad; and

a USB port.