Gesture and voice recognition for control of a device

• US 8,847,881 B2
• Filed: 03/20/2012
• Issued: 09/30/2014
• Est. Priority Date: 11/18/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Matsubayashi, Koji
• Primary Examiner(s): Sharifi-Tafreshi, Koosha
• Application Number:
  US13/424,642
• Publication Number:
  US 20130127712A1
• Time in Patent Office: 924 Days
• Field of Search: 345/173, 345/156, 345/157, 345/158, 345/169, 715/863, 382/103, 348/77, 463/36, 463/37
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/485 End-user interface for client configuration

Abstract

A user interface allows one or more gestures to be devised by a user and mapped or associated with one or more commands or operations of a TV or other device. The user can select the command/operation that is to be associated with each gesture that he/she devised or created. The user is not limited to the use of pre-set gestures that were previously programmed into a system and is not limited to using pre-set commands/operations that were previously associated with pre-set gestures. In alternative embodiments, voice commands or other audible signals are devised by a user and are mapped or associated with commands/operations of a device.

Claims

1. A method for providing an interface, comprising:

receiving, by a computing device via a camera in communication with the computing device, the computing device having a display, a first gesture of a user;

entering, by the user into the computing device, a predetermined time period which is one of a plurality of time periods available for selection by the user;

entering into the computing device a first selection for a first operation of the computing device, wherein the first operation is displaying a preselected first channel on the display;

associating, by the computing device, the first operation with the first gesture;

receiving, by the computing device via the camera, a second gesture, wherein the second gesture is substantially the same as the first gesture;

performing the first operation by the computing device in response to the receiving of the second gesture;

entering, by the user into the computing device, a second selection for a second operation of the computing device, wherein the second operation is displaying a preselected second channel on the display after the preselected first channel has been displayed for the predetermined time period, and

wherein each of the first channel and the second channel are of a plurality of channels of content available to the user

wherein the associating of the first operation with the first gesture includes associating both the first operation and the second operation with the first gesture,

wherein the performing of the first operation includes automatically performing the first operation and the second operation the receiving of the second gesture.

2. A method for providing an interface, comprising:

receiving, by a first computing device via a first camera in communication with the first computing device, a first gesture of a user;

entering, by the user into the first computing device, a selection for an operation of a second computing device, wherein the operation is one of a plurality of different operations available to the user for the selection;

associating, by the first computing device, the operation with the first gesture;

transmitting, from the first computing device to the second computing device, data corresponding to the first gesture and corresponding to the selection of the operation of the second computing device;

receiving, by the second computing device via a second camera in communication with the second computing device, a second gesture, wherein the second gesture is substantially the same as the first gesture; and

performing the operation by the second computing device in response to the receiving of the second gesture.

3. The method of claim 2 wherein the first computing device is one of a laptop computer, a desktop computer, a tablet computer, and a handheld device, and wherein the second computing device is a television.

4. A device for use by a user, comprising:

a memory;

a camera;

a display; and

a processor coupled to the memory, the camera and the display and operable to perform steps comprising;

receiving via the camera a first gesture of the user;

receiving from the user a predetermined time period which is one of a plurality of time periods available for selection by the user;

receiving a first selection by the user for a first operation to be performed by the processor, wherein the first operation is the displaying of a preselected first channel on the display;

associating the first operation with the first gesture;

receiving via the camera a second gesture, wherein the second gesture is substantially the same as the first gesture;

performing the first operation in response to the receiving of the second gesture;

receiving a second selection from the user for a second operation to be performed by the processor, wherein the second operation is the displaying of a preselected second channel on the display after the preselected first channel has been displayed for the predetermined time period, and wherein each of the first channel and the second channel are of a plurality of channels of content available to the user,

wherein the associating of the first operation with the first gesture includes associating both the first operation and the second operation with the first gesture, and

wherein the performing of the first operation includes automatically performing the first operation and the second operation in response to the receiving of the second gesture.