Method and apparatus for processing commands directed to a media center

• US 9,678,713 B2
• Filed: 10/09/2012
• Issued: 06/13/2017
• Est. Priority Date: 10/09/2012
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Dimitriadis, Dimitrios, Schroeter, Horst Juergen
• Primary Examiner(s): Vo, Tung
• Assistant Examiner(s): Jiang, Zaihan
• Application Number:
  US13/647,967
• Publication Number:
  US 20140098240A1
• Time in Patent Office: 1,708 Days
• Field of Search: 348169, 3482114, 348E05042
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/4415 using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4751 for defining user accounts, e.g. accounts for children

Abstract

A system that incorporates teachings of the subject disclosure may include, for example, a method for controlling a steering of a plurality of cameras to identify a plurality of potential sources, identifying the plurality of potential sources according to image data provided by the plurality of cameras, assigning a beam of a plurality of beams of a plurality of microphones to each of the plurality of potential sources, detecting a first command comprising one of a first audible cue based on signals from a portion of the plurality of microphones, a first visual cue based on image data from one of the plurality of cameras, or both for controlling a media center, and configuring the media center according to the first command. Other embodiments are disclosed.

Claims

1. A method, comprising:

detecting, by a processing system comprising a processor, a plurality of potential sources for controlling a media center according to image data provided by a plurality of image sensors, wherein the detecting comprises steering, by the system, the plurality of image sensors;

obtaining, by the processing system, directional information associated with the plurality of potential sources according to the image data provided by the plurality of image sensors;

steering, by the processing system, a microphone array towards one of the plurality of potential sources according to the directional information to detect speech signals generated by the plurality of potential sources;

detecting, by the processing system, a gesture of a first source of the plurality of potential sources based on images detected by the plurality of image sensors, wherein the gesture is associated with a first command for controlling the media center;

determining, by processing the system, a characteristic of the gesture that is detected;

determining, by processing the system, a quality of performance for the first command for controlling the media center according to the characteristic of the gesture that is determined;

determining, by the processing system, a conflict between the first command and a second command of a second source of the plurality of potential sources;

presenting, by the processing system, a notification indicating the conflict and requesting a resolution to the conflict;

determining, by the processing system, if a response to the notification indicates the resolution is to perform the first command;

modifying the first command to a modified command according to the quality of performance to reflect the characteristic of the gesture; and

processing, by the processing system, the modified command to control the media center responsive to determining the response indicates the resolution is to perform the first command.

2. The method of claim 1, wherein the characteristic of the gesture comprises speed of the gesture.

3. The method of claim 1, wherein the characteristic of the gesture comprises duration of the gesture.

4. The method of claim 1, further comprising selecting, by the processing system, the command provided by the first source based on a conflict resolution strategy, wherein the conflict resolution strategy comprises assigning a first priority to the first source and assigning a second priority to the second source, and wherein selecting the first command comprises selecting the first command based on the first priority having a higher priority than the second priority.

5. The method of claim 1, wherein the quality of performance of the modified command comprises a magnitude of performance.

6. The method of claim 1, wherein the quality of performance of the modified command comprises a speed of performance.

7. The method of claim 1, wherein the steering of the microphone array comprises steering, by the processing system, a plurality of beams of the microphone array according to a beamforming process.

8. The method of claim 7, further comprising using, by the processing system, the image data provided by the plurality of image sensors to assign a beam of the plurality of beams to each of the plurality of potential sources.

9. The method of claim 1, further comprising identifying, by the processing system, each of the plurality of potential sources comprises by comparing the image data with biometric profiles.

10. The method of claim 9, wherein the biometric profiles comprise one of facial data, body contour data, body color data, body surface data, or combinations thereof.

11. The method of claim 1, further comprising:

retrieving, by the processing system, a user profile associated with the first source; and

configuring, by the processing system, the media center according to the user profile.

12. The method of claim 1, further comprising:

tracking, by the processing system, movements of the plurality of potential sources according to the image data provided by the plurality of image sensors;

steering, by the processing system, the plurality of image sensors according to the movements of the plurality of potential sources; and

steering, by the processing system, the microphone array according to the movements of the plurality of potential sources.

13. The method of claim 1, further comprising:

assigning, by the processing system, one of an acoustic model, a visual model, or a combination thereof to each of the plurality of potential sources; and

processing, by the processing system, one of audible cues, visual cues, or a combination thereof generated by the plurality of potential sources according to the acoustic model, the visual model or both assigned to each of the plurality of potential sources.

14. The method of claim 1, wherein the plurality of potential sources correspond to a plurality of human figures, and wherein the media center comprises one of a set-top box, an audio system, a lighting system, an alarm system, a temperature system, a health monitoring system, or combinations thereof.

15. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processing system comprising a processor, facilitate performance of operations, comprising:

controlling a steering of a plurality of cameras to identify a plurality of potential sources;

obtaining directional information associated with the plurality of potential sources according to image data provided by the plurality of cameras;

directing a beam of a plurality of beams of a plurality of microphones by steering the beam towards one of the plurality of potential sources according to the directional information;

detecting a gesture of a first source of the plurality of potential sources based on image data from one of the plurality of cameras, for controlling a media center, wherein the gesture is associated with a first command;

determining a characteristic of the gesture that is detected;

determining a quality of performance for the first command for controlling the media center according to the characteristic of the gesture that is determined;

determining a conflict between the first command and a second command of a second source of the plurality of potential sources;

presenting a notification indicating the conflict and requesting a resolution to the conflict;

determining if a response to the notification indicates the resolution is to perform the first command;

modifying the first command according to the quality of performance to reflect the characteristic of the gesture to generate a modified command responsive to determining the response indicates the resolution is to perform the first command; and

controlling the media center according to the modified command according to the quality of performance responsive to determining the response indicates the resolution is to perform the first command.

16. The non-transitory computer-readable storage medium of claim 15, wherein the directing of the beam further comprises causing the plurality of microphones to steer the plurality of beams to direct the beam to each of the plurality of potential sources, and wherein the characteristic of the gesture comprises speed of the gesture, duration of the gesture, or any combination thereof.

17. The non-transitory computer-readable storage medium of claim 15, wherein the processor further performs operations comprising:

selecting the first command according to a conflict resolution strategy, and

wherein the quality of performance of the modified command comprises a magnitude of performance, a speed of performance, or any combination thereof.

18. A system, comprising:

a memory to store instructions; and

a processor coupled to the memory, wherein responsive to executing the instructions, the processor performs operations comprising;

controlling a plurality of microphones to steer a plurality of beams of the plurality of microphones to direct a beam by steering the beam towards one of a plurality of potential sources for controlling a media center according to directional information determined from image data obtained for the plurality of potential sources;

detecting a gesture of a first source of the plurality of potential sources based on image data from one of a plurality of image sensors, wherein the gesture is associated with a first command for controlling the media center;

determining a characteristic of the gesture that is detected;

determining a quality of performance for the first command for controlling the media center according to the characteristic of the gesture that is determined;

determining a conflict between the first command and a second command of a second source of the plurality of potential sources;

presenting a notification requesting a resolution to the conflict;

determining if a response to the notification indicates the resolution is to perform the first command;

modifying the first command according to the quality of performance to reflect the characteristic of the gesture to generate a modified command responsive to determining the response indicates the resolution is to perform the first command; and

controlling the media center to execute the modified command according to the quality of performance responsive to determining the response indicates the resolution is to perform the first command.

19. The system of claim 18, comprising the plurality of image sensors, wherein the processor further performs operations comprising:

identifying the plurality of potential sources according to image data provided by the plurality of image sensors; and

exchanging messages with one of the plurality of potential sources that generated the command, wherein the messages comprise one of audible speech, visual messages presented by way of a display device coupled to the processor, or a combination thereof.