Method and apparatus for processing commands directed to a media center

• US 10,219,021 B2
• Filed: 05/08/2017
• Issued: 02/26/2019
• Est. Priority Date: 10/09/2012
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Dimitriadis, Dimitrios, Schroeter, Horst Juergen
• Primary Examiner(s): Jiang, Zaihan
• Application Number:
  US15/589,013
• Publication Number:
  US 20170244997A1
• Time in Patent Office: 659 Days
• Field of Search: 348169, 3482114, 348E05042
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/4415 using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4751 for defining user accounts, e.g. accounts for children

Abstract

A system that incorporates teachings of the subject disclosure may include, for example, a method for controlling a steering of a plurality of cameras to identify a plurality of potential sources, identifying the plurality of potential sources according to image data provided by the plurality of cameras, assigning a beam of a plurality of beams of a plurality of microphones to each of the plurality of potential sources, detecting a first command comprising one of a first audible cue based on signals from a portion of the plurality of microphones, a first visual cue based on image data from one of the plurality of cameras, or both for controlling a media center, and configuring the media center according to the first command. Other embodiments are disclosed.

Claims

1. A method, comprising:

associating, by a processing system comprising a processor, a first gesture of a first object of a plurality of objects based on images from image data with a first command for controlling a media center, wherein the image data is captured by a plurality of image sensors;

associating, by the processing system, a second gesture of a second object of the plurality of objects based on the images from the image data with a second command for controlling the media center, wherein the second gesture is based on images detected by the plurality of image sensors;

modifying, by the processing system, the first command to a modified command according to a characteristic of the first gesture;

determining, by the processing system, a conflict between the first command and the second command;

presenting, by the processing system, a notification indicating the conflict and requesting a resolution to the conflict;

determining, by the processing system, if a response to the notification indicates the resolution is to perform the first command or the second command; and

responsive to a first determination that the response indicates the resolution is to perform the first command processing, by the processing system, the modified command to control the media center responsive to determining the response indicates the resolution is to perform the first command.

2. The method of claim 1, further comprising:

obtaining, by the processing system, directional information associated with the plurality of objects according to the image data provided by the plurality of image sensors; and

steering, by the processing system, a microphone array towards one of the plurality of objects according to the directional information to detect speech signals generated by the plurality of objects.

3. The method of claim 2, wherein the steering of the microphone array comprises steering, by the processing system, a plurality of beams of the microphone array according to a beamforming process.

4. The method of claim 3, further comprising using, by the processing system, the image data provided by the plurality of image sensors to assign a beam of the plurality of beams to each of the plurality of objects.

5. The method of claim 2, further comprising:

tracking, by the processing system, movements of the plurality of objects according to the image data provided by the plurality of image sensors;

steering, by the processing system, the plurality of image sensors according to the movements of the plurality of objects; and

steering, by the processing system, the microphone array according to the movements of the plurality of objects.

6. The method of claim 1, further comprising determining a quality of performance for the first command for controlling the media center according to the characteristic of the first gesture that is determined.

7. The method of claim 6, wherein the quality of performance of the modified command comprises a magnitude of performance.

8. The method of claim 1, further comprising selecting, by the processing system, the first command provided by the first object based on a conflict resolution strategy, wherein the conflict resolution strategy comprises assigning a first priority to the first object and assigning a second priority to the second object, and wherein selecting the first command comprises selecting the first command based on the first priority having a higher priority than the second priority.

9. The method of claim 1, wherein responsive to a second determination that the response indicates the resolution is to perform the second command processing, by the processing system, the second command to control the media center responsive to determining the response indicates the resolution is to perform the second command.

10. The method of claim 1, further comprising identifying, by the processing system, each of the plurality of objects by comparing the image data with biometric profiles.

11. The method of claim 10, wherein the biometric profiles comprise one of facial data, body contour data, body color data, body surface data, or combinations thereof.

12. The method of claim 1, wherein the characteristic of the first gesture comprises a speed of the first gesture.

13. The method of claim 1, further comprising:

assigning, by the processing system, one of an acoustic model, a visual model, or a combination thereof to each of the plurality of objects; and

processing, by the processing system, one of audible cues, visual cues, or a combination thereof generated by the plurality of objects according to the acoustic model, the visual model or both assigned to each of the plurality of objects.

14. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processing system comprising a processor, facilitate performance of operations, comprising:

detecting a first gesture of a first object of a plurality of objects captured by one of a plurality of cameras as image data for controlling a media center, wherein the first gesture is associated with a first command;

detecting a second gesture of a second object of the plurality of objects based on the image data from one of the plurality of cameras, wherein the second gesture is associated with a second command for controlling the media center;

determining a conflict between the first command and the second command;

presenting a notification indicating the conflict and requesting a resolution to the conflict;

determining if a response to the notification indicates the resolution is to perform the first command;

modifying the first command according to a quality of performance to reflect a characteristic of the first gesture to generate a modified command responsive to determining the response indicates the resolution is to perform the first command; and

controlling the media center according to the modified command according to the quality of performance responsive to determining the response indicates the resolution is to perform the first command.

15. The non-transitory computer-readable storage medium of claim 14, wherein the operations further comprise directing a plurality of microphones to steer a plurality of beams to each of the plurality of objects, and wherein the characteristic of the first gesture comprises speed of the first gesture, duration of the first gesture, or any combination thereof.

16. The non-transitory computer-readable storage medium of claim 14, wherein the processor further performs operations comprising:

selecting the first command according to a conflict resolution strategy, and

wherein the quality of performance of the modified command comprises a magnitude of performance, a speed of performance, or any combination thereof.

17. A system, comprising:

a processing system including a processor; and

a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, comprising;

detecting a first gesture of a first object of a plurality of objects based on image data captured by a plurality of image sensors, wherein the first gesture is associated with a first command for controlling a media center;

detecting a second gesture of a second object of the plurality of objects based on images captured by the plurality of image sensors, wherein the second gesture is associated with a second command for controlling the media center;

determining a conflict between the first command and the second command of a second object of the plurality of objects;

presenting a notification requesting a resolution to the conflict;

determining if a response to the notification indicates the resolution is to perform the first command;

modifying the first command according to a quality of performance to reflect a characteristic of the first gesture to generate a modified command responsive to determining the response indicates the resolution is to perform the first command; and

controlling the media center to execute the modified command according to the quality of performance responsive to determining the response indicates the resolution is to perform the first command.

18. The system of claim 17, wherein the operations further comprise:

identifying the plurality of objects according to image data provided by the plurality of image sensors; and

exchanging messages with one of the plurality of objects that generated the first command, wherein the messages comprise one of audible speech, visual messages presented by way of a display device coupled to the processor, or a combination thereof.

19. The system of claim 17, wherein the operations further comprise controlling the media center to execute the second command responsive to determining the response indicates the resolution is to perform the second command.

20. The system of claim 17, operations further comprise:

tracking movements of the plurality of objects according to the image data provided by the plurality of image sensors;

steering the plurality of image sensors according to the movements of the plurality of objects; and

steering a microphone array according to the movements of the plurality of objects.