Scrolling display of electronic program guide utilizing images of user lip movements

• US 8,798,311 B2
• Filed: 01/23/2009
• Issued: 08/05/2014
• Est. Priority Date: 01/23/2009
• Status: Active Grant

• Current Assignee: Dish Technologies LLC (Dish Network Corporation)
• Original Assignee: Eldon Technology Limited (Dish Network Corporation)
• Inventors: Capless, Jonathan
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US12/358,952
• Publication Number:
  US 20100189305A1
• Time in Patent Office: 2,020 Days
• Field of Search: 382/100, 382/181, 382/155
• US Class Current: 382/100
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4821 using a grid, e.g. sorted out by channel and broadcast time

Abstract

Systems and methods of generating device commands based upon spoken user commands are disclosed. An exemplary embodiment captures a series of images of a user of a media device, generates image information corresponding to the series of captured images, determines lip movement of the user from the generated image information, determines at least one spoken user command based upon the determined lip movement of the user, and determines a device command based upon the determined spoken user command. Then, the device command is communicated to, for example, a media presentation device, wherein an operation of the media presentation device is controlled in accordance with the determined spoken user command.

Claims

1. A set top box (STB) configured to generate device commands based upon spoken user commands, comprising:

an image capture device configured to capture a series of images of a user, and configured to generate image information corresponding to the series of captured images;

a memory configured to store electronic program guide (EPG) information;

a tuner configured to receive programming and the EPG information from a remote source; and

a processor system communicatively coupled to the image capture device, the memory, and the tuner,

wherein the processor system is configured to;

generate an electronic program guide (EPG) from the EPG information that is displayed on a display,

receive the image information from the image capture device that includes movement of the user'"'"'s lips,

continuously monitor movement of the user'"'"'s lips in the series of captured images of the user to determine an engagement command, a disengagement command, and at least one STB command associated with scrolling about the displayed EPG, wherein the engagement command, the disengagement command and the at least one STB command are based upon identified lip movements of the user shown in the captured image information,

determine a device command to scroll about the EPG based upon the corresponding determined STB command,

wherein the determined STB command is disregarded until after the engagement command is determined,

wherein the device command associated with the determined STB command is determined after the engagement command is determined, and

wherein a subsequent determined STB command is disregarded after the disengagement command is determined.

2. The STB of claim 1, further comprising:

a presentation device interface coupled to the processor system and at least one media presentation device with the display, wherein the device command controls operation of the EPG displayed by the media presentation device.

3. The STB of claim 1, wherein the image capture device resides as a component within the STB.

4. The STB of claim 1, further comprising:

an external image capture device that is external to the STB, that is communicatively coupled to the processor system, and that is operable to generate image information corresponding to the series of captured images.

5. The STB of claim 1, wherein the at least one STB command is a first STB command based upon first identified lip movements of the user, and wherein the image information from the image capture device is used to determine a second STB user command based upon next identified lip movements of the user, wherein the first identified lip movements are next followed by the next identified lip movements, and wherein the second STB command is compared with the first STB command to verify the first STB command.

6. The STB of claim 1, further comprising:

a power switch controllably coupled to the processor system; and

at least one electrical outlet coupled to the power switch, wherein the electrical outlet is operable to provide power to a home device via a power cord of the home device that is coupled to the electrical outlet,

wherein the processor system is configured to monitor movement of the user'"'"'s lips to determine a home device command,

wherein the home device command corresponds to a request to turn on the home device, and

wherein the processor system causes the power switch to provide power to the electrical outlet in response to determining the home device command.

7. The STB of claim 1, wherein the STB is a video camera.

8. A method for generating device commands to control a set top box (STB) based upon spoken user commands, the method comprising:

displaying an electronic program guide (EPG) on a display, wherein the EPG presents availability of programs that may be presented by the STB;

capturing a series of images of a plurality of users in a media room viewing the displayed EPG;

generating image information corresponding to the series of captured images;

identifying an authorized user from among the plurality of users in the series of captured images;

determining lip movement of the authorized user from the generated image information;

continuously monitoring the lip movement of the authorized user;

determining an engagement command from the continuously monitored lip movement of the authorized user;

determining, only after determining the engagement command, at least one spoken user command associated with scrolling about the displayed EPG based upon the determined lip movement of the user;

determining a STB device command, wherein the STB device command is configured to scroll about the EPG based upon the determined spoken user command; and

scrolling about the EPG in response to determining the STB device command.

9. The method of claim 8, further comprising:

communicating a media presentation device command to a media presentation device, wherein the media presentation device command corresponds to the STB device command, and wherein an operation of the media presentation device is controlled in accordance with the determined STB device command that is based upon the spoken user command.

10. The method of claim 8, wherein the STB is communicatively coupled to a television.

11. The method of claim 8, further comprising:

determining a plurality of lip movements of the plurality of users from the generated image information; and

determining at least one spoken user command based upon the plurality of determined lip movements of the plurality of users,

wherein the STB device command associated with the determined engagement command and the determined at least one spoken user command is determined only when the determined lip movement is from the authorized user.

12. The method of claim 11, further comprising:

identifying at least one of the users from the plurality of users as an unauthorized user; and

disregarding the spoken user command from the identified unauthorized user.

13. The method of claim 8, further comprising:

determining a language that is spoken by the user, wherein the spoken user command based upon the determined lip movement of the user is further based upon the determined language.

14. The method of claim 8, further comprising:

determining that the STB device command is associated with a home device; and

communicating a device command to a power switch, wherein the home device receives power from an electrical outlet controlled by the power switch.

15. The method of claim 8, wherein the determined spoken user command identifies a home device and an action for the home device, and further comprising:

generating a device command based upon the action; and

communicating the device command to the identified home device.

16. The method of claim 8, further comprising:

communicating the STB device command to the media presentation device only after determining the engagement command, wherein an operation of the media presentation device is controlled in accordance with the determined next spoken user command.

17. The method of claim 8, further comprising:

determining a disengagement command based upon the plurality of determined lip movements of the plurality of users,

wherein further STB device commands determined from words spoken by the authorized user are disregarded in response to the disengagement command.

18. The method of claim 8, further comprising:

learning a plurality of spoken user commands from the authorized user, wherein a current spoken user command is determined from the plurality of learned spoken user commands and from the determined lip movement of the authorized user.

19. The method of claim 8, wherein the at least one spoken user command is a first spoken user command based upon first identified lip movements of the authorized user, and further comprising:

capturing a second series of images of the authorized user with an external image capture device;

generating second image information corresponding to the captured second series of images;

determining a second lip movement of the authorized user from the generated second image information; and

verifying the determined first spoken user command based upon the determined second lip movement determined from the generated second image information.

20. The method of claim 8, wherein capturing the series of images of the authorized user is captured by a still image capture device.

21. The method of claim 8, wherein capturing the series of images of the authorized user is captured by a video image capture device, and comprising:

selecting the series of images of the authorized user from a video captured by the video image capture device.

22. A television controller system that generates device commands to control a television based upon spoken user commands, comprising:

a tuner residing in a set top box (STB), and wherein the STB is configured to receive programming and electronic program guide (EPG) information from a remote source;

a display configured to display the received programming and an electronic program guide (EPG) that is generated based upon the received EPG information;

an image capture device configured to capture a series of images of a user, and configured to generate image information corresponding to the series of captured images; and

a processor system communicatively coupled to the image capture device, the display and the tuner,

wherein the processor system is configured to;

generate the EPG from the EPG information,

receive the image information from the image capture device,

determine an engagement command based upon identified lip movements of the user shown in the captured image information,

determine, only after determining the engagement command, at least one spoken user command associated with scrolling about the displayed EPG based upon the identified lip movements of the user shown in the captured image information,

determine, only after determining the engagement command, a disengagement command based upon the identified lip movements of the user shown in the captured image information, and

determine a device command that scrolls about the EPG based upon the determined spoken user command,

wherein the determined device command is disregarded until after the engagement command is determined,

wherein the device command associated with the determined device command is determined after the engagement command is determined, and

wherein a subsequent determined device command is disregarded after the disengagement command is determined.

23. The television controller system of claim 22, wherein the tuner, the image capture device and the processing system reside in the STB that is communicatively coupled to the television.

24. The television controller system of claim 22, further comprising:

a digital video recorder configured to store the received programming for a later presentation on the display.