Facial cues as commands

• US 9,148,537 B1
• Filed: 05/18/2012
• Issued: 09/29/2015
• Est. Priority Date: 05/18/2012
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: Hopto Inc.
• Inventors: Currey, Robert W.
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): Camargo, Marly
• Application Number:
  US13/475,926
• Time in Patent Office: 1,229 Days
• Field of Search: 348/222.1, 348/78, 348/211.99, 348/211.4, 348/239, 34833301-33312, 348/333.03, 382/115, 382/117, 382/118, 345/156, 715/863, 340/5.52, 340/5.82, 340/5.83, 340/5.53, 396/151, 396/121, 396/147, 396/51
• US Class Current: 1/1
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 1/00458 Sequential viewing of a plurality of images, e.g. browsing or scrolling
• CPC Class Codes: H04N 23/61 based on recognised objects

Abstract

Information regarding facial cues may be received. Such information may associated each facial cue with a responsive action. Images (e.g., photo, video) of an area in front of a computing device may be captured by a front-facing camera. A user'"'"'s face may be detected, as well as determined to exhibit a facial cue. The facial cue is identified, along with its associated action. The computing device may then perform the action associated with the facial cue exhibited by the user.

Claims

1. A method for using facial cues, the method comprising:

storing information in memory regarding one or more predetermined facial cues, wherein each facial cue is associated with an action;

capturing images regarding an area in front of a computing device having a display screen and a front-facing camera;

executing instructions stored in memory, wherein execution of the instructions by a processor;

detects when a face of a user appears within a predetermined distance from the front of the computing device,

determines that the face of the user exhibits a facial cue, wherein the facial cue comprises a line-of-sight from eyes of the user directed toward a defined portion of the display screen, wherein a size or a location of the defined portion is defined and adjustable by the user,

identifies the action associated with the line-of-sight facial cue exhibited by the face of the user and with the associated defined portion of the display screen, and

performs the identified action associated with the exhibited line-of-sight facial cue and the defined portion of the display screen.

2. The method of claim 1, wherein the action associated with the line-of-sight to the portion of the display screen is scrolling towards the portion.

3. The method of claim 2, wherein the scrolling comprises vertical scrolling.

4. The method of claim 2, wherein the scrolling comprises horizontal scrolling.

5. The method of claim 1, further comprising defining a plurality of different portions of the display screen.

6. The method of claim 5, further comprising receiving adjustment input from the user, wherein the size or the location of the portion is adjusted in accordance with the adjustment input from the user.

7. The method of claim 5, wherein lines-of-sight to each portion are associated with a different specified scrolling action.

8. The method of claim 1, wherein determining that the face of the user exhibits a facial cue comprises tracking eye movement and line-of-sight.

18. The method of claim 1, wherein the action associated with the line-of-sight facial cue is designated by the user.

9. A system for using facial cues, the system comprising:

memory that stores information regarding one or more predetermined facial cues, wherein each facial cue is associated with an action;

a display screen having a plurality of defined portions, wherein a size or a location of at least one of the defined portions is defined and adjustable by the user;

a front-facing camera that captures images regarding an area in front of a computing device;

a processor that executes instructions stored in memory, wherein execution of the instructions;

detects when a face of a user appears within a predetermined distance from the front of the computing device,

determines that the face of the user exhibits a facial cue, wherein the facial cue comprises a line-of-sight from eyes of the user directed toward one of the defined portions in the display screen,

identifies the action associated with the line-of-sight facial cue exhibited by the face of the user and with the associated defined portion of the display screen, and

performs the identified action associated with the exhibited line-of-sight facial cue and the associated defined portion of the display screen.

10. The system of claim 9, wherein the action associated with the line-of-sight to the portion of the display screen is scrolling towards the portion.

11. The system of claim 10, wherein the scrolling comprises vertical scrolling.

12. The system of claim 10, wherein the scrolling comprises horizontal scrolling.

13. The system of claim 9, further comprising defining a plurality of different portions of the display screen based on user input.

14. The system of claim 13, further comprising a user interface that receives adjustment input from the user, wherein the size or the location of one of the portions is adjusted in accordance with the adjustment input from the user.

15. The system of claim 13, wherein lines-of-sight to each portion are associated with a different specified scrolling action.

16. The system of claim 9, wherein determining that the face of the user exhibits a facial cue comprises tracking eye movement and line-of-sight.

19. The system of claim 9, wherein the action associated with the line-of-sight facial cue is designated by the user.

17. A non-transitory computer-readable storage medium, having embodied thereon a program executable by a processor to perform a method for using facial cues, the method comprising:

storing information regarding one or more predetermined facial cues, wherein each facial cue is associated with an action;

capturing images regarding an area in front of a computing device having a display screen and a front-facing camera;

detecting when a face of a user appears within a predetermined distance from the front of the computing device;

determining that the face of the user exhibits a facial cue, wherein the facial cue comprises a line-of-sight from eyes of the user directed toward a defined portion of the display screen, wherein a size or a location of the defined portion is defined and adjustable by the user;

identifying the action associated with the line-of-sight facial cue exhibited by the face of the user and with the associated defined portion of the display screen; and

performing the identified action associated with the exhibited line-of-sight facial cue and with the associated defined portion of the display screen.

20. The non-transitory computer-readable storage medium of claim 17, wherein the action associated with the line-of-sight facial cue is designated by the user.