System and method for detecting user attention

• US 20070061851A1
• Filed: 03/06/2006
• Published: 03/15/2007
• Est. Priority Date: 09/15/2005
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Chen, Ruxin, Deshpande, Hrishikesh
• Granted Patent: US 8,645,985 B2
• Time in Patent Office: Days
• US Class Current: 725/88
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/49 Saving the game status; Pausing or ending the game
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/636 involving process of starting or resuming a game
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4333 Processing operations in response to a pause request
• CPC Class Codes: H04N 21/43632 involving a wired protocol, e.g. IEEE 1394 high-speed IEEE 1394 serial bus H04L12/40052
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4415 : using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861
• CPC Class Codes: H04N 21/44218 : Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

A system and method for conditioning execution of a control function on a determination of whether or not a person'"'"'s attention is directed toward a predetermined device. The method involves acquiring data concerning the activity of a person who is in the proximity of the device, the data being in the form of one or more temporal samples. One or more of the temporal samples is then analyzed to determine if the person'"'"'s activity during the time of the analyzed samples indicates that the person'"'"'s attention is not directed toward the device. The results of the determination are used to ascertain whether or not the control function should be performed.

Claims

1. A system for pausing video being displayed on a video display screen, comprising:

at least one sensor for acquiring data concerning the activity of a person who is in the proximity of the display screen, the data being in the form of one or more temporal samples; and

a processor for analyzing one or more of the temporal samples to determine if the person'"'"'s activity during the time of the analyzed samples indicates that the person'"'"'s attention is not directed toward the video display screen, and for automatically pausing the video being displayed on the screen when it is determined that the person'"'"'s attention is not directed toward the screen.

2. The system according to claim 1, wherein the at least one sensor includes a microphone.

3. The system according to claim 2, wherein the data is digital audio samples.

4. The system according to claim 3, wherein the processor analyzes the spectral content of the data.

5. The system according to claim 4, the processor determines whether or not the data includes a human voice fundamental frequency in the range of 60 Hz to 350 Hz.

6. The system according to claim 3, wherein the processor applies one or more previously trained acoustic models to the data to detect whether or not the data indicates the presence of a human voice.

7. The system according to claim 1, wherein the at least one sensor includes a camera.

8. The system according to claim 7, wherein the temporal samples are pixel arrays, each temporal sample being an array of pixels that represents an image.

9. The system according to claim 8, wherein the processor analyzes at least one of the pixel arrays to determine whether or not the image corresponding to the pixel array includes a frontal face.

10. The system according to claim 9, wherein the processor analyzes at least one of the pixel arrays by selectively applying an intensive frontal face detection algorithm and a less intensive frontal face detection algorithm.

11. The system according to claim 1, wherein the video display screen is used to display video associated with a video game, and wherein the game is paused when it is determined that the person'"'"'s attention is not directed toward the video display screen.

12. A method for pausing video being displayed on a video display screen, comprising the steps of:

acquiring data concerning the activity of a person who is in the proximity of the display screen, the data being in the form of one or more temporal samples;

analyzing one or more of the temporal samples to determine if the person'"'"'s activity during the time of the analyzed samples indicates that the person'"'"'s attention is not directed toward the video display screen; and

automatically pausing the video being displayed on the screen when it is determined that the person'"'"'s attention is not directed toward the screen.

13. The method according to claim 12, wherein the step of acquiring data comprises the step of acquiring data through use of a microphone.

14. The method according to claim 13, wherein the step of acquiring data comprises the step of acquiring plurality of digital audio samples.

15. The method according to claim 14, wherein the step of analyzing comprises the step of analyzing the spectral content of the data.

16. The method according to claim 15, wherein the step of analyzing includes the step of determining whether or not the data includes a human voice fundamental frequency in the range of 60 Hz to 350 Hz.

17. The system according to claim 12, wherein the step of analyzing includes the step of applying one or more previously trained acoustic models to the data to detect whether or not the data indicates the presence of a human voice.

18. The method according to claim 12, wherein the step of acquiring data comprises the step of acquiring data through use of a camera.

19. The method according to claim 18, wherein the step of acquiring data comprises the step of acquiring a plurality of pixel arrays representing images.

20. The method according to claim 19, wherein the step of analyzing comprises the step of analyzing at least one of the pixel arrays to determine whether or not an image corresponding to the pixel array includes a frontal face.

21. The method according to claim 20, wherein the step of analyzing includes selectively applying an intensive frontal face detection algorithm and a less intensive frontal face detection algorithm.

22. A system for pausing a video game, comprising:

at least one sensor for acquiring data concerning the activity of a person playing the video game, the data being in the form of one or more temporal samples; and

a processor for analyzing one or more of the temporal samples to determine if the person'"'"'s activity during the time of the analyzed samples indicates that the person'"'"'s attention is not directed toward the video display screen, and for automatically pausing the video game when it is determined that the person'"'"'s attention is not directed toward the screen.

23. The system according to claim 22, wherein the at least one sensor includes a microphone.

24. The system according to claim 22, wherein the at least one sensor includes a camera.

25. A system for conditioning execution of a control function on a determination of whether or not a person'"'"'s attention is directed toward a predetermined device, comprising:

at least one sensor for acquiring data concerning the activity of a person who is in the proximity of the device, the data being in the form of one or more temporal samples; and

a processor for analyzing one or more of the temporal samples to make a determination as to whether or not the person'"'"'s activity during the time of the analyzed samples indicates that the person'"'"'s attention is not directed toward the device, and for selectively executing the control function based on the determination.

26. The system according to claim 25, wherein execution of the control function is enabled only when the person'"'"'s attention is directed toward the device.

27. The system according to claim 26, wherein the control function is a function executed in response to a voice command.

28. The system according to claim 26, wherein the device is a display screen.

29. The system according to claim 25, wherein execution of the control function is enabled only when the person'"'"'s attention is not directed toward the device

30. The system according to claim 29, wherein the control function is a function executed in response to a voice command.

31. The system according to claim 29, wherein the device is a display screen.