Personal computing device control using face detection and recognition

• US 20090175509A1
• Filed: 03/06/2008
• Published: 07/09/2009
• Est. Priority Date: 01/03/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Gonion, Jeff, Kerr, Duncan Robert
• Granted Patent: US 8,600,120 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/629 to features or functions of an application
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/164 using holistic features
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Systems and methods are provided for control of a personal computing device based on user face detection and recognition techniques.

Claims

1. A personal computing device comprising:

a user interface for i) generating one or more user information outputs and ii) receiving one or more user information inputs,

an image sensor for capturing one or more images, and

a processor for i) detecting one or more faces in the one or more images and ii) controlling at least one of the generation of the one or more user information outputs and the receiving of the one or more user information inputs in response to detecting the one or more faces.

2. The device of claim 1, wherein the controlling is in response to identifying the one or more detected faces in the one or more images.

3. The device of claim 2, wherein identifying includes determining that at least one of the one or more detected faces in the one or more images is associated with an authorized user of the device.

4. The device of claim 1, wherein detecting includes employing a pattern recognition algorithm.

5. The device of claim 3, wherein the pattern recognition algorithm includes a statistical model.

6. The device of claim 1, wherein detecting includes employing at least one of face detection, face recognition, object recognition, view-based recognition, computer vision, and machine learning.

7. The device of claim 5, wherein face detection includes at least one of knowledge-based methods, feature invariant approaches, template matching methods, and appearance-based methods.

8. The device of claim 1, wherein a user information output is derived from at least one of a displaying one or more images, displaying video, displaying text, outputting audio, moving a portion of the device, and vibrating a portion of the device.

9. The device of claim 1, wherein controlling includes continuing the displaying of the one of more images for a period of time after detecting the one or more faces.

10. The device of claim 1, wherein controlling includes inhibiting the initiation of a screen saver application.

11. The device of claim 1, wherein user information input is derived from at least one of the user typing words on a keyboard, saying words, scrolling through a list, using a mouse pointer to select an element, and pressing one or more control buttons.

12. The device of claim 1, wherein controlling includes controlling the operation of one or more applications of the device.

13. The device of claim 12, wherein the operation includes at least one of logging into or out of an application, starting or launching one or more applications, stopping or ending one or more applications, selecting or de-selecting one or more elements, increasing or decreasing one or more settings, moving through a list of elements, initiating or ending a communications session, playing music or video, pausing music or video, and initiating or ending an audio or video recording session.

14. The device of claim 13, wherein the element includes at least one of a song, a video, a music file, an audio file, a video file, a photograph, a media file, an application icon, an activation icon, a control button, a data file, and contact data.

15. The device of claim 1, wherein the image sensor includes a camera.

16. The device of claim 15, wherein camera is integrated with an image display of the device.

17. The device of claim 1, wherein the personal computing device includes at least one of a personal computer, a portable computer, a cellular telephone, a wireless communications device, a media player, an MP3 player, a video player, and a PDA.

18. A personal media device comprising:

an image sensor for capturing one or more images and generating associated image data,

a data store including face detection data associated with at least one of knowledge based face detection, feature invariant based face detection, template matching based face detection, and appearance based face detection, and

a processor for i) receiving the image data ii) receiving the face detection data, iii) detecting the presence of a face in the one or more images by processing the image data and face detection data, and iv) controlling the operation of an application of the device in response to detecting the presence of a face in the one or more images.

19. The device of claim 18, wherein the operation includes at least one of logging into or out of an application, starting or launching one or more applications, stopping or ending one or more applications, selecting or de-selecting one or more elements, increasing or decreasing one or more settings, moving through a list of elements, initiating or ending a communications session, playing music or video, pausing music or video, and initiating or ending an audio or video recording session.

20. A personal media device comprising:

an image sensor for capturing one or more images,

a data store including at least one known face pattern, the at least one known face pattern being associated with an authorized user of the media device, and

a processor for i) detecting one or more faces in the captured one or more images, ii) recognizing at least one of the one or more faces as the face of the authorized user by comparing the one or faces with the data store, and iii) controlling the operation of an application of the device in response to recognizing the at least one of the one or more faces as the face of the authorized user.

21. The device of claim 20, wherein recognizing is based at least in part on a statistical model.

22. A personal communications device comprising:

a transceiver for sending and receiving user communications,

an image sensor for capturing one or more images and generating associated image data,

a data store including face detection data associated with at least one of knowledge based face detection, feature invariant based face detection, template matching based face detection, and appearance based face detection, and

a processor for i) receiving the image data ii) receiving the face detection data, iii) detecting the presence of a face in the one or more images by processing the image data and face detection data, and iv) controlling the sending or receiving of a communication of the device in response to detecting the presence of a face in the one or more images.

23. The device of claim 22, wherein a communication includes at least one of an electronic mail (e-mail) message, instant message, video message, multi-media message, audio message, and user voice call.

24. A personal communications device comprising:

a transceiver for sending and receiving user communications,

an image sensor for capturing one or more images,

a data store including at least one known face pattern, the at least one known face pattern being associated with an authorized user of the media device, and

a processor for i) detecting one or more faces in the captured one or more images, ii) recognizing at least one of the one or more faces as the face of the authorized user by comparing the one or faces with the data store, and iii) controlling the sending or receiving of a communication of the device in response to recognizing the at least one of the one or more faces as the face of the authorized user.

25. The device of claim 24, wherein recognizing is based at least in part on a statistical model.