Initializing camera subsystem for face detection based on sensor inputs

• US 9,596,084 B2
• Filed: 09/09/2011
• Issued: 03/14/2017
• Est. Priority Date: 09/09/2011
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Papakipos, Matthew Nicholas, Garcia, David Harry
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): Trehan, Akshay
• Application Number:
  US13/229,327
• Publication Number:
  US 20130063611A1
• Time in Patent Office: 2,013 Days
• Field of Search: 348/207.11, 348/333.13, 348/372, 455/410, 455/411, 455/418, 455/566
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 21/30 Authentication, i.e. establishing the identity or authorisation of security principals
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 63/107 wherein the security policies are location-dependent, e.g. entities privileges depend on current location or allowing specific operations only from locally connected terminals
• CPC Class Codes: H04L 9/32 including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
• CPC Class Codes: H04L 9/3231 Biological data, e.g. fingerprint, voice or retina network architectures or network communication protocols for supporting authentication of entities using biometrical features in a packet data network H04L63/0861
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

In one embodiment, while a mobile electronic device is in a first operation state, it receives sensor data from one or more sensors of the mobile electronic device. The mobile electronic device in a locked state analyzes the sensor data to estimate whether an unlock operation is imminent, and in response to a positive determination, initializes the camera subsystem so that the camera is ready to capture a face as soon as the user directs the camera lens to his or her face. In particular embodiments, the captured image is utilized by a facial recognition algorithm to determine whether the user is authorized to use the mobile device. In particular embodiments, the captured facial recognition image may be leveraged for use on a social network.

Claims

1. A method comprising:

by a computing device, monitoring one or more sensors, wherein the computing device is in a locked state, and the computing device is associated with an authorized user of a social-networking system;

by the computing device, in response to detecting input from at least one of the one or more sensors, determining, based on the input, whether an unlock operation is imminent;

by the computing device, upon determining that the unlock operation is imminent;

identifying, based on information associated with the authorized user, content to be downloaded from the social-networking system, and

downloading the identified content from the social-networking system, wherein the identified content is stored for display after the computing device is unlocked;

by the computing device, receiving a request to unlock the device, the request comprising authentication data and

by the computing device, determining whether to exit the locked state based at least in part on the authentication data and data associated with the authorized user in the social-networking system.

2. The method of claim 1, wherein:

the one or more sensors comprise at least an accelerometer, and

detecting input comprises detecting acceleration above a predetermined threshold.

3. The method of claim 1, wherein:

the information associated with the authorized user comprises the authorized user'"'"'s location, and downloading the identified content comprises downloading content for one or more points of interest that are identified based on the authorized user'"'"'s location.

4. The method of claim 1, wherein:

the one or more sensors comprise at least a light sensor, and

detecting input comprises detecting a transition from low to high light levels.

5. The method of claim 1, wherein:

the one or more sensors comprise at least a microphone, and

detecting input comprises detecting a transition from low audio amplitude to high audio amplitude.

6. The method of claim 1, wherein:

the one or more sensors comprise at least a microphone, and

detecting input comprises detecting a transition from muffled to clear audio.

7. The method of claim 1, wherein:

the one or more sensors comprise at least a touch sensor, and

detecting input comprises detecting a touch.

8. The method of claim 1, wherein:

the one or more sensors comprise at least a proximity sensor, and detecting input comprises detecting a transition from a detection of a nearby object to an absence of a nearby object.

9. The method of claim 1, wherein the information associated with the authorized user comprises information indicating that the authorized user habitually checks certain news sites, and downloading the identified content comprises downloading newsfeed information from the social-networking system.

10. The method of claim 1, wherein determining whether to exit the locked state comprises comparing the authentication data to the data associated with the authorized user.

11. The method of claim 10, further comprising:

by the computing device, unlocking the computing device when the authentication data matches the data associated with the authorized user.

12. The method of claim 10, further comprising:

by the computing device, prompting a device user to provide a password or PIN to unlock the computing device when the authentication data does not match the data associated with the authorized user.

13. The method of claim 10, wherein the authentication data comprises at least one digital image captured by a camera of the computing device, the data associated with the authorized user comprises an image representing a face of the authorized user, and comparing the authentication data to the data associated with the authorized user comprises:

determining whether the at least one digital image matches the image representing the face of the authorized user.

14. The method of claim 13, wherein the determining whether the at least one digital image matches the image representing the face of the authorized user comprises:

locating, using one or more facial recognition algorithms, a device user'"'"'s face in the at least one digital image, and

the at least one digital image matches the image representing the face of the authorized user when there is a sufficient amount of similarity between the device user'"'"'s face in the at least one digital image and the image representing the face of the authorized user.

15. The method of claim 13, further comprising:

by the computing device, unlocking the device when the at least one digital image matches the image representing the face of the authorized user.

16. The method of claim 13, further comprising:

by the computing device, after the computing device is unlocked, uploading the at least one digital image to the social network as a profile picture associated with the authorized user.

17. The method of claim 1, further comprising:

by the computing device, after the computing device is unlocked, displaying the identified content.

18. One or more non-transitory, computer-readable media comprising instructions operable, when executed by a computing device, to:

at the computing device, monitor one or more sensors, wherein the computing device is in a locked state and the computing device is associated with an authorized user of a social-networking system;

at the computing device, in response to detecting input from at least one of the one or more sensors, determine, based on the input, whether an unlock operation is imminent;

at the computing device, upon determining that the unlock operation is imminent;

identify, based on information associated with the authorized user, content to be downloaded from the social-networking system, and

download the identified content from the social-networking system, wherein the identified content is stored for display after the computing device is unlocked;

at the computing device, receive a request to unlock the device, the request comprising authentication data; and

at the computing device, determine whether to exit the locked state based at least in part on the authentication data and data associated with the authorized user in the social-networking system.

19. An apparatus comprising:

one or more sensors;

one or more non-transitory, computer-readable media comprising instructions operable, when executed by a computing device, to;

at the computing device, monitor one or more sensors, wherein the computing device is in a locked state and the computing device is associated with an authorized user of a social-networking system;

at the computing device, in response to detecting input from at least one of the one or more sensors, determine, based on the input, whether an unlock operation is imminent;

at the computing device, upon determining that the unlock operation is imminent;

identify, based on information associated with the authorized user, content to be downloaded from the social-networking system, and

download the identified content from the social-networking system, wherein the identified content is stored for display after the computing device is unlocked;

at the computing device, receive a request to unlock the device, the request comprising authentication data and

at the computing device, determine whether to exit the locked state based at least in part on the authentication data and data associated with the authorized user in the social-networking system.

20. The apparatus of claim 19, wherein:

the information associated with the authorized user comprises the authorized user'"'"'s location, and downloading the identified content comprises downloading content for one or more points of interest that are identified based on the authorized user'"'"'s location.