Facial recognition display control method and apparatus

• US 10,185,391 B2
• Filed: 03/13/2014
• Issued: 01/22/2019
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Wooyong, Han, Gyuhee, Park, Kwangkyu, Jeon, Seungjoon
• Primary Examiner(s): Ghebretinsae, Temesghen
• Assistant Examiner(s): Cohen, Yaron
• Application Number:
  US14/209,399
• Publication Number:
  US 20140267021A1
• Time in Patent Office: 1,776 Days
• CPC Class Codes: G06F 1/3212 Monitoring battery levels, e.g. power saving mode being initiated when battery voltage goes below a certain level
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A screen display control method is provided for controlling screen display of an electronic device more efficiently. A screen display control method includes determining a resolution of a camera, recognizing a face of a user using the camera operating at the determined resolution, and controlling screen display of the electronic device based on the face.

Claims

1. A screen display control method of an electronic device, the method comprising:

determining a resolution operation mode of a camera, the camera being configured to operate in a high resolution mode by activating all pixels of an image sensor of the camera and a low resolution mode by activating a predetermined number of pixels less than the all pixels of the image sensor of the camera;

performing a first facial recognition operation when it is determined that the camera is operating in the high resolution mode;

performing a second facial recognition operation when it is determined that the camera is operating in the low resolution mode; and

controlling a screen display of the electronic device based on the first facial recognition operation or the second facial recognition operation,

wherein the first facial recognition operation includes determining a gaze position of a user,

wherein the second facial recognition operation includes;

setting first configuration values of the camera,

determining whether the user is present,

when the user is present, changing the first configuration values of the camera to a second configuration value, and

determining a face position of the user with respect to the electronic device while operating the camera in the low resolution mode with the second configuration value, and

wherein the changing of the first configuration values of the camera to the second configuration value comprises changing at least one of frames per second, a shutter speed, or an exposure value.

2. The method of claim 1, further comprising:

determining a power state of the electronic device; and

selecting the high resolution mode or the low resolution mode based on the determined power state of the electronic device.

3. The method of claim 2, wherein the determining of the power state comprises:

determining, when the power state of the electronic device is equal to or greater than a threshold or the electronic device is connected to an external device, the power state as a high power state; and

determining, when the power state of the electronic device is less than the threshold value and the electronic device is not connected to the external device, the power state as a low power state.

4. The method of claim 3,

wherein the high resolution mode is selected when the electronic device is in the high power state, and

wherein the low resolution mode is selected when the electronic device is in the low power state.

5. The method of claim 1, when the screen display is controlled based on the gaze position, the method further comprising:

acquiring coordinate information by mapping the gaze position to a display of the electronic device; and

controlling the screen display further based on the coordinate information.

6. The method of claim 5, further comprising:

determining at least one of direction information or speed information based on a change of the coordinate information over time; and

controlling the screen display further based on the direction information or the speed information.

7. The method of claim 1, further comprising:

acquiring sensor data using at least one sensor;

acquiring coordinate information through mapping to the display of the electronic device based on the sensor data and the face position;

determining at least one of direction information and speed information based on a change of the coordinate information; and

controlling the screen display further based on the at least one of the direction information and the speed information.

8. The method of claim 1, wherein, in the high resolution mode, the method further comprising:

acquiring sensor data from a sensor;

assigning priorities to image information provided by the camera and the sensor data; and

controlling the screen display further using the image information sorted in a descending order of the priorities.

9. The method of claim 1, wherein the controlling of the screen display comprises controlling at least one of scroll, playback, pause, zoom-in, zoom-out, stay, and response control operations in association with the screen display.

10. An electronic device comprising:

a camera configured to operate in a plurality of resolution operation modes, the plurality of resolution operation modes including a high resolution mode based on an activation of all pixels of an image sensor of the camera and a low resolution mode based on an activation of a predetermined number of pixels less than the all pixels of the image sensor of the camera;

a display configured to display a screen; and

at least one processor configured to;

determine the resolution operation mode of the camera,

perform a first facial recognition operation when it is determined that the camera is operating in the high resolution mode,

perform a second facial recognition operation when it is determined that the camera is operating in the low resolution mode, and

control the display based on the first facial recognition operation or the second facial recognition operation,

wherein the first facial recognition operation includes determining a gaze position of a user,

wherein the second facial recognition operation includes;

setting first configuration values of the camera,

determining whether the user is present,

when the user is present, changing the first configuration values of the camera to a second configuration value, and

determining a face position of the user with respect to the electronic device while operating the camera in the low resolution mode with the second configuration value, and

wherein the changing of the first configuration values of the camera to the second configuration value comprises changing at least one of frames per second, a shutter speed or an exposure value.

11. The electronic device of claim 10, wherein the at least one processor is further configured to:

determine a power state of the electronic device as one of a high power state and a low power state based on whether power of the electronic device is equal to or greater than a threshold or whether the electronic device is connected to an external device, and

select the high resolution mode of the camera when the electronic device is in the high power state and the low resolution mode when the electronic device is in the low power state.

12. The electronic device of claim 10, wherein the at least one processor is further configured to:

operate the camera in the high resolution mode to acquire coordinate information by mapping the gaze position to the display of the electronic device, and

control the screen further based on the coordinate information.

13. The electronic device of claim 10, further comprising:

at least one sensor configured to provide sensor data,

wherein the at least one processor is further configured to;

acquire coordinate information through mapping to the display of the electronic device based on the sensor data and the face position,

determine at least one of direction information or speed information based on change of the coordinate information, and

control the screen further based on the at least one of the direction or speed information.

14. The electronic device of claim 10, wherein, in the high resolution mode, the electronic device further comprising:

at least one sensor configured to provide sensor data,

wherein the at least one processor is further configured to;

assign priorities to image information provided by the camera and the sensor data, and

control the screen further using the image information sorted in a descending order of the priorities.

15. The electronic device of claim 10, wherein the at least one processor is further configured to control at least one of scroll, playback, pause, zoom-in, zoom-out, stay, and response control operations in association with the screen.