Electronic apparatus for providing health status information, method of controlling the same, and computer-readable storage medium

• US 10,366,487 B2
• Filed: 03/16/2015
• Issued: 07/30/2019
• Est. Priority Date: 03/14/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Seon-ae
• Primary Examiner(s): Walsh, Daniel I
• Application Number:
  US14/658,666
• Publication Number:
  US 20150261996A1
• Time in Patent Office: 1,597 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/634 Warning indications
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Provided is a device and method of providing health status information. The device includes: a device including: a storage configured to store a first image including a face of a user and first health status information extracted from the first image; an imager configured to capture an image; a controller configured to control the imager to capture a second image including the face of the user and to extract second health status information from the captured second image; and a display configured to output the second image and information other than the stored first health status information from among the extracted second health status information.

Claims

1. A device comprising:

a computer readable storage storing instructions; and

at least one processor configured to execute the stored instructions to;

extract a plurality of pieces of first health status information from a plurality of first image files of a user, each image file including a face of the user and previously stored in the storage;

store the plurality of pieces of first health status information in the storage;

capture a second image including the face of the user and second image data;

store the second image as a second image file;

extract second health status information from the captured second image;

store the second health status information in the storage;

automatically determine one first image file, from among the plurality of the first image files, as having first image data having a value of a characteristic corresponding most closely to a value of the characteristic of the second image data of the captured second image, wherein the characteristic corresponds to a photographing circumstance at a time of capture of the second image comprising at least one of lighting, a brightness of lighting, a direction of lighting, a color temperature of lighting, a place, a background, and a time zone;

select the one first image file as a reference image file for determining the health status information of the user; and

determine the health status information of the user by outputting the second image and a difference between a piece of first health status information extracted from the reference image file and the second health status information on a display of the device.

2. The device of claim 1, wherein, the processor is configured to execute the stored instructions to:

capture the second image when a first application is executed from among a plurality of applications in the device; and

output the second image and the difference between the piece of the first health status information and the second health status information when a second application that is different from the first application is executed.

3. The device of claim 1,

wherein the processor is configured to execute the stored instructions to capture the second image and to extract the second health status information from the captured second image based on receiving a user input for unlocking the device which is in a locked state.

4. The device of claim 1,

wherein the processor is configured to execute the stored instructions to capture the second image and extract the second health status information from the captured second image based on receiving a user input for executing a video call application in the device.

5. The device of claim 1, wherein the plurality of the first images are photographs that are received from a server in which the user is registered.

6. The device of claim 1, wherein the processor is configured to execute the stored instructions to:

normalize a resolution of the face of the user in the captured second image by expanding or reducing a size of the second image to a pre-set size; and

extract the second health status information from the normalized second image.

7. The device of claim 1, wherein the processor is configured to execute the stored instructions to:

obtain a color temperature of illumination on the face of the user at a point of time when the second image is captured;

normalize a hue value of the face of the user by adjusting a hue value of the second image based on the obtained color temperature; and

extract the second health status information from the normalized second image.

8. The device of claim 1, wherein the processor is further configured to execute the stored instructions to output an indicator indicating which region in the second image is related to the difference between the piece of the first health status information and the second health status information.

9. The device of claim 1, wherein to output, in chronological order, a plurality of second images that are captured during a predetermined time period and health status information extracted from the plurality of second images.

10. The device of claim 1, wherein the processor is configured to execute the stored instructions to output photographing guide information for guiding capturing of the face of the user based on a pre-set face image obtaining condition, and the processor is configured to execute the stored instructions.

11. The device of claim 1, wherein the processor is configured to execute the stored instructions to:

obtain bio-information of the user at a point of time when the second image is captured;

determine a biological condition of the user at the point of time based on the obtained bio-information; and

extract the second health status information based on the determined biological condition.

12. A method of providing health status information, by a device, comprising a computer readable storage storing instructions, and a processor configured to perform the method, the method comprising:

extracting a plurality of pieces of first health status information from a plurality of first image files of a user, each image file including a face of the user and previously stored in the storage;

storing the plurality of pieces of first health status information in the storage;

capturing a second image including the face of the user and second image data;

storing the second image as a second image file;

extracting second health status information from the captured second image;

storing the second health status information in the storage;

automatically determining one first image file, from among the plurality of the first image files, as having first image data having a value of a characteristic corresponding most closely to a value of the characteristic of the second image data of the captured second image, wherein the characteristic corresponds to a photographing circumstance at a time of capture of the second image comprising at least one of lighting, a brightness of lighting, a direction of lighting, a color temperature of lighting, a place, a background, and a time zone;

selecting the one first image file as a reference image file for determining the health status information of the user; and

determining the health status information of the user by outputting the second image and a difference between a piece of first health status information extracted from the reference image file and the second health status information on a display of the device.

13. The method of claim 12, wherein:

the capturing the second image comprises, capture the second image when a first application is executed from among a plurality of applications in the device; and

the outputting the second image and the difference comprises, controlling to output the second image and the difference between the piece of the first health status information and the second health status information when a second application that is different from the first application is executed.

14. The method of claim 12,

wherein the capturing the second image comprises controlling to capture the second image based on receiving a user input for unlocking the device which is in a locked state.

15. The method of claim 12,

wherein the capturing the second image comprises, controlling to capture the second image based on receiving a user input for executing a video call application in the device.

16. The method of claim 12, wherein the plurality of first images are photographs that are received from a server in which the user is registered.

17. The method of claim 12, wherein the controlling to extract the second health status information comprises:

normalizing a resolution of the face of the user in the second image by expanding or reducing a size of the second image to a pre-set size; and

extracting the second health status information from the normalized second image.

18. The method of claim 12, wherein the controlling to extract the second health status information comprises:

obtaining a color temperature of illumination on the face of the user at a point of time when the second image is captured;

normalizing a hue value of the face of the user by adjusting a hue value of the second image based on the obtained color temperature; and

extracting the second health status information from the normalized second image.

19. The method of claim 12, further comprising outputting an indicator indicating which region in the second image is related to the difference between the piece of the first health status information and the second health status information.

20. The method of claim 12, wherein the controlling to output the second image and the difference comprises outputting, in chronological order, a plurality of the second images that are captured during a predetermined time period and health status information extracted from the plurality of second images.

21. The method of claim 12, wherein the controlling to capture the second image comprises outputting photographing guide information for guiding capturing of the face of the user based on a pre-set face image obtaining condition, and the second image is an image satisfying the pre-set face image obtaining condition.

22. The method of claim 12,

wherein the extracting the second health status information comprises:

obtaining bio-information of the user at a point of time when the second image is captured;

determining a biological condition of the user at the point of time based on the detected bio-information; and

extracting the second health status information based on the determined biological condition.

23. A non-transitory computer-readable recording medium having recorded thereon a program executable by a computer for performing the method of claim 12.