Apparatus and system for processing diagnostic data on the basis of medical interview data and camera data

• US 10,706,967 B2
• Filed: 04/21/2017
• Issued: 07/07/2020
• Est. Priority Date: 03/09/2017
• Status: Active Grant

• Current Assignee: Healeat, Inc.
• Original Assignee: Partners & Co., Inc.
• Inventors: Chun, Seichul, Choe, Wonjung, Kim, Yeonseop, Kim, Hyunho, Lee, Haebeom
• Primary Examiner(s): Weisenfeld, Aryan E
• Assistant Examiner(s): Elshaer, Alaaeldin M
• Application Number:
  US15/493,140
• Publication Number:
  US 20180260529A1
• Time in Patent Office: 1,173 Days
• Field of Search: 705 2, 705 3, 351178
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 20/10 relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 20/90 relating to alternative medicines, e.g. homeopathy or oriental medicines
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A diagnostic data processing apparatus based on medical interview data and camera data according to the present invention includes: an interview engine for transmitting information on a question about previously provided standardized items to a user terminal or a smart mirror; an analysis engine for receiving medical interview information including information on an answer corresponding to the information on the question from the user terminal or the smart mirror, determining a state of a user by analyzing the medical interview information, and diagnosing the state of the user by analyzing a picture or an image received from the user terminal or the smart mirror; and a proposal engine for proposing complementary and alternative medical contents corresponding to information on a result of the analysis to the user terminal or the smart mirror through the information on the result of the analysis performed by the analysis engine.

Claims

1. A diagnostic data processing method comprising:

pre-storing a group of medical interview questions subdivided into a plurality of subgroups;

selecting only one representative question from each subgroup, transmitting a first questionnaire including the selected representative questions only to a mobile terminal through a mobile network, and determining a provisional state of a user based on first answers received in response to the representative questions, wherein a total number of the selected representative questions included in the first questionnaire is identical to a total number of the plurality of subgroups;

transmitting a second questionnaire including all of the group of medial interview questions including the selected representative questions to the mobile terminal through the mobile network when the provisional state of the user is determined as mibyou based on the first answers, determining a final state of the user based on second answers received in response to all the questions in the group, and performing a first diagnosis by analyzing the second answers;

recognizing a face region from a picture or image received from the mobile terminal, extracting major parts from the face region, and performing a second diagnosis based on the face region and the extracted major parts;

setting a range of a numeric value of a specific color corresponding to a facial skin color to recognize the face region, setting a region showing numeric value recognized within the range as a face candidate region, and applying an ellipsoidal mask operation to the face candidate region; and

transmitting proposed complementary and alternative medical contents corresponding to the first and second diagnosis to the mobile terminal.

2. A diagnostic data processing apparatus comprising:

a processor configured to;

pre-store a group of medical interview questions subdivided into a plurality of subgroups;

select only one representative question from each subgroup, transmit a first questionnaire including the selected representative questions only to a mobile terminal through a mobile network, and determine a provisional state of a user based on first answers received in response to the representative questions, wherein a total number of the selected representative questions included in the first questionnaire is identical to a total number of the plurality of subgroups;

transmit a second questionnaire including all of the group of medial interview questions including the selected representative questions to the mobile terminal through the mobile network when the provisional state of the user is determined as mibyou based on the first answers, determine a final state of the user based on second answers received in response to all the questions in the group, and perform a first diagnosis by analyzing the second answers;

recognize a face region from a picture or image received from the mobile terminal, extract major parts from the face region, and perform a second diagnosis based on the face region and the extracted major parts;

set a range of a numeric value of a specific color corresponding to a facial skin color to recognize the face region, set a region showing numeric value recognized within the range as a face candidate region, and apply an ellipsoidal mask operation to the face candidate region; and

transmit proposed complementary and alternative medical contents corresponding to the first and second diagnosis to the mobile terminal.

3. The apparatus of claim 2, wherein the major parts correspond to feature points of the face region, and the processor detects and formulates locations of the feature points and a ratio of distance between the feature points as an evaluation function and performs a diagnosis on the state of the user through information on the formulated evaluation function.

4. The apparatus of claim 2, wherein if there are two or more set face candidate regions, the processor determines a face region of a largest label as a face region.

5. The apparatus of claim 2, wherein to extract eyes, which is a major part, the processor examines whether a target object exists above and on a left side of a horizontal center line of the face region, examines whether a symmetrical object exists in a y coordinate symmetrical to a coordinate of the target object if the target object exists above and on a left side of a horizontal center line of the face region, and determines a symmetrical pair as eye regions if there is one symmetric pair as a result of the examination.

6. The apparatus of claim 5, wherein to extract a nose, which is a major part, the processor examines whether a target object is located below and between the eye regions, examines whether a y coordinate distance of the target object from the eye regions is smaller than a reference value if the target object is located below and between the eye regions as a result of the examination, and determines the target object as the nose region if there is one target object as a result of the examination.

7. The apparatus of claim 6, wherein to extract a mouth, which is a major part, the processor examines whether a target object exists below the nose region, examines whether the target object exists within outer boundaries of coordinates of the two eyes if the target object exists below the nose region, and determines the target object as the mouth region if there is one target object as a result of the examination.

8. The apparatus of claim 7, wherein if feature points including the face region, the eye regions, the nose regions and the mouth regions are determined, the processor analyzes correlations between the feature points.

9. The apparatus of claim 8, wherein the processor determines a correlation with a backbone through symmetry of a body, including symmetry of the feature points of the face region, angular symmetry of inclined shoulder lines, and symmetry of a pelvic region.