Method and apparatus for image analysis

• US 9,953,248 B2
• Filed: 01/08/2016
• Issued: 04/24/2018
• Est. Priority Date: 01/15/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Cho, Seong Ho
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US14/991,094
• Publication Number:
  US 20160210536A1
• Time in Patent Office: 837 Days
• Field of Search: 382128
• CPC Class Codes: A23L 33/30 Dietetic or nutritional methods, e.g. for losing weight A23L33/10 takes precedence
• CPC Class Codes: A23V 2002/00 Food compositions, function of food ingredients or processes for food or foodstuffs
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

An electronic device comprising: a memory; a communication circuit; and at least one processor operatively coupled to the memory and the communication circuit, configured to: obtain a first image; extract a feature associated with an object depicted in the first image; and control the communication circuit to transmit an indication of the feature to a server; and receive analysis information associated with the object from the server through the communication circuit.

Claims

1. An electronic device, comprising:

a memory;

a communication circuit; and

at least one processor operatively coupled to the memory and the communication circuit, configured to;

obtain a first image and classifying the first image by executing a first comparison on the first image;

in response to detecting that the first image is classified into a category of images stored in the memory, extract a feature associated with an object depicted in the first image; and

control the communication circuit to transmit an indication of the feature to a server for execution of a second comparison by the server to retrieve analysis information corresponding to the feature; and

receive the analysis information from the server through the communication circuit.

2. The electronic device of claim 1, wherein the category indicates the object is classified in a food item category, and

the at least one processor is further configured to:

in response to detecting a selection of the food item category from among a plurality of categories, display a selectable option to add the object to a list together with at least a portion of the analysis information received from the server.

3. The electronic device of claim 2, wherein the analysis information comprises at least one of:

(i) an indication of one or more ingredients of the food item,

(ii) nutritional information associated with the food item,

(iii) material information,

(iv) a recipe associated with the food item, and

(v) medical information associated with the food item.

4. The electronic device of claim 1, wherein the feature comprises at least one of an image descriptor, MPEG standards Compact Descriptors for Visual Search (CDVS), Histograms of Oriented Gradient (HOG) feature, Scale Invariant Feature Transform (SIFT) feature, dense-SIFT feature, and Speeded Up Robust Features (SURF) feature.

5. The electronic device of claim 1, wherein:

the at least one processor is further configured to select a portion of the first image that includes the object, and

the feature is extracted from the portion.

6. The electronic device of claim 1, wherein the at least one processor is further configured to:

obtain additional information associated with the first image; and

transmit the additional information to the server, the at least one processor further configured to;

when an image gallery is executed, control a display to display a list of selectable predetermined categories including the category into which the first image is classified, and

in response to detecting selection of the category from the displayed list, control the display to display a notification indicating the analysis information is received from the server.

7. The electronic device of claim 6, wherein the additional information comprises metadata associated with the first image.

8. The electronic device of claim 6, wherein the additional information comprises an indication of a time when the first image is obtained and/or an indication of a location where the first image is obtained.

9. The electronic device of claim 1, wherein the at least one processor is further configured to transmit, to the server, user information associated with the electronic device.

10. The electronic device of claim 1, wherein the memory is configured to store a second image, and the at least one processor is further configured to:

compare the first image to the second image; and

extract the feature from the first image in response to detecting that the first image matches the second image.

11. The electronic device of claim 10, wherein:

comparing the second image to the first image includes generating a score indicating a degree of similarity between the first image and the second image, and

the at least one processor is further configured to detect whether the first image matches the second image based on the score.

12. The electronic device of claim 1, wherein the analysis information is provided through an application that is executed by the at least one processor.

13. The electronic device of claim 1, further comprising a display, wherein the processor is further configured to control the display to display the analysis information.

14. A server, comprising:

a memory;

a communication circuit; and

at least one processor operatively coupled to the memory and the communication circuit, configured to;

receive, from an electronic device, a feature of an object depicted in a first image classified according to a first comparison executed by the electronic device, through the communication circuit;

analyze the feature by executing a second comparison to retrieve analysis information corresponding to the feature; and

control the communication circuit to transmit the analysis information to the electronic device,

wherein an indication of the feature comprises at least one of a MPEG standards Compact Descriptors for Visual Search (CDVS), Histograms of Oriented Gradient (HOG) feature, Scale Invariant Feature Transform (SIFT) feature, dense-SIFT feature, and Speeded Up Robust Features (SURF) feature.

15. The server of claim 14, wherein:

the at least one processor is further configured to identify a second image that matches to the feature; and

the analysis information is obtained based on the second image.

16. The server of claim 14, wherein:

the at least one processor is further configured to receive, from the electronic device, additional information associated with the first image, and

the analysis information is obtained based on a second image that matches to the feature and the additional information.

17. The server of claim 16, wherein the additional information comprises at least one of an indication of a time when the first image is obtained and an indication of a location where the first image is obtained.

18. The server of claim 14, wherein:

the memory is configured to store an analysis history database containing analysis information previously obtained by a plurality of users,

the at least one processor is further configured to receive user information associated with the electronic device, and

the analysis information is obtained based on the user information and previous analysis information that is retrieved from the analysis history database.

19. A method, comprising:

obtaining, by an electronic device, an image and classifying the image by executing a first comparison on the image;

in response to detecting that the image is classified into a category of images stored in a memory, extracting, by the electronic device, a feature on an object depicted in the image;

transmitting an indication of the feature to a server executing a second comparison on the feature to retrieve analysis information corresponding to the feature; and

receiving the analysis information associated with the object from the server.

20. The method of claim 19, wherein the feature comprises at least one of an image descriptor, MPEG standards Compact Descriptors for Visual Search (CDVS), Histograms of Oriented Gradient (HOG) feature, Scale Invariant Feature Transform (SIFT) feature, dense-SIFT feature, and Speeded Up Robust Features (SURF) feature, the method further comprising:

in response to detecting a selection of a food item category from among a plurality of categories, controlling a display to display a selectable option to add the object to a list together with at least a portion of the analysis information received from the server.