Method and system for diet management

• US 9,449,029 B2
• Filed: 04/02/2013
• Issued: 09/20/2016
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Ting, Chuan-Wei, Wang, Ching-Yao, Chen, Ju-Chin
• Primary Examiner(s): Lee, Wilson
• Application Number:
  US13/854,970
• Publication Number:
  US 20140170608A1
• Time in Patent Office: 1,267 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: G09B 19/0092 Nutrition
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A system and a method for diet management based on image analysis are provided. The system includes a database and a comparison device. The comparison device is coupled to the database. The comparison device performs similarity comparison in the database based on a supervector related to at least one diet image so as to find out at least one similar population and provides information related to the at least one similar population.

Claims

1. A computer-implemented method for diet management, adapted to a diet management system, comprising:

providing a supervector, wherein the supervector is related to at least one diet image;

performing similarity comparison in a database based on the supervector by a comparison device, so as to find out at least one similar population; and

providing information related to the similar population,

wherein the providing the information related to the similar population comprises;

calculating at least one statistic of the similar population by using at least one multidimensional data of the similar population; and

providing the information related to the similar population based on the statistic of the similar population.

2. The method for diet management of claim 1, wherein the providing the supervector comprises:

capturing the at least one diet image via an image capture device;

pre-processing the at least one diet image so as to obtain at least one detailed food segment;

extracting at least one diet image feature from the detailed food segment; and

generating the supervector based on the at least one diet image feature.

3. The method for diet management of claim 2, wherein the pre-processing the at least one diet image comprises:

transforming the at least one diet image to a normalized space.

4. The method for diet management of claim 2, wherein the pre-processing the at least one diet image comprises:

performing color correction, bright correction, or white balance correction on the at least one diet image; and

obtaining the at least one detailed food segment from the at least one diet image.

5. The method for diet management of claim 4, wherein the obtaining the at least one detailed food segment from the at least one diet image comprises:

performing background removal on the at least one diet image so as to obtain at least one diet region; and

performing detailed food segment segmentation on the at least one diet region so as to segment out the at least one detailed food segment from the diet region.

6. The method for diet management of claim 2, wherein the at least one diet image feature comprises image capturing time, image capturing location, color, texture complexity, or reflectivity.

7. The method for diet management of claim 6, wherein the texture complexity comprises texture magnitude, texture orientation, texture length, texture amount, or texture regularity.

8. The method for diet management of claim 2, wherein the extracting the at least one diet image feature comprises:

performing an X-direction difference calculation and a Y-direction difference calculation on the detailed food segment so as to obtain an X-direction difference image and a Y-direction difference image;

calculating texture magnitude information and texture orientation information by using the X-direction difference image and the Y-direction difference image;

calculating at least one texture coding in at least one direction in the detailed food segment by using the texture magnitude information and the texture orientation information; and

integrating the at least one texture coding in the at least one direction so as to obtain the at least one diet image feature.

9. The method for diet management of claim 8, wherein the calculating the texture magnitude information and the texture orientation information comprises:

calculating the texture magnitude information e(x,y)=√

{square root over (g_x²+g_y²)}, wherein gx represents a pixel value of the X-direction difference image at (x,y), and gy represents a pixel value of the Y-direction difference image at (x,y); and

calculating the texture orientation information

10. The method for diet management of claim 8, wherein the calculating the at least one texture coding in the at least one direction in the detailed food segment comprises:

selecting a target pixel from the detailed food segment;

selecting a coding region along the at least one direction in the detailed food segment, wherein the coding region comprises the target pixel and a plurality of neighboring pixels;

performing binarization on the texture orientation information of the neighboring pixels in the coding region so as to obtain binary values of the neighboring pixels;

transforming the binary values of the neighboring pixels to a category value of the target pixel;

determining a height value of the target pixel based on the texture magnitude information of the target pixel; and

determining the at least one texture coding in the at least one direction in the detailed food segment based on the category values and the height values in the detailed food segment.

11. The method for diet management of claim 10, wherein the performing binarization on the texture orientation information of the neighboring pixels in the coding region comprises:

selecting a neighboring pixel from the neighboring pixels;

the binary value of the neighboring pixel being 1 if |θ

i−

θ

n|≦

γ

θ

, wherein θ

i represents the texture orientation information of the neighboring pixel, and wherein θ

n represents an angle of a normal vector of the at least one direction, and wherein γ

θ

represents an angle threshold value; and

the binary value of the neighboring pixel being 0 if |θ

i−

θ

n|<

γ

θ

.

12. The method for diet management of claim 10, wherein the transforming the binary values of the neighboring pixels to the category value of the target pixel comprises:

calculating the category value

13. The method for diet management of claim 10, wherein the determining the height value of the target pixel based on the texture magnitude information of the target pixel comprises:

setting the texture magnitude information of the target pixel to be the height value of the target pixel.

14. The method for diet management of claim 10, wherein the determining the at least one texture coding in the at least one direction in the detailed food segment based on the category values and the height values in the detailed food segment comprises:

among a plurality of the pixels in the detailed food segment, accumulating the height values of the pixels with the same category value so as to obtain a histogram, wherein a horizontal axis of the histogram represents the category values and a vertical axis of the histogram represents the height values; and

determining the at least one texture coding in the at least one direction in the detailed food segment based on the histogram.

15. The method for diet management of claim 8, wherein the at least one direction comprises a plurality of the directions, and wherein the integrating the at least one texture coding in the at least one direction comprises:

connecting the at least one texture coding in the directions to each other in a connection order so as to obtain the at least one diet image feature of the detailed food segment.

16. The method for diet management of claim 2, wherein the at least one diet image comprises a plurality of the detailed food segments, and wherein the supervector comprises the diet image features of the detailed food segments.

17. The method for diet management of claim 2, wherein the at least one diet image comprises a plurality of the diet images, and wherein the generating the supervector based on the at least one diet image feature comprises:

connecting the diet image features of the plurality of the diet images to each other to form the supervector in an order based on capturing time.

18. The method for diet management of claim 2, wherein the providing the supervector further comprises:

defining a diet recording period;

wherein the capturing the at least one diet image is performed during the diet recording period.

19. The method for diet management of claim 18, wherein the performing similarity comparison comprises:

performing similarity comparison between the supervector and all other people'"'"'s supervectors with a same duration as the diet recording period in the database so as to select the at least one similar population.

20. The method for diet management of claim 18, wherein the supervector presents a diet characteristic within the diet recording period.

21. The method for diet management of claim 1, wherein the performing similarity comparison comprises:

performing similarity comparison between the supervector and all other people'"'"'s supervectors in the database so as to select the at least one similar population.

22. The method for diet management of claim 1, wherein the performing similarity comparison comprises:

calculating a Euclidean distance, an angle, a correlation coefficient, or mutual information between the supervector and another person'"'"'s supervector in the database.

23. The method for diet management of claim 1, wherein the multidimensional data comprises age, gender, weight, residency, occupation type, diet time, physical health status, mental health status, or disease development status.

24. The method for diet management of claim 1, wherein the statistic of the similar population comprises demographic data, personal health statistical data, or diet time statistical data of the at least one similar population.

25. The method for diet management of claim 1, wherein the providing the information related to the similar population further comprises:

calculating at least one statistic of a non-similar population by using at least one multidimensional data of the non-similar population; and

comparing the statistic of the similar population and the statistic of the non-similar population so as to provide a comparison result.

26. The method for diet management of claim 25, wherein the providing the information related to the similar population further comprises:

finding out categories of the information with significant differences between the statistic of the similar population and the statistic of the non-similar population to be a diet habit characteristic of the similar population by using a data mining technique or a statistical test, and providing the diet habit characteristic of the similar population to a user.

27. The method for diet management of claim 1 further comprising:

capturing the at least one diet image via an image capture device;

providing the supervector related to the at least one diet image to a server, wherein the server performs similarity comparison in the database based on the supervector so as to find out the at least one similar population; and

receiving the information related to the similar population from the server.

28. A system for diet management comprising a computer and a memory device used as a database, the computer comprises a processor and a plurality of program instructions, wherein the plurality of program instructions are loaded into the processor to perform the following operations:

performing similarity comparison in the database based on a supervector so as to find out at least one similar population, and providing information related to the similar population, wherein the supervector is related to at least one diet image,

wherein the operation of providing information related to the similar population comprises;

calculating at least one statistic of the similar population by using at least one multidimensional data of the similar population, and providing the information related to the similar population based on the statistic of the similar population.

29. The system for diet management of claim 28, wherein the plurality of program instructions are loaded into the processor to further perform the following operations:

capturing the at least one diet image;

pre-processing the at least one diet image so as to obtain at least one detailed food segment; and

extracting at least one diet image feature from the detailed food segment, and generating the supervector based on the at least one diet image feature.

30. The system for diet management of claim 29, wherein the operation of pre-processing the at least one diet image comprises:

transforming the at least one diet image to a normalized space.

31. The system for diet management of claim 29, wherein the operation of pre-processing the at least one diet image comprises:

performing color correction, bright correction, or white balance correction on the at least one diet image, and obtaining the at least one detailed food segment from the at least one diet image.

32. The system for diet management of claim 31, wherein the operation of obtaining the at least one detailed food segment from the at least one diet image comprises:

performing background removal on the at least one diet image so as to obtain at least one diet region, and performing detailed food segment segmentation on the at least one diet region so as to segment out the at least one detailed food segment from the diet region.

33. The system for diet management of claim 29, wherein the at least one diet image feature comprises image capturing time, image capturing location, color, texture complexity, or reflectivity.

34. The system for diet management of claim 29, wherein the at least one diet image comprises a plurality of the detailed food segments, and wherein the supervector comprises the diet image features of the detailed food segments.

35. The system for diet management of claim 29, wherein the at least one diet image comprises a plurality of the diet images, and wherein the operation of generating the supervector based on the at least one diet image feature comprises:

connecting the diet image features of the plurality of the diet images to each other to form the supervector in an order based on capturing time.

36. The system for diet management of claim 29, wherein the operation of capturing the at least one diet image further comprises:

capturing the at least one diet image during a diet record period.

37. The system for diet management of claim 28, wherein the operation of performing similarity comparison comprises:

calculating a Euclidean distance, an angle, a correlation coefficient, or mutual information between the supervector and another person'"'"'s supervector in the database.

38. The system for diet management of claim 28, wherein the multidimensional data comprises age, gender, weight, residency, occupation type, diet time, physical health status, mental health status, or disease development status.

39. The system for diet management of claim 28, wherein the statistic of the similar population comprises demographic data, personal health statistical data, or diet time statistical data of the at least one similar population.

40. The system for diet management of claim 28, wherein the operation of providing the information related to the similar population further comprises:

calculating at least one statistic of a non-similar population by using at least one multidimensional data of the non-similar population, and comparing the statistic of the similar population and the statistic of the non-similar population so as to provide a comparison result.

41. The system for diet management of claim 40, wherein the operation of providing the information related to the similar population further comprises:

finding out categories of the information with significant differences between the statistic of the similar population and the statistic of the non-similar population to be a diet habit characteristic of the similar population by using a data mining technique or a statistical test, and providing the diet habit characteristic of the similar population to a user.

42. An electronic device comprising:

a database; and

a processing circuit, electrically coupled to the database for performing similarity comparison in the database based on a supervector so as to find out at least one similar population, and providing information related to the similar population, wherein the supervector is related to at least one diet image, the processing circuit calculates at least one statistic of the similar population by using at least one multidimensional data of the similar population, and the processing circuit provides the information related to the similar population based on the statistic of the similar population.