METHOD AND SYSTEM FOR DIET MANAGEMENT

• US 20140170608A1
• Filed: 04/02/2013
• Published: 06/19/2014
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Ting, Chuan-Wei, Wang, Ching-Yao, Chen, Ju-Chin
• Granted Patent: US 9,449,029 B2
• Time in Patent Office: Days
• US Class Current: 434/127
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: G09B 19/0092 Nutrition
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A system and a method for diet management based on image analysis are provided. The system includes a database and a comparison device. The comparison device is coupled to the database. The comparison device performs similarity comparison in the database based on a supervector related to at least one diet image so as to find out at least one similar population and provides information related to the at least one similar population.

Claims

1. A computer-implemented method for diet management, adapted to a diet management system, comprising:

providing a supervector, wherein the supervector is related to at least one diet image;

performing similarity comparison in a database based on the supervector by a comparison device, so as to find out at least one similar population; and

providing information related to the similar population.

2. The method for diet management of claim 1, wherein the providing the supervector comprises:

capturing the at least one diet image via an image capture device;

pre-processing the at least one diet image so as to obtain at least one detailed food segment;

extracting at least one diet image feature from the detailed food segment; and

generating the supervector based on the at least one diet image feature.

3. The method for diet management of claim 2, wherein the pre-processing the at least one diet image comprises:

transforming the at least one diet image to a normalized space.

4. The method for diet management of claim 2, wherein the pre-processing the at least one diet image comprises:

performing color correction, bright correction, or white balance correction on the at least one diet image; and

obtaining the at least one detailed food segment from the at least one diet image.

5. The method for diet management of claim 4, wherein the obtaining the at least one detailed food segment from the at least one diet image comprises:

performing background removal on the at least one diet image so as to obtain at least one diet region; and

performing detailed food segment segmentation on the at least one diet region so as to segment out the at least one detailed food segment from the diet region.

6. The method for diet management of claim 2, wherein the at least one diet image feature comprises image capturing time, image capturing location, color, texture complexity, or reflectivity.

7. The method for diet management of claim 6, wherein the texture complexity comprises texture magnitude, texture orientation, texture length, texture amount, or texture regularity.

8. The method for diet management of claim 2, wherein the extracting the at least one diet image feature comprises:

performing an X-direction difference calculation and a Y-direction difference calculation on the detailed food segment so as to obtain an X-direction difference image and a Y-direction difference image;

calculating texture magnitude information and texture orientation information by using the X-direction difference image and the Y-direction diffemece image;

calculating at least one texture coding in at least one direction in the detailed food segment by using the texture magnitude information and the texture orientation information; and

integrating the at least one texture coding in the at least one direction so as to obtain the at least one diet image feature.

9. The method for diet management of claim 8, wherein the calculating the texture magnitude information and the texture orientation information comprises:

calculating the texture magnitude information e(x,y)=√

{square root over (g_x²+g_y²)}, wherein g_x represents a pixel value of the X-direction difference image at (x,y), and g_y represents a pixel value of the Y-direction difference image at (x,y); and

calculating the texture orientation information

10. The method for diet management of claim 8, wherein the calculating the at least one texture coding in the at least one direction in the detailed food segment comprises:

selecting a target pixel from the detailed food segment;

selecting a coding region along the at least one direction in the detailed food segment, wherein the coding region comprises the target pixel and a plurality of neighboring pixels;

performing binarization on the texture orientation information of the neighboring pixels in the coding region so as to obtain binary values of the neighboring pixels;

transforming the binary values of the neighboring pixels to a category value of the target pixel;

determining a height value of the target pixel based on the texture magnitude information of the target pixel; and

determining the at least one texture coding in the at least one direction in the detailed food segment based on the category values and the height values in the detailed food segment.

11. The method for diet management of claim 10, wherein the performing binarization on the texture orientation information of the neighboring pixels in the coding region comprises:

selecting a neighboring pixel from the neighboring pixels;

the binary value of the neighboring pixel being 1 if |θ

i−

θ

n|≦

γ

_θ

, wherein θ

i represents the texture orientation information of the neighboring pixel, and wherein θ

n represents an angle of a normal vector of the at least one direction, and wherein γ

_θ

represents an angle threshold value; and

the binary value of the neighboring pixel being 0 if |θ

i−

θ

n|>

γ

_θ

.

12. The method for diet management of claim 10, wherein the transforming the binary values of the neighboring pixels to the category value of the target pixel comprises:

calculating the category value

13. The method for diet management of claim 10, wherein the determining the height value of the target pixel based on the texture magnitude information of the target pixel comprises:

setting the texture magnitude information of the target pixel to be the height value of the target pixel.

14. The method for diet management of claim 10, wherein the determining the at least one texture coding in the at least one direction in the detailed food segment based on the category values and the height values in the detailed food segment comprises:

among a plurality of the pixels in the detailed food segment, accumulating the height values of the pixels with the same category value so as to obtain a histogram, wherein a horizontal axis of the histogram represents the category values and a vertical axis of the histogram represents the height values; and

determining the at least one texture coding in the at least one direction in the detailed food segment based on the histogram.

15. The method for diet management of claim 8, wherein the at least one direction comprises a plurality of the directions, and wherein the integrating the at least one texture coding in the at least one direction comprises:

connecting the at least one texture coding in the directions to each other in a connection order so as to obtain the at least one diet image feature of the detailed food segment.

16. The method for diet management of claim 2, wherein the at least one diet image comprises a plurality of the detailed food segments, and wherein the supervector comprises the diet image features of the detailed food segments.

17. The method for diet management of claim 2, wherein the at least one diet image comprises a plurality of the diet images, and wherein the generating the supervector based on the at least one diet image feature comprises:

connecting the diet image features of the plurality of the diet images to each other to form the supervector in an order based on capturing time.

18. The method for diet management of claim 2, wherein the providing the supervector further comprises:

defining a diet recording period;

wherein the capturing the at least one diet image is performed during the diet recording period.

19. The method for diet management of claim 18, wherein the performing similarity comparison comprises:

performing similarity comparison between the supervector and all other people'"'"'s supervectors with a same duration as the diet recording period in the database so as to select the at least one similar population.

20. The method for diet management of claim 18, wherein the supervector presents a diet characteristic within the diet recording period.

21. The method for diet management of claim 1, wherein the performing similarity comparison comprises:

performing similarity comparison between the supervector and all other people'"'"'s supervectors in the database so as to select the at least one similar population.

22. The method for diet management of claim 1, wherein the performing similarity comparison comprises:

calculating a Euclidean distance, an angle, a correlation coefficient, or mutual information between the supervector and another person'"'"'s supervector in the database.

23. The method for diet management of claim 1, wherein the providing the information related to the similar population comprises:

calculating at least one statistic of the similar population by using at least one multidimensional data of the similar population; and

providing the information related to the similar population based on the statistic of the similar population.

24. The method for diet management of claim 23, wherein the multidimensional data comprises age, gender, weight, residency, occupation type, diet time, physical health status, mental health status, or disease development status.

25. The method for diet management of claim 23, wherein the statistic of the similar population comprises demographic data, personal health statistical data, or diet time statistical data of the at least one similar population.

26. The method for diet management of claim 23, wherein the providing the information related to the similar population further comprises:

calculating at least one statistic of a non-similar population by using at least one multidimensional data of the non-similar population; and

comparing the statistic of the similar population and the statistic of the non-similar population so as to provide a comparison result.

27. The method for diet management of claim 26, wherein the providing the information related to the similar population further comprises:

finding out categories of the information with significant differences between the statistic of the similar population and the statistic of the non-similar population to be a diet habit characteristic of the similar population by using a data mining technique or a statistical test, and providing the diet habit characteristic of the similar population to a user.

28. The method for diet management of claim 1 further comprising:

capturing the at least one diet image via an image capture device;

providing the supervector related to the at least one diet image to a server, wherein the server performs similarity comparison in the database based on the supervector so as to find out the at least one similar population; and

receiving the information related to the similar population from the server.

29. A system for diet management comprising:

a database; and

a comparison device, coupled to the database, for performing similarity comparison in the database based on a supervector so as to find out at least one similar population, and providing information related to the similar population, wherein the supervector is related to at least one diet image.

30. The system for diet management of claim 29 further comprising:

an image capture device for capturing the at least one diet image;

a processing device, coupled to the image capture device, for pre-processing the at least one diet image so as to obtain at least one detailed food segment; and

a feature extraction device, coupled to the processing device, for extracting at least one diet image feature from the detailed food segment, and generating the supervector based on the at least one diet image feature to the comparison device.

31. The system for diet management of claim 30, wherein the processing device transform the at least one diet image to a noimalized space.

32. The system for diet management of claim 30, wherein the processing device perfoims color correction, bright correction, or white balance correction on the at least one diet image, and wherein the processing device obtains the at least one detailed food segment from the at least one diet image.

33. The system for diet management of claim 32, wherein the processing device performs background removal on the at least one diet image so as to obtain at least one diet region, and wherein the processsing device performs detailed food segment segmentation on the at least one diet region so as to segment out the at least one detailed food segment from the diet region.

34. The system for diet management of claim 30, wherein the at least one diet image feature comprises image capturing time, image capturing location, color, texture complexity, or reflectivity.

35. The system for diet management of claim 30, wherein the at least one diet image comprises a plurality of the detailed food segments, and wherein the supervector comprises the diet image features of the detailed food segments.

36. The system for diet management of claim 30, wherein the at least one diet image comprises a plurality of the diet images, and wherein the feature extraction device connects the diet image features of the plurality of the diet images to each other to form the supervector in an order based on capturing time.

37. The system for diet management of claim 30, wherein the image capture device captures the at least one diet image during a diet record period.

38. The system for diet management of claim 29, wherein the comparison device calculates a Euclidean distance, an angle, a correlation coefficient, or mutual information between the supervector and another person'"'"'s supervector in the database.

39. The system for diet management of claim 29, wherein the comparison device calculates at least one statistic of the similar population by using at least one multidimensional data of the similar population, and wherein the comparison device provides the information related to the similar population based on the statistic of the similar population.

40. The system for diet management of claim 39, wherein the multidimensional data comprises age, gender, weight, residency, occupation type, diet time, physical health status, mental health status, or disease development status.

41. The system for diet management of claim 39, wherein the statistic of the similar population comprises demographic data, personal health statistical data, or diet time statistical data of the at least one similar population.

42. The system for diet management of claim 39, wherein the comparison device calculates at least one statistic of a non-similar population by using at least one multidimensional data of the non-similar population, and wherein the comparison device compares the statistic of the similar population and the statistic of the non-similar population so as to provide a comparison result.

43. The system for diet management of claim 42, wherein the comparison device finds out categories of the information with significant differences between the statistic of the similar population and the statistic of the non-similar population to be a diet habit characteristic of the similar population by using a data mining technique or a statistical test, and providing the diet habit characteristic of the similar population to a user.

44. A computer-implemented method for diet management, adapted to a diet management system, comprising:

capturing at least one diet image via an image capture device;

pre-processing the at least one diet image so as to obtain at least one diet region from the at least one diet image and obtain at least one detailed food segment from the diet region;

extracting at least one diet image feature from the at least one detailed food segment;

determining a diet type of the at least one detailed food segment based on the at least one diet image feature; and

providing a personal diet characteristic analysis based on the diet type and an area of the at least one detailed food segment.

45. The method for diet management of claim 44, wherein the pre-processing the at least one diet image comprises:

transforming the at least one diet image to a normalized space.

46. The method for diet management of claim 44, wherein the pre-processing the at least one diet image comprises:

performing color correction, bright correction, or white balance correction on the at least one diet image;

performing background removal on the at least one diet image so as to obtain the at least one diet region; and

performing detailed food segment segmentation on the at least one diet region so as to segment out the at least one detailed food segment from the diet region.

47. The method for diet management of claim 44, wherein the at least one image feature comprises image capturing time, image capturing location, color, texture complexity, or reflectivity.

48. The method for diet management of claim 44, further comprises:

performing similarity comparison between the at least one diet image feature and at least one feature vectors in a database so as to determine the diet type of the at least one detailed food segment.

49. The method for diet management of claim 48, wherein the performing similarity comparison comprises:

calculating an Euclidean distance, an angle, a correlation coefficient, or mutual information between the at least one diet image feature and one of the feature vectors in the database.

50. The method for diet management of claim 44, wherein the determining the diet type of the at least one detailed food segment comprises:

performing similarity comparison in a database or via a diet type classifier.