Automated Food Recognition and Nutritional Estimation With a Personal Mobile Electronic Device
First Claim
1. A food recognition assistant embodied in one or more non-transitory machine accessible storage media, the food recognition assistant comprising instructions executable by one or more processors to cause a personal mobile electronic device to:
- receive a single digital image;
execute one or more feature detection algorithms using at least two different scales of resolution to determine one or more salient regions of the image without relying on a user to specify the one or more salient regions, the one or more salient regions depicting food;
execute a semantic reasoning technique to identify a food present in the image by comparing a characteristic of the one or more salient regions with a characteristic of a known food; and
by an output device of the personal mobile electronic device, output a name of the identified food.
1 Assignment
0 Petitions
Accused Products
Abstract
A food recognition assistant system includes technologies to recognize foods and combinations of foods depicted in a digital picture of food. Some embodiments include technologies to estimate portion size and calories, and to estimate nutritional value of the foods. In some embodiments, data identifying recognized foods and related information are generated in an automated fashion without relying on human assistance to identify the foods. In some embodiments, the system includes technologies for achieving automatic food detection and recognition in a real-life setting with a cluttered background, without the images being taken in a controlled lab setting, and without requiring additional user input (such as user-defined bounding boxes). Some embodiments of the system include technologies for personalizing the food classification based on user-specific habits, location and/or other criteria.
-
Citations
21 Claims
-
1. A food recognition assistant embodied in one or more non-transitory machine accessible storage media, the food recognition assistant comprising instructions executable by one or more processors to cause a personal mobile electronic device to:
-
receive a single digital image; execute one or more feature detection algorithms using at least two different scales of resolution to determine one or more salient regions of the image without relying on a user to specify the one or more salient regions, the one or more salient regions depicting food; execute a semantic reasoning technique to identify a food present in the image by comparing a characteristic of the one or more salient regions with a characteristic of a known food; and by an output device of the personal mobile electronic device, output a name of the identified food. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A food recognition assistant embodied in one or more non-transitory machine accessible storage media, the food recognition assistant comprising instructions executable by one or more processors to cause a personal mobile electronic device to:
-
receive a digital image; execute one or more feature detection algorithms to determine a plurality of salient regions of the image, each salient region being a region of the image that depicts food; and execute a semantic reasoning technique to identify a food present in at least one of the salient regions, at least partly based on;
(a) comparing a characteristic of the salient region with a characteristic of a known food, and (b) a likelihood of a different food being present in a different one of the salient regions;wherein at least one of executing the feature detection algorithms and executing the semantic reasoning technique is performed using at least two different scales of resolution with respect to the image; and by an output device of the personal mobile electronic device, output data indicative of the identified food and the different food. - View Dependent Claims (14, 15, 16, 17, 18, 19)
-
-
20. A personal mobile electronic device comprising:
-
one or more processors; an input/output subsystem coupled to the one or more processors; a camera coupled to the input/output subsystem; a display device coupled to the input/output subsystem; and memory comprising instructions executable by the one or more processors to cause the personal mobile electronic device to; with the camera, take a single picture of a plate of food; with the input/output subsystem, transmit the picture of the plate of food to a server computing device; with the input/output subsystem, receive from the server computing device, data identifying a food present in the picture of the plate of food and an algorithmically determined-confidence level associated with the identified food, wherein the data identifying the food present in the picture of the plate of food results from a multi-scale computer vision analysis of the single picture of the plate of food; and with the display device, display a label indicative of the identified food. - View Dependent Claims (21)
-
Specification