System and method for food categorization

• US 10,671,657 B2
• Filed: 10/13/2014
• Issued: 06/02/2020
• Est. Priority Date: 10/13/2014
• Status: Active Grant

• Current Assignee: Myfitnesspal Incorporated
• Original Assignee: Under Armour Incorporated
• Inventors: Kim, Joohyun, Lee, Chul, Levine, Bryan
• Primary Examiner(s): Yip, Jack
• Application Number:
  US14/513,061
• Publication Number:
  US 20160103910A1
• Time in Patent Office: 2,059 Days
• Field of Search: 434127
• CPC Class Codes: A23V 2002/00 Food compositions, function of food ingredients or processes for food or foodstuffs
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G09B 19/0092 Nutrition
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets

Abstract

Disclosed embodiments include apparatuses, methods and storage media associated with food item categorization. In embodiments, an apparatus may include one or more processors and a food categorization engine. The food categorization engine may be operated by the one or more processors to cause the apparatus to receive a description of a food item; analyze the description of the food item; and select one or more food categories with which to associate the food item based at least in part on a result of the analysis. Other embodiments may be described and claimed.

Claims

1. A method comprising:

receiving, by a food categorization engine provided on a server and operating in a training mode, a training food data set including a plurality of text descriptions of a plurality of food items and a corresponding association of each of the plurality of food items of the training food data set with one or more food categories of a plurality of food categories, wherein the food categories are food nutrient categories, wherein the training food data set is provided from a database of crowd-sourced data records of food items, each of the data records entered into the database by a plurality of users of a fitness management system;

extracting, by the food categorization engine, one or more features from each text description of the training food data set to generate a feature set for each food item of the training food data set;

analyzing, by the food categorization engine, the feature set for each food item of the training food data set to determine sets of food categorization features that correlate to individual ones of the one or more food categories, wherein the sets of food categorization features are defined by a plurality of category vectors, each of the category vectors associated with one of the food categories;

receiving, by the food categorization engine operating in an operational mode, a text description of a food item entered by a user via a client device, wherein the client device is in communication with the server, wherein the text description of the food item entered by the user is associated with a data record selected by the user as being indicative of a consumed item, the data record being provided from the database of crowd-sourced data records of food items;

standardizing the text description of the food item by removing nonstandard words and/or characters from the one or more individual words and/or characters of the text description of the food item, wherein the nonstandard words and/or characters are inconsistent naming and spelling errors in the text description of the food item;

analyzing, by the food categorization engine, one or more individual words and/or characters of the text description of the food item, wherein analyzing the one or more individual words and/or characters of the text description of the food item comprises generating a numerical vector representative thereof;

automatically selecting, by the food categorization engine, one or more food categories with which to associate the food item based at least in part on a result of the analysis of the one or more individual words and/or characters of the text description of the food item, wherein automatically selecting one or more food categories comprises comparing the numerical vector to the plurality of category vectors associated with individual ones of the one or more food categories, wherein said comparing is accomplished by matrix-vector multiplication wherein the plurality of category vectors define a matrix and the numerical vector is multiplied by the matrix;

amending the data record associated with the food item in the database to include the selected one or more food categories;

selecting the amended data record based on the one or more food categories; and

presenting data from the amended data record to the user on a display of the client device.

2. The method of claim 1, further comprising:

establishing, by the food categorization engine, one or more associations of the food item with the one or more food categories in a food data storage arrangement of a fitness management system.

3. The method of claim 2, wherein the act of establishing the one or more associations comprises one or more of:

adding one or more text identifiers descriptive of the one or more categories to a category field associated with the text description of the food item; and

adding an identifier from the text description of the food item to a food item field of each of the one or more food categories.

4. The method of claim 1, wherein the act of removing nonstandard words and/or characters from the one or more individual words and/or characters of the text description of the food item further comprises one or more of:

removing non-English characters from the one or more individual words and/or characters of the text description of the food item;

removing numeric words from the one or more individual words and/or characters of the text description of the food item;

removing words related to quantities from the one or more individual words and/or characters of the text description of the food item;

removing punctuations from the one or more individual words and/or characters of the text description of the food item; and

removing special characters from the one or more individual words and/or characters of the text description of the food item.

5. The method of claim 1, wherein:

the act of analyzing the one or more individual words and/or characters of the text description of the food item comprises extracting one or more features of the food item from the text description; and

the act of selecting the one or more food categories with which to associate the food item is based at least in part on the extracted features.

6. The method of claim 1, further comprising:

receiving, by the food categorization engine, a second training food data set including a second plurality of text descriptions of a second plurality of food items, the second training food data set omitting corresponding associations thereof with one or more of the plurality of food categories;

extracting, by the food categorization engine, one or more features from each text description of the second training food data set in order to generate a feature set for each food item of the second training food data set; and

analyzing, by the food categorization engine, the feature set for each food item of the second training food data set to identify one or more possible food categories therefor based at least in part on the common set of one or more features that correlate to individual ones of the one or more food categories.

7. The method of claim 6, further comprising:

generating a series of questions concerning the one or more possible food categories identified for each food item of the second training food data set; and

adjusting the common set of one or more features based on answers for the series of questions received from one or more of the plurality of users.

8. One or more non-transitory computer-readable media having a plurality of instructions stored thereon, the instructions, in response to execution by a processor of a network-side computing device in communication with a crowd-sourced database of data records of food items, provide the computing device with a food item categorization engine operated to:

receive, in a training mode, a training food data set including a plurality of text descriptions of a plurality of food items and a corresponding association of each of the plurality of food items of the training food data set with one or more food categories of a plurality of food categories, wherein the food categories are food nutrient categories, wherein the training food data set is provided from a database of crowd-sourced data records of food items, each of the data records entered into the database by a plurality of users of a fitness management system;

extract one or more features from each text description of the training food data set to generate a feature set for each food item of the training food data set;

analyze the feature set for each food item of the training food data set to determine sets of food categorization features that correlate to individual ones of the one or more food categories, wherein the sets of food categorization features are defined by a plurality of category vectors, each of the category vectors associated with one of the food categories;

receive, in an operational mode, a data record including a text description of a food item from a client-side application configured to log consumption of food items, the client-side application in communication with the network-side computing device, wherein the received data record is selected by the user as being indicative of a consumed item, the data record being provided from the database of crowd-sourced data records of food items, each of the data records in the database of crowd-sourced data records previously entered into the database by a plurality of users of the fitness management system;

standardize the text description of the food item included with the received data record by removing nonstandard words and/or characters from one or more individual words and/or characters of the text description of the food item, wherein the nonstandard words and/or characters are inconsistent naming and spelling errors in the text description of the food item;

analyze one or more individual words and/or characters of the text description of the food item to create a numerical vector representative thereof;

automatically select one or more food categories with which to associate the received data record and associated food item based at least in part on a result of a comparison of the numerical vector to the plurality of category vectors associated with individual ones of the one or more food categories, wherein the food categories are food nutrient categories, and wherein the comparison is accomplished by matrix-vector multiplication wherein the plurality of category vectors associated with individual ones of the one or more food categories define a matrix that is multiplied by the numerical vector;

amend the data record associated with the food item in the database to include the automatically selected one or more food categories;

select the amended data record based on the one or more food categories; and

present data from the amended data record to the user on a display of the client device.

9. The computer-readable media of claim 8, wherein the food item categorization engine is further configured to store the food item numerical vector with the plurality of numerical vectors associated with the individual ones of the one or more food categories in a food data storage arrangement of a fitness management system.

10. The computer-readable media of claim 8, wherein the food item categorization engine is further configured to:

add one or more identifiers of the respective one or more categories to a category field of the data record of the food item; and

add an identifier of the food item to a food item field of a data record of each of the one or more food categories.

11. The computer-readable media of claim 8,

wherein the analysis of the text description further comprises extraction of one or more features from the numerical vector of the food item;

and

wherein the food item categorization engine is further configured to select the one or more food categories with which to associate the food item based at least in part on the features extracted from the numerical vector of the food item.

12. The computer-readable media of claim 8, wherein the food item categorization engine is further configured to:

transmit an association of the data record for the food item and the selected one or more food categories to the client-side application thereby enabling the application to indicate in said logged consumption of food items the quantity of consumption in each of a plurality of food categories.

13. The computer-readable media of claim 8 wherein comparison of the numerical vector to a plurality of category vectors comprises, identification of a one of the plurality of food categories associated to a one of the plurality of sets of the plurality of category vectors to which the numerical vector matches within a predetermined threshold.

14. The computer-readable media of claim 8, wherein the analysis of the one or more individual words and/or characters of the text description of the food item comprises removal of nonstandard words and/or characters therefrom.

15. The computer-readable media of claim 8, wherein the one or more processors are further configured to execute a plurality of instructions of a food categorization engine training application, which when executed causes the computing device to:

generate the plurality of sets of the plurality of category vectors for each of the plurality of food categories.

16. The computer-readable media of claim 15, wherein the execution of the food categorization engine training application generates the plurality of sets of the plurality of category vectors for each of the plurality of food categories by causing the computing device to:

receive a plurality of training data from a plurality of client devices in communication with the computing device, each of the plurality training data comprising at least a description of a food item and a corresponding association of the food item to one or more of the plurality of food categories;

for each of the plurality of training data, standardize the description of the food item to create a plurality of tokens representative of one or more individual words and/or characters thereof;

convert the plurality tokens associated with each of the plurality of training data to a numerical vector; and

place individual ones of the numerical vectors into one or more of the plurality of sets of a plurality of category vectors based on the associations in the training data.

17. A server apparatus, comprising:

at least one transceiver;

a storage apparatus configured to store a plurality of instructions of a food categorization engine; and

one or more processors configured to execute the plurality of instructions of the food categorization engine, which when executed cause the server apparatus to;

receive, in a training mode, a training food data set including a plurality of text descriptions of a plurality of food items and a corresponding association of each of the plurality of food items of the training food data set with one or more food categories of a plurality of food categories, wherein the food categories are food nutrient categories, wherein the training food data set is provided from a database of crowd-sourced data records of food items, each of the data records entered into the database by a plurality of users of a fitness management system;

extract one or more features from each text description of the training food data set to generate a feature set for each food item of the training food data set;

analyze the feature set for each food item of the training food data set to determine sets of food categorization features that correlate to individual ones of the one or more food categories, wherein the sets of food categorization features are defined by a plurality of category vectors, each of the category vectors associated with one of the food categories;

receive, in an operational mode, at the transceiver an input from a client device comprising a data record including a description of a food item, wherein the received data record is indicative of a consumed item, the data record being provided from the database of crowd-sourced data records of food items, each of the data records in the database of crowd-sourced data records previously entered into the database by a plurality of users of the fitness management system;

standardize the description of the food item to create a plurality of tokens representative of one or more individual words and/or characters thereof, wherein standardizing the description of the food item includes removing non-standard words and/or characters from the one or more individual words and/or characters, wherein the non-standard words and/or characters are inconsistent naming and spelling errors in the text description of the food item;

convert the plurality tokens to a numerical vector;

compare the numerical vector to a plurality of sets of the plurality of category vectors, each set being associated to a respective one of a plurality of food categories, wherein the plurality of food categories are a plurality of food nutrient categories, and wherein said comparison is accomplished by matrix-vector multiplication wherein the plurality of category vectors define a matrix that is multiplied by the numerical vector;

based on the comparison, identify a one of the plurality of food categories associated to a one of the plurality of sets of the plurality of category vectors to which the numerical vector matches within a predetermined threshold;

amend the data record to include the identified one of a plurality of food categories, the data record associated with the description of the food item in the database;

select the amended data record from the database based on the identified one of the plurality of food categories; and

present data from the amended data record to the client device.

18. The server apparatus of claim 17, wherein the execution of the food categorization engine further causes the server apparatus to:

transmit the identified one of the plurality of food categories to the client device via the at least one transceiver, the client device configured to run at least one fitness management application thereon which is configured to utilize the identified one of the plurality of food categories in a running log of a user'"'"'s consumption across each of the plurality of food categories during a time period.

19. The server apparatus of claim 17, wherein the one or more processors are further configured to execute a plurality of instructions of a network-side fitness management application, which when executed causes the server apparatus to:

utilize the identified one of the plurality of food categories in a running log of a user'"'"'s consumption across each of the plurality of food categories during a time period; and

cause the at least one transceiver apparatus to transmit the running log to a client-side fitness management application run at the client device.

20. The server apparatus of claim 17, wherein the execution of the food categorization engine further causes the server apparatus to:

process the tokens via removal of nonstandard words and/or characters therefrom.

21. The server apparatus of claim 17, wherein the execution of the food categorization engine further causes the server apparatus to:

associate the description of the food item to the identified one of the plurality of food categories by placing the numerical vector in the one of the plurality of sets of the plurality of category vectors.

22. The server apparatus of claim 17, wherein the one or more processors are further configured to execute a plurality of instructions of a food categorization engine training application, which when executed causes the server apparatus to:

generate the plurality of sets of the plurality of category vectors for each of the plurality of food categories.

23. The server apparatus of claim 22, wherein the execution of the food categorization engine training application generates the plurality of sets of the plurality of category vectors for each of the plurality of food categories by causing the server apparatus to:

receive a plurality of training data from a plurality of client devices in communication with the server apparatus, each of the plurality training data comprising at least a description of a food item and a corresponding association of the food item to one or more of the plurality of food categories;

for each of the plurality of training data, standardize the description of the food item to create a plurality of tokens representative of one or more individual words and/or characters thereof;

convert the plurality tokens associated with each of the plurality of training data to a numerical vector; and

place individual ones of the numerical vectors into one or more of the plurality of sets of a plurality of category vectors based on the associations in the training data.