Information processing apparatus, method, and computer-readable storage medium for generating food item images

• US 9,703,928 B2
• Filed: 05/31/2012
• Issued: 07/11/2017
• Est. Priority Date: 07/26/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Mochizuki, Shunsuke, Sun, Yun, Mochizuki, Daisuke, Okamura, Yuki, Gotoh, Tomohiko, Sato, Tatsuhito, Yaeda, Takeshi
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): Vu, Khoa
• Application Number:
  US13/484,730
• Publication Number:
  US 20130027424A1
• Time in Patent Office: 1,867 Days
• Field of Search: None
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 40/63 for local operation

Abstract

An information processing apparatus includes a clipping section that clips, from a captured image in which a food is captured as a subject, regions for each food to generate food images, and a grouping section that groups the generated food images for each identical subject.

Claims

1. An information processing apparatus comprising:

circuitry configured to

clip, from a plurality of captured images in which food is captured, regions that include a food item to generate a plurality of food item images, the plurality of food item images including images of a plurality of different food items;

select, after the circuitry clips the regions that include the food item to generate the plurality of food item images, a representative food item image from among the plurality of food item images based on an evaluation value, a food item image in which a food item is captured from a particular position having a higher evaluation value and being selected as the representative food item image over a food item image in which a food item is captured from another position;

perform image analysis to determine at least one of a number of calories and a name of a food item in the representative food item image;

provide, to a user, after performing the image analysis, dietary history information based on the determination of the at least one of the number of calories corresponding to an amount of the food item and the name of the food item in the representative food item image; and

provide, to the user, a food log including images of the different food items from the plurality of captured images that are associated with a user selectable display format.

2. The information processing apparatus according to claim 1, wherein the circuitry is further configured to

determine a nutrient balance of each of the different food items in the plurality of food item images;

and

concurrently present to the user results of the nutrient balance determination of the different food items consumed during a predetermined period of time made by the circuitry on a same display.

3. The information processing apparatus according to claim 1, wherein the circuitry is further configured to

determine which of the plurality of food item images contain the plurality of different food items based on dates and times of capture of the plurality of food item images or locations of capture of the plurality of food item images.

4. The information processing apparatus according to claim 1, wherein the circuitry is further configured to

clip the regions by detecting tableware on which each food item is served to generate the food item images.

5. The information processing apparatus according to claim 1, further comprising:

an image capturing sensor configured to capture a plurality of different images to generate the plurality of captured images.

6. The information processing apparatus according to claim 1, wherein

the plurality of captured images include images of food that are consumed on different days.

7. The information processing apparatus according to claim 1, wherein the circuitry is further configured to

group the generated plurality of food item images into one or more groups, each of the one or more groups containing at least one of the food item images of an identical food item.

8. The information processing apparatus according to claim 7, wherein the circuitry is further configured to

select a representative food item image from one or more of the one or more groups, and determine at least one of a number of calories and a name of a food item in the selected representative food item image from the one or more of the one or more groups.

9. The information processing apparatus according to claim 8, wherein the circuitry is further configured to

select the representative food item image from the one or more of the one or more groups in accordance with an operation performed by the user.

10. The information processing apparatus according to claim 8, wherein the circuitry is further configured to

pair all the generated food item images to group the food item images for each identical food item based on results of a determination as to whether or not two food item images forming each pair have an identical food item.

11. The information processing apparatus according to claim 1, wherein the circuitry is further configured to

allow the user to select the user selectable display format from a plurality of different formats including one day, one week, and one month display formats.

12. The information processing apparatus according to claim 1, wherein the circuitry is configured to select the representative food item image from among the plurality of food item images based on the evaluation value that corresponds to a characteristic of the food item depicted within the representative food item image and a characteristic of the representative food item image.

13. An information processing method performed by an information processing apparatus, comprising:

clipping by circuitry of the information processing apparatus, from a plurality of captured images in which food is captured, regions that include a food item to generate a plurality of food item images, the plurality of food item images including images of a plurality of different food items;

selecting, by the circuitry after clipping the regions that include the food item to generate the plurality of food item images, a representative food item image from among the plurality of food item images based on an evaluation value, a food item image in which a food item is captured from a particular position having a higher evaluation value and being selected as the representative food item image over a food item image in which a food item is captured from another position;

performing, by the circuitry, image analysis to determine at least one of a number of calories and a name of a food item in the representative food item image;

providing to a user, by the circuitry, after performing the image analysis, dietary history information based on the determination of the at least one of the number of calories corresponding to an amount of the food item and the name of the food item in the representative food item image; and

providing, to the user, a food log including images of the different food items from the plurality of captured images that are associated with a user selectable display format.

14. A non-transitory computer-readable storage medium storing a program that, when executed, causes a computer to perform a method, the method comprising:

clipping, from a plurality of captured images in which food is captured, regions that include food item to generate a plurality of food item images, the plurality of food item images including images of a plurality of different food items;

selecting, after clipping the regions that include the food item to generate the plurality of food item images, a representative food item image from among the plurality of food item images based on an evaluation value, a food item image in which a food item is captured from a particular position having a higher evaluation value and being selected as the representative food item image over a food item image in which a food item is captured from another position;

performing image analysis to determine at least one of a number of calories and a name of a food item in the representative food item image;

providing, to a user, after performing the image analysis, dietary history information based on the determination of the at least one of the number of calories corresponding to an amount of the food item and the name of the food item in the representative food item image; and

providing, to the user, a food log including images of the different food items from the plurality of captured images that are associated with a user selectable display format.