Information processing device and storage medium

• US 9,881,517 B2
• Filed: 03/25/2014
• Issued: 01/30/2018
• Est. Priority Date: 04/18/2013
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Hayashi, Kazunori, Sako, Yoichiro, Kon, Takayasu, Kamada, Yasunori, Nakamura, Takatoshi, Hanaya, Hiroyuki, Onuma, Tomoya, Tange, Akira
• Primary Examiner(s): Yao, Sam
• Assistant Examiner(s): Carlos, Alvin
• Application Number:
  US14/224,330
• Publication Number:
  US 20140315160A1
• Time in Patent Office: 1,407 Days
• Field of Search: 434127
• CPC Class Codes: A47G 21/00 Table-ware crumb trays A47L13/52; table knives B26B
• CPC Class Codes: A47G 21/103 Chop-sticks
• CPC Class Codes: A47G 2200/08 Illumination
• CPC Class Codes: A47G 2200/205 Vibrating parts
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G09B 19/0092 Nutrition
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets

Abstract

There is provided an information processing device including a detection unit configured to detect a plurality of pieces of food which are eating targets from an image obtained by imaging eating of a user, and a recommendation unit configured to recommend at least the food to be subsequently ingested in real time among the plurality of pieces of food so that the user ingests the plurality of pieces of food detected by the detection unit in an order satisfying a predetermined purpose.

Claims

1. An information processing device comprising:

a detection unit configured to detect a plurality of eating targets from an image obtained by imaging the eating targets and eating utensil of a user, wherein each detected eating target of the plurality of eating targets comprises a type distinct from other detected eating targets of the plurality of eating targets in the obtained image; and

a recommendation unit configured to recommend what particular eating target of the detected plurality of eating targets the user should ingest next in time, according to an order of ingesting the detected plurality of eating targets satisfying a predetermined purpose and based on a determination of previously ingested eating targets of the detected plurality of eating targets,

wherein the recommendation unit is further configured to generate guidance information based on a real-time recognition of whether the detected plurality of eating targets have been ingested in the recommended order of ingesting,

wherein whether the detected plurality of eating targets have been ingested in the recommended order of ingesting is recognized based on positional information of the eating utensil,

wherein the generated guidance information is presented to the user in an overlapping manner by a head mounted display (HMD) configured to be worn by the user with at least one eating target of the plurality of eating targets in order to recommend the particular eating target while the user views the plurality of eating targets, and

wherein the detection unit and the recommendation unit are each implemented via at least one processor.

2. The information processing device according to claim 1, wherein the recommendation unit generates an emphasis image showing the particular eating target of the plurality of eating targets to be ingested next in time in an emphatic manner and causes a display unit to display the emphasis image.

3. The information processing device according to claim 1, wherein the recommendation unit generates an inhibitive image showing eating targets of the detected plurality of eating targets to be inhibited from being ingested next in time in an inhibitive manner and causes a display unit to display the inhibitive image.

4. The information processing device according to claim 1, further comprising:

an accumulation unit configured to accumulate a history of the previously ingested eating targets of the user,

wherein the recommendation unit recommends at least the particular eating target of the plurality of eating targets to be ingested next in time further based on the history accumulated by the accumulation unit, and

wherein the accumulation unit is implemented via at least one processor.

5. The information processing device according to claim 1, further comprising:

a recognition unit configured to recognize an eating situation of the user from the image obtained by imaging the eating targets of the user,

wherein the recommendation unit recommends at least the particular eating target of the detected plurality of eating targets to be ingested next in time further based on the situation recognized by the recognition unit, and

wherein the recognition unit is implemented via at least one processor.

6. The information processing device according to claim 1, further comprising:

a transmission unit configured to transmit guidance information for guiding the user to each particular eating target of the detected plurality of eating targets recommended by the recommendation unit to an eating utensil having a function of guiding the user based on received information regarding the recommended order of the detected plurality of eating targets,

wherein the transmission unit is implemented via at least one processor.

7. The information processing device according to claim 2, wherein the emphasis image is an image beautifying an outer appearance of the particular eating target of the detected plurality of eating targets.

8. The information processing device according to claim 3, wherein the inhibitive image is an image degrading an outer appearance of the eating targets of the detected plurality of eating targets to be inhibited.

9. The information processing device according to claim 3, wherein the inhibitive image is an image screening the eating targets of the detected plurality of eating targets to be inhibited from the user.

10. The information processing device according to claim 1, wherein the order satisfying the predetermined purpose is an order of the detected plurality of eating targets suppressing an increase in a blood-sugar level of the user.

12. The information processing device according to claim 1, wherein the detected plurality of eating targets comprise a plurality of pieces of food.

11. A non-transitory computer-readable storage medium having a program stored therein, which when executed by an information processing device of a computer, causes the computer to execute a method, the method comprising:

detecting a plurality of eating targets from an image obtained by imaging the eating targets and eating utensil of a user, wherein each detected eating target of the plurality of eating targets comprises a type distinct from each other detected eating target of the plurality of eating targets in the obtained image; and

recommending what particular eating target of the detected plurality of eating targets the user should ingest next in time, according to an order of ingesting the detected plurality of eating targets satisfying a predetermined purpose and based on a determination of previously ingested eating targets of the detected plurality of eating targets,

wherein the recommending further comprises generating guidance information based on a real-time recognition of whether the detected plurality of eating targets have been ingested in the recommended order of ingesting, and

wherein whether the detected plurality of eating targets have been ingested in the recommended order of ingesting is recognized based on positional information of eating utensil,

wherein the generated guidance information is presented to the user in an overlapping manner by a head mounted display (HMD) configured to be worn by the user with at least one eating target of the plurality of eating targets in order to recommend the particular eating target while the user views the plurality of eating targets.

13. The non-transitory computer-readable storage medium according to claim 11, wherein the detected plurality of eating targets comprise a plurality of pieces of food.