Incentivizing foodstuff consumption through the use of augmented reality features

• US 9,754,168 B1
• Filed: 05/16/2017
• Issued: 09/05/2017
• Est. Priority Date: 05/16/2017
• Status: Expired due to Fees

• Current Assignee: Mnemonic Health, Inc.
• Original Assignee: Sounds Food, Inc.
• Inventors: Henderson, Mark Lindsay
• Primary Examiner(s): Zalalee, Sultana M
• Application Number:
  US15/596,643
• Time in Patent Office: 112 Days
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06K 19/06028 using bar codes
• CPC Class Codes: G06K 19/06037 multi-dimensional coding
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04W 4/02 Services making use of location information

Abstract

Introduced here are techniques for incentivizing the consumption of foodstuffs through the use of augmented reality features. A parent may initiate an application on a mobile phone and scan a plate of one or more foodstuffs by initiating a live view captured by a camera. The parent can then select a given foodstuff that a child is unlikely to consume. When the child views the given foodstuff through the mobile phone, a portion of an augmented reality feature could be shown. For example, the child may be able to see the tail of a creature, a corner of a item, etc. As the child consumes the given foodstuff, additional portion(s) of the augmented reality feature are exposed, and the augmented reality feature becomes increasingly visible. Such a technique incentivizes the child to continue eating the given foodstuff so that the augmented reality feature can be fully seen.

Claims

1. A method for incentivizing the consumption of foodstuffs through the use of augmented reality features, the method comprising:

launching an application on a mobile phone that includes a camera;

causing a communication link to be established between the mobile phone and a network-connected server system responsible for supporting the application;

initiating a live view of one or more foodstuffs that is captured by the camera;

automatically identifying each foodstuff; and

generating an augmented reality feature that is at least partially hidden beneath a particular foodstuff of the one or more foodstuffs when viewed using the mobile phone.

2. The method of claim 1, wherein each foodstuff is a food or a beverage.

3. The method of claim 1, wherein the augmented reality feature is a digital creature or a digital item.

4. The method of claim 1, wherein the particular foodstuff is one of multiple foodstuffs on a plate, a cup, or a bowl that include distinct augmented reality features.

5. The method of claim 1, further comprising:

animating a portion of the augmented reality feature that is exposed from the particular foodstuff.

6. The method of claim 1, further comprising:

animating a surface of the particular foodstuff,

wherein said animating indicates where the augmented reality feature is hidden within the particular foodstuff.

7. The method of claim 1, further comprising:

monitoring the live view of the one or more foodstuffs that is captured by the camera;

determining that the particular foodstuff has been partially or entirely consumed by an individual;

displaying additional portions of the augmented reality feature based on an amount of the particular foodstuff that has been consumed by the individual;

generating metadata specifying that the additional portions of the augmented reality feature have been exposed; and

transmitting the metadata to the network-connected server system.

8. The method of claim 7, wherein reception of the metadata prompts the network-connected server system to create a database record that specifies the augmented reality feature should be made accessible to the individual through the application.

9. A method for creating augmented reality features that can be used to incentivize the consumption of foodstuffs, the method comprising:

launching an application on a computing device that includes an optical sensor;

causing a communication link to be established between the computing device and a network-connected server system responsible for supporting the application;

initiating a live view of a foodstuff that is captured by the optical sensor;

receiving first input from a user indicative of a selection of the foodstuff;

generating an augmented reality feature that is at least partially hidden from an individual beneath the foodstuff when viewed using the computing device; and

saving an association between the foodstuff and the augmented reality feature in a memory.

10. The method of claim 9, further comprising:

receiving second input from the user indicative of a specification of a characteristic of the augmented reality feature.

11. The method of claim of claim 10, wherein the characteristic of the augmented reality feature is an animation, a color, or a feature type.

12. The method of claim 9, wherein the foodstuff is one of multiple foodstuffs on a plate.

13. The method of claim 12, wherein some or all of the multiple foodstuffs are associated with a corresponding augmented reality feature.

14. The method of claim 12, further comprising:

automatically identifying each foodstuff of the multiple foodstuffs by

performing image processing in order to identify shapes, colors, or some combination thereof that are indicative of certain foodstuffs, or

recognizing a branded code that conveys information about a corresponding foodstuff.

15. The method of claim 14, wherein the branded code is affixed to packaging that secures the corresponding foodstuff or a loose item secured within the packaging.

16. The method of claim 14, wherein the branded code includes machine-readable elements, human-readable elements, structural elements, or some combination thereof.

17. The method of claim 14, wherein the branded code includes a bar code or a Quick Response (QR) code that is readily scannable by the application.

18. The method of claim 14, wherein the branded code is included on a purchase receipt that represents proof of purchase of the foodstuff.

19. The method of claim 9, further comprising:

monitoring the live view of the foodstuff that is captured by the optical sensor;

determining that the foodstuff has been partially or entirely consumed by the individual; and

exposing additional portions of the augmented reality feature as the individual consumes the foodstuff.

20. The method of claim 19, further comprising:

transmitting metadata regarding the foodstuff, the augmented reality feature, or both to the network-connected server system,

wherein reception of the metadata prompts the network-connected server system to create a database record that specifies the augmented reality feature should be made accessible to the individual through the application.

21. A computer system comprising:

a processor operable to execute instructions stored in a memory; and

the memory that includes specific instructions for creating augmented reality features in order to incentivize the consumption of foodstuffs,

wherein execution of the specific instructions cause the processor to;

capture a view of a foodstuff using an optical sensor;

generate an augmented reality feature having

a first portion that is hidden beneath the foodstuff, and

a second portion that is exposed from beneath the particular foodstuff and visible when viewed using the computing device;

monitoring the live view of the foodstuff that is captured by the optical sensor;

determining that the foodstuff has been partially or entirely consumed; and

presenting a third portion of the augmented reality feature that is exposed due to consumption of the foodstuff.

22. The computer system of claim 21, wherein the third portion is initially part of the first portion that is hidden beneath the foodstuff.

23. The computer system of claim 21, wherein the processor and the memory are included in a network-connected server system responsible for supporting the application.

24. The computer system of claim 21, wherein capturing the view of the foodstuff using the optical sensor comprises:

initiating a live view of the foodstuff;

or

capturing a static image of the foodstuff.

25. The computer system of claim 21, wherein execution of the specific instructions further cause the processor to:

perform image processing on individual video frames of a live view of the foodstuff.

26. The computer system of claim 25, wherein said performing comprises:

performing image segmentation on the individual video frames in order to automatically detect changes in an amount of the foodstuff.