Incentivizing foodstuff consumption through the use of augmented reality features

• US 10,019,628 B1
• Filed: 08/08/2017
• Issued: 07/10/2018
• Est. Priority Date: 05/16/2017
• Status: Active Grant

• Current Assignee: Mnemonic Health, Inc.
• Original Assignee: Sounds Food, Inc.
• Inventors: Henderson, Mark Lindsay
• Primary Examiner(s): Zalalee, Sultana M
• Application Number:
  US15/672,142
• Time in Patent Office: 336 Days
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06K 19/06028 using bar codes
• CPC Class Codes: G06K 19/06037 multi-dimensional coding
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04W 4/02 Services making use of location information

Abstract

Introduced here are techniques for incentivizing the consumption of foodstuffs through the use of augmented reality features. A parent may initiate an application on a mobile phone and scan a plate of one or more foodstuffs by initiating a live view captured by a camera. The parent can then select a given foodstuff that a child is unlikely to consume. When the child views the given foodstuff through the mobile phone, a portion of an augmented reality feature could be shown. For example, the child may be able to see the tail of a creature, a corner of a item, etc. As the child consumes the given foodstuff, additional portion(s) of the augmented reality feature are exposed, and the augmented reality feature becomes increasingly visible. Such a technique incentivizes the child to continue eating the given foodstuff so that the augmented reality feature can be fully seen.

Claims

1. A computer-implemented method comprising:

launching a software program on a computing device that includes a camera;

initiating a live view of one or more foodstuffs that is captured by the camera;

automatically identifying each foodstuff; and

generating an augmented reality feature that is at least partially hidden beneath a particular foodstuff of the one or more foodstuffs when viewed using the computing device.

2. The computer-implemented method of claim 1, wherein each foodstuff is a food or a beverage.

3. The computer-implemented method of claim 1, wherein the augmented reality feature is a digital creature or a digital item.

4. The computer-implemented method of claim 1, wherein the particular foodstuff is one of multiple foodstuffs on a plate, a cup, or a bowl that include distinct augmented reality features.

5. The computer-implemented method of claim 1, further comprising:

animating a portion of the augmented reality feature that is exposed from the particular foodstuff.

6. The computer-implemented method of claim 1, further comprising:

animating a surface of the particular foodstuff,

wherein said animating indicates where the augmented reality feature is hidden within the particular foodstuff.

7. The computer-implemented method of claim 1, further comprising:

monitoring the live view of the one or more foodstuffs that is captured by the camera;

determining that the particular foodstuff has been at least partially consumed by an individual;

displaying an additional portion of the augmented reality feature based on an amount of the particular foodstuff that has been consumed by the individual;

generating metadata specifying that the additional portion of the augmented reality feature has been exposed.

8. The computer-implemented method of claim 7, further comprising:

causing a communication link to be established between the computing device and a network-connected server system responsible for supporting the software program; and

transmitting the metadata to the network-connected server system.

9. The computer-implemented method of claim 8, wherein reception of the metadata prompts the network-connected server system to create a database record that specifies the augmented reality feature should be made accessible to the individual through the software program.

10. A computer-implemented method comprising:

launching a software program on a computing device that includes an optical sensor;

causing the optical sensor to capture a view of a foodstuff;

receiving first user input indicative of a selection of the foodstuff;

generating an augmented reality feature that is at least partially hidden beneath the foodstuff when viewed using the computing device; and

saving an association between the foodstuff and the augmented reality feature in a memory.

11. The computer-implemented method of claim 10, wherein the view includes a moving image or a static image.

12. The computer-implemented method of claim 10, further comprising:

receiving second user input indicative of a specification of a characteristic of the augmented reality feature.

13. The computer-implemented method of claim 12, wherein the characteristic of the augmented reality feature is an animation, a color, or a feature type.

14. The computer-implemented method of claim 10, wherein the foodstuff is one of multiple foodstuffs on a plate.

15. The computer-implemented method of claim 14, further comprising:

automatically identifying each foodstuff of the multiple foodstuffs by

performing image processing in order to identify shapes, colors, or a combination thereof that are indicative of certain foodstuffs, or

recognizing a branded code that conveys information about a corresponding foodstuff.

16. The computer-implemented method of claim 15, wherein the branded code is affixed to packaging that secures the corresponding foodstuff or a loose item secured within the packaging.

17. The computer-implemented method of claim 15, wherein the branded code includes a machine-readable element, a human-readable element, a structural element, or a combination thereof.

18. The computer-implemented method of claim 15, wherein the branded code includes a bar code or a Quick Response (QR) code that is scannable by the software program.

19. The computer-implemented method of claim 10, further comprising:

causing a communication link to be established between the computing device and a network-connected server system responsible for supporting the software program; and

transmitting metadata regarding the foodstuff, the augmented reality feature, or both to the network-connected server system.

20. A computer system comprising:

a processor operable to execute instructions stored in a memory; and

the memory that includes specific instructions for creating augmented reality features to incentivize the consumption of foodstuffs,

wherein execution of the specific instructions cause the processor to;

enable a first image of a foodstuff to be captured using an optical sensor;

generate an augmented reality feature having

a first portion hidden beneath the foodstuff, and

a second portion exposed from beneath the foodstuff;

enable a second image of the foodstuff to be captured using the optical sensor;

examine the second image to determine whether the foodstuff has been at least partially consumed; and

responsive to a determination that the foodstuff has been at least partially consumed,

cause a third portion of the augmented reality feature to be exposed from beneath the foodstuff due to consumption of the foodstuff.

21. The computer system of claim 20, wherein the third portion is initially part of the first portion that is hidden beneath the foodstuff.

22. The computer system of claim 20, wherein the processor and the memory are included in a network-connected server system responsible for supporting a software program executing on a computing device.

23. The computer system of claim 20, wherein the first image and the second image are static images or moving images included in a live view.