Augmented reality and blockchain technology for decision augmentation systems and methods using contextual filtering and personalized program generation

• US 10,360,495 B2
• Filed: 12/29/2017
• Issued: 07/23/2019
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Suggestic, Inc.
• Original Assignee: Suggestic, Inc.
• Inventors: Chapela, Victor, Corral Corral, Ricardo
• Primary Examiner(s): Gray, Ryan M
• Application Number:
  US15/859,062
• Publication Number:
  US 20180190375A1
• Time in Patent Office: 571 Days
• Field of Search: None
• CPC Class Codes: A23L 33/40 Complete food formulations for specific consumer groups or specific purposes, e.g. infant formula
• CPC Class Codes: A23V 2002/00 Food compositions, function of food ingredients or processes for food or foodstuffs
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/02438 with portable devices, e.g. worn by the patient
• CPC Class Codes: A61B 5/08 Detecting, measuring or recording devices for evaluating the respiratory organs A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/14532 for measuring glucose, e.g. by tissue impedance measurement
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/4806 Sleep evaluation A61B5/4821 takes precedence; devices for inducing sleep A61M21/02
• CPC Class Codes: A61B 5/486 Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/4866 Evaluating metabolism A61B5/083 takes precedence
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/681 Wristwatch-type devices
• CPC Class Codes: A61M 2005/14208 with a programmable infusion control system, characterised by the infusion program
• CPC Class Codes: A61M 5/14244 adapted to be carried by the patient, e.g. portable on the body
• CPC Class Codes: A61M 5/1723 using feedback of body parameters, e.g. blood-sugar, pressure measurement of body parameters A61B5/00
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/02 : Neural networks
• CPC Class Codes: G06N 3/042 : Knowledge-based neural networks; Logical representations of neural networks
• CPC Class Codes: G06N 3/045 : Combinations of networks
• CPC Class Codes: G06N 3/08 : Learning methods
• CPC Class Codes: G06N 5/022 : Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/025 : Extracting rules from data
• CPC Class Codes: G06N 5/045 : Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06T 11/206 : Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09B 19/00 : Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 19/0092 : Nutrition
• CPC Class Codes: G09B 5/02 : with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: G16H 20/60 : relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment

Abstract

Augmented reality and blockchain technology for decision augmentation systems and methods using contextual filtering and personalized program generation are provided herein. An example method includes receiving any of an image, a video stream, and contextual data from a mobile device camera, evaluating any of the image, the video stream, and the contextual data for target food or beverage content by determining ingredient and nutritional components of the target food or beverage content, and applying an augmented reality overlay to the target food or beverage content based on the ingredient and nutritional components.

Claims

1. A method, comprising:

receiving any of an image, a video stream, and contextual data from a mobile device;

evaluating any of the image, the video stream, and the contextual data for target food or beverage content by;

determining ingredient and nutritional components of the target food or beverage content; and

applying an augmented reality overlay to the target food or beverage content based on the ingredient and nutritional components;

wherein a personalized plan is generated by;

obtaining input from empirical evidence-based lifestyle and nutritional programs; and

selecting a dietary program for the user that is based on information comprising genetics, biomarkers, profile, activities, background, clinical data, and combinations thereof;

converting lifestyle programs, dietary plans, nutrition plans, and empirical evidence-based programs into rule sets; and

creating a ruleset stack by merging a plurality of any of lifestyle programs, dietary plans, nutrition plans, and empirical evidence-based programs, wherein the ruleset stack is further configured based on any of goals, biometrics, biomarkers, genetics, demographics, lifestyle and combinations thereof.

2. The method according to claim 1, wherein the target food or beverage content comprises any of restaurant menus, ingredient lists, beverages, food products, groceries, supplements, medications, food labels, and combinations thereof.

3. The method according to claim 2, further comprising generating a dynamic adherence score for any of the target food or beverage content wherein the dynamic adherence score is indicative of how well an item adheres to the personalized plan established by considering for the item any of required time lapse, periodicity, quantity, sequence, food and activity logs, sensors, external data sources, user context, and combinations thereof.

4. The method according to claim 3, wherein the augmented reality overlay comprises any of an icon, text, a number applied proximately to, or over, a portion of the target food or beverage content so as to highlight or obscure items included in the target food or beverage content that do or do not comply with the personalized plan established for the user.

5. The method according to claim 1, further comprising:

receiving biometric or user-generated feedback; and

updating the personalized plan established for the user based on the feedback.

6. The method according to claim 5, wherein the biometric or user-generated feedback comprises any of relations and specific sequences of foods, activities, symptoms, and outcomes.

7. The method according to claim 1, further comprising matching the image, video stream or contextual data to markers and data in a database, wherein the image or the video stream captures any of a restaurant menu, an ingredient list, a beverage, a food product, a grocery item, a supplement, a medication or a food label and data in the database when a match is found, wherein the image or the video stream is converted into an augmented reality enabled item or image.

8. The method according to claim 6, further comprising:

receiving a request for any of a recommended image, markers, and data options overlay; and

calculating and displaying an adherence score that is indicative of how well at least one item adheres to the personalized plan established for the user.

9. The method according to claim 6, further comprising converting the image or video stream into an augmented reality enabled image process pipeline that allows for any of image correction, marker creation, and links for each element to the ingredient and nutritional components using a multi-model, multi-ontology, multi-label deep neural network (mLOM).

10. The method according to claim 9, wherein the mLOM is used to generate specific ingredient, substance and nutritional components of items in the target food or beverage content that are indicative of nutritional values.

11. The method according to claim 1, further comprising prioritizing rulesets in the ruleset stack according to medical needs.

12. The method according to claim 1, wherein at least a portion of the lifestyle programs, dietary plans, nutrition plans, and empirical evidence-based programs selected for use are obtained from a database based on a comparison of the user to a plurality of other users with respect to any of goals, biometrics, biomarkers, genetics, demographics, lifestyle and combinations thereof.

13. The method according to claim 1, further comprising updating the personalized plan using empirical feedback gathered from the user or from biometric measuring devices.

14. The method according to claim 13, wherein the empirical feedback is processed using multivariate causation discovery to find patterns and sequences that best predict one or more desired outcomes, lab test results, environment and personal information based on one or more selected rulesets.

15. The method according to claim 14, further comprising generating a merged program or dietary plan for the user or a group of users based on multiple applied rulesets for the user or the group of users that also have individual ruleset based programs.

16. The method according to claim 15, wherein the personalized plan is updated using updated rulesets, empirical feedback, and active and passive feedback obtained from biometric feedback devices utilized by the user.