REAL-TIME IMAGE ANALYSIS FOR PROVIDING HEALTH RELATED INFORMATION

• US 20120229624A1
• Filed: 01/01/2012
• Published: 09/13/2012
• Est. Priority Date: 03/08/2011
• Status: Abandoned Application

• Current Assignee: Bank of America Corp.
• Original Assignee: Bank of America Corp.
• Inventors: Calman, Matthew A., Ross, Erik Stephen, Hamilton, Alfred
• Application Number:
  US13/342,052
• Publication Number:
  US 20120229624A1
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44231 Monitoring of peripheral device or external card, e.g. to detect processing problems in a handheld device or the failure of an external recording device configuring of peripheral devices in general G06F9/4411; reporting information sensed by appliance or service execution status of appliance services in a home automation network H04L12/2823; monitoring the status of connected device in data switched networks H04L43/0817
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

Systems, methods, and computer program products are provided for using real-time video analysis and AR or the like to assist the user of mobile devices with commerce activities. Through the use of real-time vision object recognition objects, logos, artwork, products, locations and other features that can be recognized in the real-time video stream can be matched to health related information regarding the object and/or the user. The health related information may be presenting to the user of the mobile device in conjunction with display of the associated object in a live video stream.

Claims

1. A method for providing health related information, the method comprising:

identifying, via a computing device processor, one or more products proximate in location to a mobile communication device;

determining, via a computing device processor, which of the one or more identified products have associated health related information; and

presenting, via a display of the live video stream on the mobile communication device, one or more health related information indicators, each health related information indicator presented proximate a location of the one or more products determined to have associated health related information.

2. The method of claim 1, wherein identifying one or more products comprises identifying one or more images in a video stream captured on a mobile communication device that correspond to one or more products.

3. The method of claim 1, wherein identifying one or more products comprises identifying one or more objects in an image captured on a mobile communication device that corresponds to one or more products.

4. The method of claim 1, wherein identifying one or more products comprises identifying one or more products based at least in part on global positioning system information associated with the mobile communication device or other information indicating a location of the mobile communication device.

5. The method of claim 1, wherein the health related information comprises at least one of nutrition information, caloric information, medical indication information, proper medicinal treatment information, or financial impact information.

6. The method of claim 1, wherein determining which of the identified products have associated health related information comprises comparing the identified products to a database listing of products currently associated with health related information.

7. The method of claim 6, wherein the database listing comprises a listing of product health related information of products specific to a retail location at which the products were identified or a listing of product health related information of products specific to at least one physical or online merchant at which a user of the mobile communication device typically shops.

8. The method of claim 1, wherein the health related information indicator is associated with health related information dictated at least in part by user preference.

9. The method of claim 1, further comprising:

storing information related to at least one medical or health need or want of a social network member of a user of the mobile communication device;

determining whether any of the identified products meet the health need or want of the social network member; and

presenting at least one health related information indicator indicating that at least one identified product meets the health need or want of the social network member.

10. The method of claim 1, further comprising:

storing information related to at least one medical or health characteristic of a user of the mobile communication device;

determining whether any of the identified products effect the health characteristic of the user; and

presenting at least one health related information indicator indicating that at least one identified product effects the health characteristic of the user.

11. The method of claim 1, wherein presenting the one or more health related information indicators comprises presenting one or more health related information indicators associated with health related information related to medical first aid.

12. An apparatus for providing health related information, the apparatus comprising:

a computing platform having a processor, a memory in communication with the processor, and

image capture logic stored in the memory, executable by the processor and configured to capture at least one image, and

product identification logic stored in the memory, executable by the processor and configured to identify which objects in the at least one image captured by a mobile communication device correspond to a product, and

health related information logic stored in the memory, executable by the processor and configured to determine whether the identified product has associated health related information; and

health related information presentation logic stored in the memory, executable by the processor and configured to present, on a display of the mobile communication device, one or more health related information indicators, each health related information indicator presented proximate a location of the product determined to have associated health related information.

13. The apparatus of claim 12, wherein the product identification logic is configured to identify one or more images in a video stream captured on a mobile communication device that correspond to one or more products.

14. The apparatus of claim 12, wherein the product identification logic is configured to identify one or more objects in an image captured on a mobile communication device that corresponds to one or more products.

15. The apparatus of claim 12, wherein the product identification logic is configured to identify one or more products based at least in part on global positioning system information associated with the mobile communication device or other information indicating a location of the mobile communication device.

16. The apparatus of claim 12, wherein the health related information comprises at least one of nutrition information, caloric information, medical indication information, proper medicinal treatment information, or financial impact information.

17. The apparatus of claim 12, wherein the health related information logic is configured to compare the identified products to a database listing of products currently associated with health related information.

18. The apparatus of claim 17, wherein the database listing comprises a listing of product health related information of products specific to a retail location at which the products were identified or a listing of product health related information of products specific to at least one physical or online merchant at which a user of the mobile communication device typically shops.

19. The apparatus of claim 12, wherein health related information indicator is associated with health related information dictated at least in part by user preference.

20. The apparatus of claim 12, wherein the health related information logic is configured to:

store information related to at least one medical or health need or want of a social network member of a user of the mobile communication device; and

determine whether any of the identified products meet the health need or want of the social network member; and

wherein the health related information presentation logic is configured to;

present at least one health related information indicator indicating that at least one identified product meets the health need or want of the social network member.

21. The apparatus of claim 12, wherein the health related information logic is configured to:

store information related to at least one medical or health characteristic of a user of the mobile communication device; and

determine whether any of the identified products effect the health characteristic of the user; and

wherein the health related information presentation logic is configured to;

present at least one health related information indicator indicating that at least one identified product effects the health characteristic of the user.

22. The apparatus of claim 12, wherein the health related information presentation logic is configured to present one or more health related information indicators associated with health related information related to medical first aid.

23. A computer program product comprising a non-transitory computer-readable medium comprising computer-executable instructions for providing health related information, the instructions comprising:

instructions for identifying which objects in an image captured on a mobile communication device correspond to a product;

instructions for determining which of one or more products identified in the image have associated health related information; and

instructions for presenting one or more health related information indicators in the image displayed on the mobile communication device, each health related information indicator presented proximate a location of the one or more determined products.

24. The computer program product of claim 23, wherein identifying one or more products comprises identifying one or more images in a video stream captured on a mobile communication device that correspond to one or more products.

25. The computer program product of claim 23, wherein identifying one or more products comprises identifying one or more objects in an image captured on a mobile communication device that corresponds to one or more products.

26. The computer program product of claim 23, wherein identifying one or more products comprises identifying one or more products based at least in part on global positioning system information associated with the mobile communication device or other information indicating a location of the mobile communication device.

27. The computer program product of claim 23, wherein the health related information comprises at least one of nutrition information, caloric information, medical indication information, proper medicinal treatment information, or financial impact information.

28. The computer program product of claim 23, wherein determining which of the identified products have associated health related information comprises comparing the identified products to a database listing of products currently associated with health related information.

29. The computer program product of claim 28, wherein the database listing comprises a listing of product health related information of products specific to a retail location at which the products were identified or a listing of product health related information of products specific to at least one physical or online merchant at which a user of the mobile communication device typically shops.

30. The computer program product of claim 23, wherein the health related information indicator is associated with health related information dictated at least in part by user preference.

31. The computer program product of claim 23, wherein the instructions further comprise instructions for:

storing information related to at least one medical or health need or want of a social network member of a user of the mobile communication device;

determining whether any of the identified products meet the health need or want of the social network member; and

presenting at least one health related information indicator indicating that at least one identified product meets the health need or want of the social network member.

32. The computer program product of claim 23, wherein the instructions further comprise instructions for:

storing information related to at least one medical or health characteristic of a user of the mobile communication device;

determining whether any of the identified products effect the health characteristic of the user; and

presenting at least one health related information indicator indicating that at least one identified product effects the health characteristic of the user.

33. The computer program product of claim 23, wherein presenting the one or more health related information indicators comprises presenting one or more health related information indicators associated with health related information related to medical first aid.

34. A method for providing health related information, the method comprising:

identifying, via a server in communication with a mobile communication device, which objects in an image captured on a mobile communication device correspond to a product;

determining, via the server, which of one or more products identified in the image have associated health related information; and

communicating instructions to the mobile communication device, via the server, for presenting a display of the image on the mobile communication device, one or more health related information indicators, each health related information indicator presented proximate a location of the one or more products determined to have associated health related information.

35. The method of claim 34, identifying one or more products comprises identifying one or more images in a video stream captured on a mobile communication device that correspond to one or more products.

36. The method of claim 34, wherein identifying one or more products comprises identifying one or more objects in an image captured on a mobile communication device that corresponds to one or more products.

37. The method of claim 34, wherein identifying one or more products comprises identifying one or more products based at least in part on global positioning system information associated with the mobile communication device or other information indicating a location of the mobile communication device.

38. The method of claim 34, wherein the health related information comprises at least one of nutrition information, caloric information, medical indication information, proper medicinal treatment information, or financial impact information.

39. The method of claim 34, wherein determining which of the identified products have associated health related information comprises comparing the identified products to a database listing of products currently associated with health related information.

40. The method of claim 39, wherein the database listing comprises a listing of product health related information of products specific to a retail location at which the products were identified or a listing of product health related information of products specific to at least one physical or online merchant at which a user of the mobile communication device typically shops.

41. The method of claim 34, wherein the health related information indicator is associated with health related information dictated at least in part by user preference.

42. The method of claim 34, further comprising:

storing information related to at least one medical or health need or want of a social network member of a user of the mobile communication device;

determining whether any of the identified products meet the health need or want of the social network member; and

communicating instructions to the mobile communication device, via the server, for presenting at least one health related information indicator indicating that at least one identified product meets the health need or want of the social network member.

43. The method of claim 34, further comprising:

storing information related to at least one medical or health characteristic of a user of the mobile communication device;

determining whether any of the identified products effect the health characteristic of the user; and

communicating instructions to the mobile communication device, via the server, for presenting at least one health related information indicator indicating that at least one identified product effects the health characteristic of the user.

44. The method of claim 34, wherein communicating instructions to the mobile communication device for presenting the one or more health related information indicators comprises communicating instructions to the mobile communication device, via the server, for presenting one or more health related information indicators associated with health related information related to medical first aid.

45. An apparatus for providing health related information, the apparatus comprising:

a server having a processor, a memory in communication with the processor, and

communication logic stored in the memory, executable by the processor and configured to receive data from a mobile communication device, the data corresponding to one or more objects in an image captured by the mobile communication device;

product identification logic stored in the memory, executable by the processor and configured to identify which objects in the image captured by the mobile communication device correspond to a product;

health related information logic stored in the memory, executable by the processor and configured to determine whether the identified product has associated health related information; and

health related information presentation logic stored in the memory, executable by the processor and configured to communicate instructions for presenting, on a display of the mobile communication device, one or more health related information indicators, each health related information indicator presented in the image proximate a location of the product determined to have associated health related information.

46. The apparatus of claim 45, wherein the product identification logic is configured to identify one or more images in a video stream captured on a mobile communication device that correspond to one or more products.

47. The apparatus of claim 45, wherein the product identification logic is configured to identify one or more objects in an image captured on a mobile communication device that corresponds to one or more products.

48. The apparatus of claim 45, wherein the product identification logic is configured to identify one or more products based at least in part on global positioning system information associated with the mobile communication device or other information indicating a location of the mobile communication device.

49. The apparatus of claim 45, wherein the health related information comprises at least one of nutrition information, caloric information, medical indication information, proper medicinal treatment information, or financial impact information.

50. The apparatus of claim 45, wherein the health related information logic is configured to compare the identified products to a database listing of products currently associated with health related information.

51. The apparatus of claim 50, wherein the database listing comprises a listing of product health related information of products specific to a retail location at which the products were identified or a listing of product health related information of products specific to at least one physical or online merchant at which a user of the mobile communication device typically shops.

52. The apparatus of claim 45, wherein the health related information indicator is associated with health related information dictated at least in part by user preference.

53. The apparatus of claim 45, wherein the health related information logic is configured to:

store information related to at least one medical or health need or want of a social network member of a user of the mobile communication device; and

determine whether any of the identified products meet the health need or want of the social network member; and

wherein the health related information presentation logic is configured to;

present at least one health related information indicator indicating that at least one identified product meets the health need or want of the social network member.

54. The apparatus of claim 45, wherein the health related information logic is configured to:

store information related to at least one medical or health characteristic of a user of the mobile communication device; and

determine whether any of the identified products effect the health characteristic of the user; and

wherein the health related information presentation logic is configured to;

present at least one health related information indicator indicating that at least one identified product effects the health characteristic of the user.

55. The apparatus of claim 45, wherein the health related information presentation logic is configured to:

present one or more health related information indicators associated with health related information related to medical first aid.

56. A computer program product comprising a non-transitory computer-readable medium comprising computer-executable instructions for execution on a server in communication with a mobile communication device, the instructions for providing health related information, the instructions comprising:

instructions for identifying which objects in an image captured on a mobile communication device correspond to a product;

instructions for determining which of one or more products identified in the image have associated health related information; and

instructions for communicating instructions to the mobile communication device for presenting one or more health related information indicators with the image displayed on the mobile communication device, each health related information indicator presented proximate a location of the one or more determined products.