SYSTEMS AND METHODS FOR PROVIDING RECOMMENDATIONS BASED ON TRACKED ACTIVITIES

• US 20170064260A1
• Filed: 08/31/2016
• Published: 03/02/2017
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: OrCam Technologies Ltd
• Original Assignee: OrCam Technologies Ltd
• Inventors: Wexler, Yonatan, Shashua, Amnon
• Granted Patent: US 11,115,698 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18105 related to colour
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 : Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 : Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/61 : based on recognised objects
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control

Abstract

A wearable apparatus is provided for capturing and processing images from an environment of a user. In one implementation, a wearable apparatus for monitoring activities includes a wearable image sensor configured to capture a plurality of images from an environment of a user of the wearable apparatus. The wearable apparatus also includes at least one processing device programmed to analyze the plurality of images to identify in one or more of the plurality of images at least one indicator of an activity, and to transmit, to an external device, the at least one indicator of the activity.

Claims

1. A wearable apparatus for monitoring activities, the wearable apparatus comprising:

a wearable image sensor configured to capture a plurality of images from an environment of a user of the wearable apparatus; and

at least one processing device programmed to;

analyze the plurality of images to identify in one or more of the plurality of images at least one indicator of an activity; and

transmit, to an external device, the at least one indicator of the activity.

2. The wearable apparatus of claim 1, wherein the external device is programmed to analyze the at least one indicator of the activity.

3. The wearable apparatus of claim 1, wherein the external device includes at least one of:

a smartphone, a tablet, or a smartwatch.

4. The wearable apparatus of claim 1, wherein the external device includes a server.

5. The wearable apparatus of claim 1, further comprising a transmitter configured to enable wireless pairing with a receiver in the external device.

6. The wearable apparatus of claim 1, wherein the activity involves the user of the wearable apparatus.

7. The wearable apparatus of claim 1, wherein the activity involves a person other than the user of the wearable apparatus.

8. The wearable apparatus of claim 1, wherein the at least one indicator of the activity comprises at least one of:

a description of the activity, a classification of the activity, or a category of the activity.

9. The wearable apparatus of claim 1, wherein the transmitted information enables the external device to select at least one content item based on the activity.

10. The wearable apparatus of claim 9, wherein the external device is programmed to transmit the at least one content item to a device paired with the wearable apparatus.

11. The wearable apparatus of claim 10, wherein the paired device includes a smartphone, a tablet, or a watch.

12. The wearable apparatus of claim 9, wherein the at least one content item comprises at least one keyword.

13. The wearable apparatus of claim 10, wherein the external device is programmed to:

prepare at least one search query based on the at least one keyword; and

transmit the at least one search query to a search engine.

14. The wearable apparatus of claim 9, wherein the at least one content item includes a recommendation.

15. The wearable apparatus of claim 1, wherein the at least one indicator of the activity includes an identifier of an item the user ate.

16. The wearable apparatus of claim 1, wherein the activity includes a fitness activity.

17. The wearable apparatus of claim 1, wherein the activity relates to wellness or health of the user.

18. The wearable apparatus of claim 1, wherein the at least one indicator of the activity includes information related to the surroundings of the user.

19. The wearable apparatus of claim 18, wherein the external device is programmed to provide at least one recommendation related to the surroundings of the user.

20. The wearable apparatus of claim 19, wherein the at least one recommendation relates to adjusting or modifying at least one aspect of the surroundings of the user.

21. The wearable apparatus of claim 20, wherein the at least one aspect of the surroundings of the user includes a light level of a room or a position of a chair in the room.

22. The wearable apparatus of claim 1, wherein the at least one indicator of the activity includes a number of exercise repetitions performed by the user.

23. The wearable apparatus of claim 22, wherein the external device is programmed to provide at least one recommendation based on the number of exercise repetitions performed by the user.

24. The wearable apparatus of claim 1, wherein the external device is programmed to provide at least one graphical representation related to the activity.

25. The wearable apparatus of claim 24, wherein the graphical representation includes an indication associated with the amount of time the user engaged in the activity.

26. The wearable apparatus of claim 24, wherein the graphical representation comprises at least one of:

a graph or a pie chart.

27. The wearable apparatus of claim 24, wherein the external device is programmed to display the at least one graphical representation on a display of the external device.

28. The wearable apparatus of claim 1, wherein the external device is programmed to provide at least one reminder related to the activity.

29. The wearable apparatus of claim 28, wherein the reminder includes a progress level related to the activity.

30. The wearable apparatus of claim 1, wherein the at least one processing device is further programmed to:

analyze the plurality of images to estimate a distance associated with the activity.

31. The wearable apparatus of claim 30, wherein the at least one processing device is further programmed to:

based on the distance associated with the activity, determine whether to transmit the at least one indicator of the activity.

32. The wearable apparatus of claim 30, wherein the at least one processing device is further programmed to:

analyze the plurality of images to identify in one or more of the plurality of images at least one indicator of a second activity;

analyze the plurality of images to estimate a distance associated with the second activity; and

based on the distance associated with the activity and on the distance associated with the second activity, select at least one activity of a group of activities, the group of activities comprising the activity and the second activity.

33. A method for monitoring activities using a wearable apparatus, the method apparatus comprising:

obtaining a plurality of images from an environment of a user of the wearable apparatus; and

analyzing the plurality of images to identify in one or more of the plurality of images at least one indicator of the activity.

34. The method of claim 33, wherein the activity involves the user of the wearable apparatus.

35. The method of claim 33, wherein the activity involves a person other than the user of the wearable apparatus.

36. The method of claim 33, wherein the at least one indicator of the activity comprises at least one of:

a description of the activity, a classification of the activity, or a category of the activity.

37. The method of claim 33, further comprising:

selecting at least one content item based on the activity.

38. The method of claim 37, wherein the at least one content item comprises at least one keyword.

39. The method of claim 38, further comprising:

preparing at least one search query based on the at least one keywords; and

transmitting the at least one search query to a search engine.

40. The method of claim 37, wherein the at least one content item includes a recommendation.

41. The method of claim 33, wherein the at least one indicator of the activity includes an identifier of an item the user ate.

42. The method of claim 33, wherein the activity includes a fitness activity.

43. The method of claim 33, wherein the activity relates to wellness or health of the user.

44. The method of claim 33, wherein the at least one indicator of the activity includes information related to the surroundings of the user.

45. The method of claim 44, further comprising:

providing at least one recommendation related to the surroundings of the user.

46. The method of claim 45, wherein the at least one recommendation relates to adjusting or modifying at least one aspect of the surroundings of the user.

47. The method of claim 46, wherein the at least one aspect of the surroundings of the user includes a light level of a room or a position of a chair in the room.

48. The method of claim 33, wherein the at least one indicator of the activity includes a number of exercise repetitions performed by the user.

49. The method of claim 48, further comprising

providing at least one recommendation based on the number of exercise repetitions performed by the user.

50. The method of claim 33, further comprising:

providing at least one graphical representation related to the activity.

51. The method of claim 50, wherein the graphical representation includes an indication associated with the amount of time the user engaged in the activity.

52. The method of claim 50, wherein the graphical representation comprises at least one of a graph and a pie chart.

53. The method of claim 50, further comprising:

displaying the at least one graphical representation on a display of the external device.

54. The method of claim 33, further comprising:

providing at least one reminder related to the activity.

55. The method of claim 54, wherein the reminder includes a progress level related to the activity.

56. The method of claim 33, further comprising:

analyzing the plurality of images to estimate a distance associated with the activity.

57. The method of claim 56, further comprising:

based on the distance associated with the activity, determining whether to transmit the at least one indicator of the activity.

58. The method of claim 56, further comprising:

analyzing the plurality of images to identify in one or more of the plurality of images at least one indicator of a second activity;

analyzing the plurality of images to estimate a distance associated with the second activity; and

based on the distance associated with the activity and on the distance associated with the second activity, selecting at least one activity of a group of activities, the group of activities comprising the activity and the second activity.

59. A software product stored on a non-transitory computer readable medium and comprising data and computer implementable instructions for carrying out the method of claim 33.