Crowd-sourced vision-based information collection

• US 10,169,654 B2
• Filed: 04/20/2017
• Issued: 01/01/2019
• Est. Priority Date: 01/06/2016
• Status: Active Grant

• Current Assignee: OrCam Technologies Ltd
• Original Assignee: OrCam Technologies Ltd
• Inventors: Wexler, Yonatan, Shashua, Amnon
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US15/493,101
• Publication Number:
  US 20170223076A1
• Time in Patent Office: 621 Days
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06K 7/10366 the interrogation device being adapted for miscellaneous applications
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/35 : Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 20/41 : Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 40/113 : Recognition of static hand signs
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 : Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G08B 21/18 : Status alarms G08B21/02 takes precedence
• CPC Class Codes: H04L 65/61 : for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04N 1/00342 : with a radio frequency tag transmitter or receiver storage of additional information in an RFID tag attached to an image bearing paper sheet H04N1/32138
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/56 : provided with illuminating means
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A wearable apparatus is provided for capturing and processing images from an environment of a user. In one implementation, a server-based system includes a data interface for receiving a data stream including image-based information. The image based information may include images captured by a camera present on a particular wearable apparatus from among the plurality of wearable apparatuses worn by different users. The server-based system also includes at least one processing device programmed to receive the data stream from each of the plurality of wearable apparatuses using the data interface, and analyze the data streams to determine at least one trait common to two or more of the different users of the plurality of wearable apparatuses. The at least one processing device may also be programmed to store in a database information relating to the determined at least one trait.

Claims

1. A server-based system for interacting with a plurality of wearable apparatuses each being associated with a different user, the system comprising:

a data interface; and

at least one processing device programmed to;

receive, via the data interface and for each of the plurality of wearable apparatuses, a data stream including image-based information associated with images captured by a camera present on a particular wearable apparatus from among the plurality of wearable apparatuses;

analyze the image-based information of the data streams received from each of the plurality of wearable apparatuses to determine at least one trait common to two or more of the different users of the plurality of wearable apparatuses;

determine at least one frequency related to the at least one trait;

store, in at least one database, the determined at least one frequency related to the at least one trait;

store, in the at least one database, information relating to the determined at least one trait; and

report the information relating to the determined at least one trait to at least one user of at least one of the plurality of wearable apparatuses.

2. The system of claim 1, wherein the at least one processing device is further configured to output to a graphical display information based on the stored information relating to the determined at least one trait.

3. The system of claim 1, wherein the at least one processing device is further configured to respond to a received query with information based on the stored information relating to the determined at least one trait.

4. The system of claim 1, wherein at least one of the data streams includes position information identifying locations where the images were captured by the cameras present on the wearable apparatuses.

5. The system of claim 1, wherein at least one of the data streams includes timing information identifying times at which the images were captured by the cameras present on the wearable apparatuses.

6. The system of claim 1, wherein at least one of the data streams includes motion information related to motion of the respective wearable apparatus.

7. The system of claim 1, wherein the at least one processing device is further configured to recognize a location of a particular wearable apparatus based on analysis of the data stream received from the particular wearable apparatus.

8. The system of claim 1, wherein:

the at least one trait includes interaction by the two or more of the different users with a commercial product, and

determining the at least one frequency related to the at least one trait comprises determining a frequency at which the two or more of the different users interact with the commercial product.

9. The system of claim 1, wherein:

the at least one trait includes interaction by the two or more of the different users with a person, and

determining the at least one frequency related to the at least one trait comprises determining a frequency at which the two or more of the different users interact with the person.

10. The system of claim 1, wherein the at least one trait includes a visit by the two or more of the different users to a particular website.

11. The system of claim 10, wherein determining the at least one frequency related to the at least one trait comprises determining a frequency by which the wearers visit the particular website.

12. The system of claim 10, wherein the at least one processing device is further programmed to determine how much time each of the two or more of the different users spends on the particular website.

13. The system of claim 1, wherein the at least one trait is related to a hand gesture of the two or more of the different users.

14. The system of claim 1, wherein storage in the at least one database of information relating to the determined at least one trait is based on demographic information associated with the two or more of the different users.

22. The system of claim 1, wherein the information relating to the determined at least one trait comprises the determined at least one frequency.

15. A method for interacting with a plurality of wearable apparatuses each being associated with a different user, the method comprising:

receiving, for each of the plurality of wearable apparatuses, a data stream including image-based information associated with images captured by a camera present on a particular wearable apparatus from among the plurality of wearable apparatuses;

analyzing the image-based information of the data streams received from each of the plurality of wearable apparatuses to determine at least one trait common to two or more of the different users of the plurality of wearable apparatuses;

determining at least one frequency related to the at least one trait;

storing, in at least one database, the determined at least one frequency related to the at least one trait;

storing, in the at least one database, information relating to the determined at least one trait; and

reporting the information relating to the determined at least one trait to at least one user of at least one of the plurality of wearable apparatuses.

16. The method of claim 15, further comprising outputting to a graphical display information based on the stored information relating to the determined at least one trait.

17. The method of claim 15, wherein at least one of the data streams includes position information identifying locations where the images were captured by the cameras present on the wearable apparatuses.

18. The method of claim 15, wherein at least one of the data streams includes motion information related to motion of the respective wearable apparatus.

19. The method of claim 15, further comprising recognizing a location of a particular wearable apparatus based on analysis of the data stream received from the particular wearable apparatus.

20. The method of claim 15, wherein:

the at least one trait includes interaction by the two or more of the different users with a commercial product, and

determining the at least one frequency related to the at least one trait comprises determining a frequency at which the two or more different users interact with the commercial product.

21. A non-transitory computer readable medium storing computer implementable instructions for carrying out the method of claim 15.

23. The method of claim 15, wherein the information relating to the determined at least one trait comprises the determined at least one frequency.

24. A server-based system for interacting with a plurality of wearable apparatuses each being associated with a different user, the system comprising:

a data interface; and

at least one processing device programmed to;

receive, via the data interface and for each of the plurality of wearable apparatuses, a data stream including image-based information associated with images captured by a camera present on a particular wearable apparatus from among the plurality of wearable apparatuses;

analyze the image-based information of the data streams received from each of the plurality of wearable apparatuses to determine at least one trait common to two or more of the different users of the plurality of wearable apparatuses;

determine at least one time duration related to the at least one trait;

store, in at least one database, the determined at least one time duration related to the at least one trait;

store, in the at least one database, information relating to the determined at least one trait; and

report the information relating to the determined at least one trait to at least one user of at least one of the plurality of wearable apparatuses.

25. The system of claim 24, wherein the at least one trait includes a visit by the two or more of the different users to a particular website.

26. The system of claim 25, wherein the at least one time duration includes how much time each of the two or more of the different users spends on the particular website.

27. A method for interacting with a plurality of wearable apparatuses each being associated with a different user, the method comprising:

receiving, for each of the plurality of wearable apparatuses, a data stream including image-based information associated with images captured by a camera present on a particular wearable apparatus from among the plurality of wearable apparatuses;

analyzing the image-based information of the data streams received from each of the plurality of wearable apparatuses to determine at least one trait common to two or more of the different users of the plurality of wearable apparatuses;

determining at least one time duration related to the at least one trait;

storing, in at least one database, the determined at least one time duration related to the at least one trait;

storing, in the at least one database, information relating to the determined at least one trait; and

reporting the information relating to the determined at least one trait to at least one user of at least one of the plurality of wearable apparatuses.

28. A non-transitory computer readable medium storing computer implementable instructions for carrying out the method of claim 27.