Video identification and analytical recognition system

• US 9,762,865 B2
• Filed: 08/04/2015
• Issued: 09/12/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: James Carey
• Original Assignee: James Carey
• Inventors: Carey, James
• Primary Examiner(s): Aghevli, Reza
• Application Number:
  US14/817,871
• Publication Number:
  US 20150341599A1
• Time in Patent Office: 770 Days
• Field of Search: None
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control
• CPC Class Codes: H04W 8/005 Discovery of network devices, e.g. terminals

Abstract

An analytical recognition system includes a video camera, an antenna, one or more processors, and one or more memories. The one or more memories store instructions that, when executed by the one or more processors, cause the one or more processors to function as a data analytics module and an investigation module. The video camera is configured to capture video data. The antenna is configured to capture mobile communication device data. The data analytics module is configured to correlate the video data and the mobile communication device data, generate a profile of a person associated with the video data and the mobile communication device data, detect a behavior of the person, and store behavioral data in the profile. The investigation module is configured to mine the video data, the mobile communication device data, the profile data, and/or sales data, and generate a report based on the mining.

Claims

1. An analytical recognition system, comprising:

a video camera configured to capture video data;

an antenna configured to capture mobile communication device data;

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to function as;

a data analytics module configured to;

correlate the video data and the mobile communication device data,

generate a profile of a person associated with the video data and the mobile communication device data, the profile including profile data including any one or a combination of the captured video data and the captured mobile communication data,

detect a behavior of the person by correlating any one or a combination of the video data, the mobile communication device data, and the profile data with any one or a combination of a mapping of aisle locations at a premises, a mapping of merchandise locations at the premises, and a mapping of shelf locations at the premises, and

store in the profile behavioral data corresponding to the behavior,

wherein at least one of the memories stores instructions that, when executed by at least one of the processors, cause the at least one of the processors to function as;

an investigation module configured to;

mine, based on report criteria, any one or a combination of the video data, the mobile communication device data, the profile data, and sales data; and

generate a report based on the mining of the any one or the combination of the video data, the mobile communication device data, and the profile data,

wherein the report includes a sales close rate corresponding to any one or a combination of an item of merchandise, a merchandise category, an aisle at the premises, a shelf at the premises, or a predetermined location at the premises.

2. The system of claim 1, wherein the data analytics module is configured to:

determine any one or a combination of an arrival time of the person at a location and a departure time of the person at the location based on any one or a combination of the video data and the mobile communication device data, and

correlate the video data and the mobile communication device data based on any one or a combination of the arrival time and the departure time.

3. The system of claim 1, wherein the video camera is one of a plurality of video cameras included in the system and configured to capture a plurality of video data, the antenna is one of a plurality of antennae included in the system and configured to capture a plurality of mobile communication device data, and the data analytics module is further configured to:

correlate the plurality of video data and the plurality of mobile communication device data to generate a plurality of profiles of a plurality of people, respectively associated with the plurality of video data and the plurality of mobile communication device data.

4. The system of claim 3, wherein the plurality of video cameras and the plurality of antennae are located at a plurality of premises.

5. The system of claim 4, further comprising a user interface configured to enable the plurality of profiles to be mined based on a user-inputted criterion.

6. The system of claim 3, wherein at least one of the plurality of the antennae is affixed to at least one of the plurality of the video cameras and at least one of the plurality of antennae is located remote from the plurality of cameras.

7. The system of claim 1, wherein:

the profile includes any one or a combination of the captured video data, the captured mobile communication device data, temporal data associated with the captured video data or the captured mobile communication device data, and location data associated with the captured video data or the captured mobile communication device data;

the captured video data includes any one or a combination of a captured still image and video footage;

the mobile communication device data includes any one or a combination of a WiFi identifier, a media access control (MAC) identifier, a Bluetooth identifier, a cellular identifier, a near field communication identifier, and a radio frequency identifier associated with the mobile communication device in communication with the antenna;

the temporal data includes any one or a combination of a time the video data is captured and a time the mobile communication device data is captured; and

the location data includes any one or a combination of a location at which the video data is captured and a location at which the mobile communication device data is captured.

8. The system of claim 1, wherein the data analytics module is further configured to add to the profile, based on the correlated video data and mobile communication device data, any one or a combination of a number of visits of the person to one or more premises and a frequency of visits of the person to the one or more premises.

9. The system of claim 1, wherein the data analytics module is further configured to identify the person based on a comparison between data from a first source and any one or a combination of the captured video data, the captured mobile communication device data, the correlated video data and mobile communication device data, and the profile, wherein the first source includes any one or a combination of a non-government database, a government database, and one or more previously generated profiles.

10. The system of claim 1, wherein the investigation module is further configured to receive investigation criteria and mine any one or a combination of the video data, the mobile communication device data, and the profile data based on the criteria.

11. The system of claim 10, wherein the investigation criteria includes a time frame, and wherein the investigation module is further configured to generate a list of people whose presence was detected on a premises during the time frame.

12. The system of claim 10, wherein the investigation module is further configured to determine a current location of the person by detecting a signal that matches the mobile communication device data obtained at a first location.

13. The system of claim 1, wherein the antenna includes any one or a combination of a WiFi antenna, a media access control (MAC) antenna, a Bluetooth antenna, a cellular antenna, a near field communication antenna, and a radio frequency identification antenna.

14. The system of claim 1, wherein the data analytics module is further configured to assign the person to a positive list, an undetermined list, or a negative list based on any one or a combination of the video data, the mobile communication device data, the profile data, and a user-inputted criteria.

15. The system of claim 14, wherein the data analytics module is further configured to assign the person to the positive list based on a determination that any one or a combination of the video data, the mobile communication device data, and the profile corresponds to an employee or to a person on a predetermined list of people.

16. The system of claim 1, wherein the antenna is configured to capture the mobile communication device data by wirelessly receiving data from the mobile communication device located within a range of the antenna.

17. The system of claim 1, wherein the data analytics module is configured to detect the behavior of the person by extracting behavioral information from any one or a combination of the video data and the mobile communication device data, and

wherein the behavior includes any one or a combination of looking in a direction, reaching for an item of merchandise, purchasing the item of merchandise, traveling along a path at the premises, visiting an aisle or a location at the premises, spending an amount of time at the premises, spending an amount of time at the location at the premises, and visiting the premises on a number of separate instances.

18. The system of claim 17, wherein the data analytics module is configured to classify the person as a new customer or a repeat customer at the premises based on premises visit data stored in the profile, and add to the profile, or update in the profile, an indicator of whether the person is a new customer or a repeat customer at the premises.

19. The system of claim 1, wherein the data analytics module is further configured to:

generate, based on any one or a combination of the video data and the mobile communication device data, location data corresponding to the behavior, and

store the location data in the profile in association with the behavioral data.