IDENTIFICATION USING VIDEO ANALYTICS TOGETHER WITH INERTIAL SENSOR DATA

• US 20150085111A1
• Filed: 09/25/2013
• Published: 03/26/2015
• Est. Priority Date: 09/25/2013
• Status: Abandoned Application

• Current Assignee: Symbol Technologies, Inc. (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, Inc. (Zebra Technologies Corporation)
• Inventors: LAVERY, RICHARD J
• Application Number:
  US14/036,142
• Publication Number:
  US 20150085111A1
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H04W 4/027 using movement velocity, acceleration information
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]

Abstract

A technique for identification using video analytics together with inertial sensor data is described. The technique includes capturing video of an environment and tracking particular users in the captured video. Motion signals are received from at least one inertial sensor of at least one mobile communication device being carried by a user. The video motion of each tracked user in the captured video and the motion signals of each mobile communication device are correlated in order to associate one of the mobile communication devices with a particular tracked user in the video.

Claims

1. A method for identification using video analytics together with inertial sensor data, the method comprising:

capturing video of an environment;

tracking particular users in the captured video;

receiving motion signals from at least one inertial sensor of at least one mobile communication device being carried by a user; and

correlating the video motion of each tracked user in the captured video and the motion signals of each mobile communication device to associate one of the mobile communication devices with a particular tracked user in the video.

2. The method of claim 1, wherein the at least one inertial sensor includes one or more of an accelerometer and a gyroscope.

3. The method of claim 1, wherein receiving also includes receiving an identification from the mobile communication device to identify the user being tracked in the video.

4. The method of claim 1, wherein correlating includes keeping a record of video motions and motion signals over time to provide an increased confidence in correlation for longer time periods.

5. The method of claim 1, wherein correlating includes calibrating the timing of the input signals versus the captured video such that the video motion and motion signals are time-aligned.

6. The method of claim 1, further comprising:

determining a location of the mobile device in the environment using a locationing system; and

associating the location with a particular user in the video.

7. A system for identification using video analytics together with inertial sensor data, the system comprising:

an imaging apparatus operable to capture video of an environment;

a backend server coupled to the imaging device, the server operable to track particular users in the captured video;

a wireless communication network coupled to the backend server; and

at least one mobile communication device operable to be carried by a user and coupled to the backend server through the communication network, the at least one mobile communication device including at least one inertial sensor,

wherein the backend server further operable to track a video motion of users in the video and input motion signals from the inertial sensors of the at least one mobile communication device, the backend server operable to correlate the video motion of each user and the motion signals of each mobile communication device to associate one of the mobile communication devices with a particular tracked user in the video.

8. The system of claim 7, wherein the at least one inertial sensor includes one or more of an accelerometer and a gyroscope.

9. The system of claim 7, wherein mobile communication device also provides an identification to the backend server to identify the user being tracked in the video.

10. The system of claim 7, wherein the backend server is further operable to keep a record of video motions and motion signals over time to provide an increased confidence in correlation for longer time periods.

11. The system of claim 7, wherein the backend server is further operable to calibrate the timing of the input signals versus the captured video such that the video motion and motion signals are time-aligned.

12. The system of claim 7, further comprising a locationing system operable to determine a location of the mobile device in the environment and associate the location with a particular user in the video.

13. The system of claim 12, wherein the locationing system includes a plurality of ultrasonic transmitters at known fixed positions in the environment and operable to provide ultrasonic signals to be received by the mobile communication device, wherein the mobile device is further operable to measure timing information of these received ultrasonic signals for the backend server to determine a location of the mobile device in the environment.

14. The system of claim 7, wherein the imaging apparatus is at least one video camera.