Methods and systems for measuring human interaction

• US 10,049,281 B2
• Filed: 11/11/2013
• Issued: 08/14/2018
• Est. Priority Date: 11/12/2012
• Status: Active Grant

• Current Assignee: Shopperception, Inc.
• Original Assignee: Shopperception, Inc.
• Inventors: Verano, Raul Ignacio, Di Stefano, Ariel Alejandro, Porta, Juan Ignacio
• Primary Examiner(s): Lee, Y
• Assistant Examiner(s): Carter, Richard
• Application Number:
  US14/077,091
• Publication Number:
  US 20140132728A1
• Time in Patent Office: 1,737 Days
• Field of Search: 348 46, 348143, 348152, 345419, 382154, 705 729, 705 271
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method and system for measuring and reacting to human interaction with elements in a space, such as public places (retail stores, showrooms, etc.) is disclosed which may determine information about an interaction of a three dimensional object of interest within a three dimensional zone of interest with a point cloud 3D scanner having an image frame generator generating a point cloud 3D scanner frame comprising an array of depth coordinates for respective two dimensional coordinates of at least part of a surface of the object of interest, within the three dimensional zone of interest, comprising a three dimensional coverage zone encompassing a three dimensional engagement zone and a computing comparing respective frames to determine the time and location of a collision between the object of interest and a surface of at least one of the three dimensional coverage zone or the three dimensional engagement zone encompassed by the dimensional coverage zone.

Claims

1. A system for determining information about an interaction of a first three-dimensional object of interest occurring with a second three-dimensional object of interest within a three dimensional zone of interest comprising:

a point cloud 3D scanner positioned at first location comprising an image frame generator configured to;

generate a 3D engagement zone having a height a width and a depth from a point cloud 3D scanner frame comprising an array of depth coordinates for respective two dimensional coordinates of a surface of the object of interest, within the three dimensional zone of interest, comprising a three dimensional coverage zone defined by the location and orientation of the point cloud 3D scanner and encompassing within the three dimensional coverage zone a three dimensional engagement zone, and wherein the point cloud 3D scanner frame comprises a three dimensional depth map of at least part of a surface of the object of interest within the three dimensional coverage zone or within both the three dimensional coverage zone and the three dimensional engagement zone;

a memory configured to store temporally separated point cloud 3D scanner image frames;

and

a computing device configured to;

divide the 3D engagement zone into one or more two-dimensional sub-zones by use of a planogram;

correlate a sub-zone location to a sub-zone height and a sub-zone width inside the two-dimensional sub-zone of the 3D engagement zone;

determine a location of the second three-dimensional object of interest within the two-dimensional sub-zone;

compare respective temporally separated point cloud 3D scanner image frames and-determine a time and a location of a collision between a surface of the first three-dimensional object of interest and a surface of the second three-dimensional object of interest within the two-dimensional sub-zone, and a surface of at least one of the three dimensional coverage zone or the three dimensional engagement zone encompassed by the three dimensional coverage zone;

generate a two-dimensional image of the two-dimensional sub-zone with a marker positioned at one or more collision locations between the first three-dimensional object of interest and the second three-dimensional object of interest on the two-dimensional image;

and wherein the object of interest is a person and the surface of the object of interest is on a portion of a body of the person.

2. The system of claim 1, wherein the object of interest is a person and the surface of the object of interest is on a portion of the body of the person and the system is further configured to determine a sequence of actions.

3. The system of claim 2, comprising:

the computing device configured to;

utilize the time and location of the collision between the surface of the object of interest and the surface of the three dimensional sub zone encompassed by at least one of the three dimensional coverage zone or the three dimensional engagement zone to characterize the interaction by the object of interest with another object of interest within the three dimensional sub zone; and

further configured to analyze the sequence of actions to identify an interaction type.

4. The system of claim 1 further comprising:

the computing device configured to;

utilize the time and location of the collision between the surface of the object of interest and the at least one of the surface of the three dimensional coverage zone or the three dimensional engagement zone encompassed by the dimensional coverage zone to characterize the interaction by the first three-dimensional object of interest with another object of interest different than the second three-dimensional object of interest within the three dimensional coverage zone, the three dimensional engagement zone or both.

5. The system of claim 4 wherein the another object of interest comprises a tracked product in a product location zone within the three dimensional engagement zone.

6. The system of claim 4, wherein the characterization of the interaction with the another object of interest is indicative of an activity of the person being performed with the another object of interest.

7. The system of claim 1, wherein the another second three-dimensional object of interest comprises a tracked product in a product location zone within the three dimensional sub-zone.