System and method for tracking and annotating multiple objects in a 3D model

• US 10,049,462 B2
• Filed: 02/24/2017
• Issued: 08/14/2018
• Est. Priority Date: 03/23/2016
• Status: Active Grant

• Current Assignee: Akcelita LLC
• Original Assignee: Akcelita LLC
• Inventors: Esparra, Rafael, Negron, Edwin, Yamin, Rene
• Primary Examiner(s): Bhatnagar, Anand
• Application Number:
  US15/441,736
• Publication Number:
  US 20170278256A1
• Time in Patent Office: 536 Days
• CPC Class Codes: G01F 17/00 Methods or apparatus for determining the capacity of containers or cavities, or the volume of solid bodies measuring linear dimensions to determine volume G01B
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 2210/32 Image data format
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/55 from multiple images

Abstract

A system and method for tracking and annotating objects in a 3D model is disclosed. The method includes receiving a signal feed from a group of sensors including at least one depth sensor, determining a reference signal feed from the signal feed, determining a volumetric representation of at least one object not present in the reference signal feed, assigning a unique volumetric profile to at least one object, and/or storing the unique volumetric profile in an object database. The signal feed may include at least one 3D area of interest. Also, the unique volumetric profile may include a unique volumetric id obtained from a group of attributes. The unique volumetric profile may include at least a unique volumetric position and a unique volumetric centroid. The group of sensors may further include video cameras, thermal, and chemical sensors.

Claims

1. A computing system comprising:

a plurality of capture servers, each of the plurality of capture servers located in a distinct physical site, comprising a processor and a memory storing instructions that, when executed by the processor, configure the servers to execute the steps of;

receiving a signal feed from a plurality of sensors, said signal feed comprising at least one area of interest, said plurality of sensors comprising at least one depth sensor,

determining a reference signal feed from the signal feed,

determining a volumetric representation of at least one object not part of the reference signal feed,

assigning a unique volumetric profile to the at least one object,

storing the unique volumetric profile in an object database, and

sending the unique volumetric profile to at least one analytics server,

wherein the unique volumetric profile comprises a unique volumetric id obtained from a plurality of volumetric attributes comprising at least a unique volumetric position and a unique volumetric centroid;

the analytics server comprising a processor and a memory storing instructions that, when executed by the processor, configure the apparatus to a processor execute the steps of;

receiving the unique volumetric profile from the plurality of capture servers,

wherein the analytics server processes an aggregated data based on the received unique volumetric profile; and

a web server comprising a processor and a memory storing instructions that, when executed by the processor, configure the apparatus to a processor execute the steps of;

providing access to a user to the aggregated data.

2. The computing system of claim 1, wherein the unique volumetric attributes further comprise a total volume, height, and weight.

3. The computing system of claim 1, wherein the plurality of sensors further comprises at least one video sensor and the unique volumetric attributes further comprise color.

4. The computing system of claim 1, wherein the plurality of sensors further comprises at least one thermal sensor and the unique volumetric attributes further comprise temperature.

5. The computing system of claim 1, wherein the aggregated data comprises number of customers in a queue, time spent in the queue, and productivity of employees in a customer service area.

6. A method comprising:

receiving, by a capture server, a signal feed from a plurality of sensors, said signal feed comprising at least one area of interest, said plurality of sensors comprising at least one depth sensor,

determining a reference signal feed from the signal feed,

determining a volumetric representation of at least one object not part of the reference signal feed,

assigning a unique volumetric profile to the at least one object,

storing the unique volumetric profile in an object database,

wherein the unique volumetric profile comprises a unique volumetric id obtained from a plurality of volumetric attributes comprising at least a unique volumetric position and a unique volumetric centroid.

7. The method of claim 6, further comprising processing an aggregated data based on a plurality of unique volumetric profiles stored in the object database.

8. The method of claim 6, wherein the unique volumetric attributes further comprise a total volume, height, and weight.

9. The method of claim 6, wherein the plurality of sensors further comprises at least one video sensor and the unique volumetric attributes further comprise color.

10. The method of claim 6, wherein the plurality of sensors further comprises at least one thermal sensor and the unique volumetric attributes further comprise temperature.

11. The method of claim 7, wherein the aggregated data comprises number of customers in a queue, time spent in the queue, and productivity of employees in a customer service area.

12. A computing apparatus, the computing apparatus comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the apparatus to a processor execute the steps of;

receive a signal feed from a volumetric sensor,

calculate a reference signal feed from the signal feed,

define a queue entry point, a queue exit point and a queue path in the reference signal feed,

detect the presence of an object in the reference signal feed,

assign a unique volumetric profile to the object,

said unique volumetric profile comprising a unique volumetric id and a unique volumetric position in the queue path,

store the unique volumetric profile in an object database, and

retrieve the unique volumetric profile from the object database,

wherein the unique volumetric position in the queue path is determined with respect to the queue entry point and the queue exit point.

13. The computing apparatus of claim 12, wherein the unique volumetric profile further comprises a total volume, height, and weight.

14. The computing apparatus of claim 12, wherein the plurality of sensors further comprises at least one video sensor and the unique volumetric profile further comprises color.

15. The computing apparatus of claim 12, wherein the plurality of sensors further comprises at least one thermal sensor and the unique volumetric profile further comprises temperature.

16. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to a processor execute the steps of:

receive a signal feed from a volumetric sensor,

calculate a reference signal feed from the signal feed,

define a queue entry point, a queue exit point and a queue path in the reference signal feed,

detect the presence of an object in the reference signal feed,

assign a unique volumetric profile to the object,

said unique volumetric profile comprising a unique volumetric id and a unique volumetric position in the queue path,

store the unique volumetric profile in an object database, and

retrieve the unique volumetric profile from the object database,

wherein the unique volumetric position in the queue path is determined with respect to the queue entry point and the queue exit point.

17. The non-transitory computer-readable storage medium of claim 16, wherein the unique volumetric profile further comprises a total volume, height, and weight.

18. The non-transitory computer-readable storage medium of claim 16, wherein the plurality of sensors further comprises at least one video sensor and the unique volumetric profile further comprises color.

19. The non-transitory computer-readable storage medium of claim 16, wherein the plurality of sensors further comprises at least one thermal sensor and the unique volumetric profile further comprises temperature.