Methods and systems for detecting and recognizing objects in a controlled wide area

• US 7,257,236 B2
• Filed: 03/31/2003
• Issued: 08/14/2007
• Est. Priority Date: 05/22/2002
• Status: Expired due to Fees

• Current Assignee: Bioscrypt, Inc. (Advent International Corporation)
• Original Assignee: A4VISION
• Inventors: Yukhin, Artiom, Klimov, Andrei
• Primary Examiner(s): Mancuso; Joseph
• Assistant Examiner(s): Akhavannik; Hadi
• Application Number:
  US10/404,407
• Publication Number:
  US 20030235335A1
• Time in Patent Office: 1,597 Days
• Field of Search: 382/154, 382/103, 348/14.08, 348135-161
• US Class Current: 382/103
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Methods and systems are described herein for high-speed observation and recognition of objects or persons within a controlled area using 3D image data and tomography, stereo-photogrammetry, range finding and/or structured illumination. A wide area security system may comprise multiple pairs of 3D sensors surrounding a controlled area. Using at least two pairs of 3D sensors and a first 3D data collection technique such as tomography, a zone of interest within the controlled area is detected. Using at least one pair of 3D sensors and a second 3D data collection technique such as structured illumination, more 3D image data related to an object of interest is collected. The 3D image data may be compared to data representing known objects and used to identify an object of interest. Methods for fast processing of 3D data and recognition of objects of interest are also described.

Claims

1. A method of identifying an object of interest in a controlled area the method comprising:

collecting first 3D image data representing the controlled area using a first 3D sensor, wherein the first 3D sensor independently acquires first 3D image data from a single viewing angle within the controlled area, at a first time;

detecting a zone of interest within the controlled area based on the first 3D image data;

collecting second image data representing the detected zone of interest utilizing a second 3D sensor, wherein the second 3D sensor independently acquires second 3D image data from a single viewing angle within the controlled area, at a second time; and

identifying an object of interest in the controlled area based on the second 3D image data.

2. The method of claim 1, wherein identifying an object of interest in the controlled area based on the second 3D image data further includes:

detecting the object of interest within the detected zone of interest based on the second 3D image data;

collecting third 3D image data representing the detected object of interest utilizing at least a third 3D sensor; and

comparing the third 3D image data to data representing known objects.

3. The method of claim 1 wherein collecting first 3D image data representing the controlled area includes utilizing tomography.

4. The method of claim 3, wherein collecting first 3D image data representing the controlled area further includes utilizing at least two pairs of 3D sensors to collect image data.

5. The method of claim 1 wherein collecting second 3D image data of the zone of interest includes utilizing stereo-photogrammetry.

6. The method of claim 5, wherein collecting second 3D image data of the zone of interest further includes utilizing at least one pair of 3D sensors to collect image data.

7. The method of claim 2, wherein collecting third 3D image data of the object of interest includes utilizing structured illumination.

8. The method of claim 7, wherein collecting third 3D image data of the object of interest further includes determining the coordinates of the object of interest based on the second 3D image data.

9. The method of claim 8, wherein determining the coordinates of the object of interest based on the second 3D image data includes utilizing triangulation.

10. The method of claim 8, wherein determining the coordinates of the object of interest based on the second 3D image data includes determining a distance between the object of interest and at least one 3D sensor.

11. The method of claim 2, wherein identifying the object of interest based on the third 3D image data includes:

generating a feature vector based on the third 3D image data, the feature vector representing features of the detected object of interest; and

comparing the feature vector to data representing features of known objects.

13. The method of claim 1, wherein collecting first 3D image data representing the controlled area and collecting second 3D image data of the zone of interest occur either simultaneously or in succession.

14. The method of claim 2, further comprising checking the quality of some or all of the first, second, or third 3D image data.

15. The method of claim 14, wherein checking the quality of some or all of the first, second, or third 3D image data comprises utilizing a rejection algorithm.

16. The method of claim 15, wherein the rejection algorithm includes at least one of the steps of:

determining if the first, second, or third 3D image data is good; and

determining if the first, second, or third 3D image data comprises enough data for recognition.

24. The method of claim 1, wherein the first and second 3D sensors are the same 3D sensor.

25. The method of claim 1, wherein the first and second 3D sensors are different 3D sensors.

26. The method of claim 2, wherein the third 3D sensor is different from the first and second 3D sensor.

27. The method of claim 2, wherein the first, second, and third 3D sensors are all the same sensor.

12. A method of identifying an object of interest in a controlled area, the method comprising:

collecting first 3D image data representing the controlled area;

detecting a zone of interest within the controlled area based on the first 3D image data;

collecting second image data representing the detected zone of interest;

detecting an object of interest within the detected zone of interest based on the second 3D image data;

collecting third 3D image data representing the detected object of interest;

comparing the third 3D image data to data representing known objects; and

identifying an object of interest in the controlled area based on the third 3D image data, wherein identifying the object of interest based on the third 3D image data includes;

generating a feature vector based on the second 3D image data, the feature vector representing features of the object; and

constructing a filter vector based on the feature vector;

determining a subset of data representing known objects based on the filter vector; and

comparing the second 3D image data to the subset of data representing features of known objects.

17. A system for identifying an object in a controlled area surrounded

by at least two pairs of 3D sensors, the system comprising:

at least two pairs of 3D sensors, each pair comprising a first and a second 3D sensor, each first 3D sensor in a pair of 3D sensors is situated directly across the controlled area from its corresponding second 3D sensor, wherein the first and second 3D sensor each independently acquire 3D image data from a single viewing angle within the controlled area, each 3D sensor comprising;

an illumination unit capable of illuminating the controlled area and the object;

a registration unit comprising;

a detecting unit capable of detecting light reflected by at least one of the controlled area and the object, and

an analog-to-digital converter for converting the detected light into digital data; and

a processing unit capable of generating 3D image data based on the converted data.

18. The system of claim 17, wherein the processing unit identifies the object based on the generated 3D image data.

19. The system of claim 17, wherein at least two 3D sensors of the at least two pairs of 3D sensors are operatively connected for operating in parallel.

20. The system of claim 17, wherein one or more 3D sensors of the at least two pairs of 3D sensors further comprise a memory unit for storing 3D image data.

21. A system for identifying an object of interest in a controlled area, the system comprising:

a processing unit adapted to perform steps of;

obtaining first 3D image data representing the controlled area from a first 3D sensor, wherein the first 3D sensor independently acquires first 3D image data from a single viewing angle within the controlled area, at a first time;

detecting a zone of interest within the controlled area based on the first 3D image data;

obtaining second image data representing the detected zone of interest, from a second 3D sensor wherein the second 3D sensor independently acquires second 3D image data from a single viewing within the controlled area, at a second time;

detecting an object of interest within the detected zone of interest based on the second 3D image data;

obtaining third 3D image data representing the detected object of interest;

comparing the third 3D image data to data representing known objects;

and identifying an object of interest in the controlled area based on the third 3D image data; and

a memory unit capable of storing 3D image data.

28. The method of claim 21, wherein the first and second 3D sensors are the same 3D sensor.

29. The method of claim 21, wherein the first and second 3D sensors are different 3D sensors.

22. A computer readable medium containing instructions for

controlling a computer system to perform a method of identifying an object of interest in a controlled area the method comprising:

collecting first 3D image data representing the controlled area using a first 3D sensor, wherein the first 3D sensor independently acquires first 3D image data from a single viewing angle within the controlled area, at a first time;

detecting a zone of interest within the controlled area based on the first 3D image data;

collecting second image data representing the detected zone of interest utilizing a second 3D sensor, wherein the second 3D sensor independently acquires second 3D image data from a single viewing angle within the controlled area, at a second time; and

identifying an object of interest in the controlled area based on the second 3D image data.

30. The method of claim 22, wherein the first and second 3D sensors are the same 3D sensor.

31. The method of claim 22, wherein the first and second 3D sensors are different 3D sensors.

23. A computer readable medium containing instructions for controlling a computer system to perform a method of identifying an object of interest in a controlled area, the method comprising:

collecting first 3D image data representing the controlled area using a first 3D sensor, wherein the first 3D sensor independently acquires first 3D image data from a single viewing angle within the controlled area, at a first time;

detecting a zone of interest within the controlled area based on the first 3D image data;

collecting second image data representing the detected zone of interest using a second 3D sensor, wherein the second 3D sensor independently acquires second 3D image data from a single viewing angle within the controlled area, at a second time;

detecting an object of interest within the detected zone of interest based on the second 3D image data;

collecting third 3D image data representing the detected object of interest;

comparing the third 3D image data to data representing known objects; and

identifying an object of interest in the controlled area based on the third 3D image data.