Methods for detecting, identifying and displaying object information with a multi-camera vision system

• US 10,331,960 B2
• Filed: 05/10/2017
• Issued: 06/25/2019
• Est. Priority Date: 05/10/2017
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia, Inc.)
• Original Assignee: FotoNation Limited (Adeia, Inc.)
• Inventors: Corcoran, Peter
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US15/591,434
• Publication Number:
  US 20180330171A1
• Time in Patent Office: 776 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Methods for detecting, identifying and displaying object information with a multi-camera vision system. In one embodiment presence of an object is detected in a region extending above a surface and positioned beyond a border, the region including at least first and second zones. A frame of image data is acquired which spans a field of view angle corresponding to a portion of a region from a position along the border. Object detection criteria are applied to analyze and classify an image of an object among a set of object types. A determination is made whether the object is present in the first zone based on application of a first series of scan windows varying in size over a first range of window sizes relative to the size of the image frame, this only permitting detection of images limited in size to a first range of image sizes.

Claims

1. A method of identifying an object of interest in a zone of a region about a moving vehicle based on sizes of object images, comprising:

acquiring a scene image in a different field of view for each in a series of at least two zones of the region about the structure where a first zone is relatively close to the structure and a second zone, separate and distinct from the first zone, extends farther away from the structure than the first zone, wherein;

a first of the scene images is acquired with an image acquisition depth of field setting that sufficiently resolves an image of a first object positioned in the first zone for identification as a first object type with template matching criteria, while an image of a second object of the first object type positioned in the second zone cannot be sufficiently resolved in the first of the scene images for identification as a first object type with the template matching criteria; and

a second of the scene images is acquired with an image acquisition depth of field setting that sufficiently resolves an image of a second object positioned in the second zone for identification as a first object type with the template matching criteria, while an image of the first object of the first object type positioned in the first zone cannot be sufficiently resolved in the second of the scene images for identification as a first object type with the template matching criteria.

2. The method of claim 1 wherein the first scene image subtends a first field of view angle and the second scene image subtends a second field of view angle, the method including:

applying a first additional criterion requiring, in order for the first object to be identified as an object of interest in the first zone, that the image of the first object be within a predetermined first range of sizes relative to the given size of the frame of image data based on the first field of view angle, said first additional criterion enabling identification of images of objects positioned within the first zone.

3. The method of claim 1 further including:

applying a second criterion to identify an image of the second object when present in a second of the acquired scene images as the first object type, while the second object is positioned in the second zone and wherein the second scene image subtends a second field of view angle,

the second criterion requiring, in order for the second object to be identified as an object of interest in the second zone, that the image of the second object in the second of the acquired scene images be within a predetermined second range of sizes relative to the given size of the frame of image data based on the second field of view angle, said second criterion enabling identification of the image of the second object in the second scene image when the second object is positioned within the second zone based, in part, on size of the image of the second object in the second scene relative to the given size of the frame of image data based on the second field of view angle.

18. The method of claim 1, wherein the first scene images are captured with an imaging device having a first focal length, and the second scene images are captured with another imaging device having a second focal length, the second focal length being different from the first focal length.

19. The method of claim 1 wherein at least one of the first focal length or the second focal length is a fixed focal length.

4. A method of detecting the presence of an object in a region extending above a surface and positioned beyond a border, the region including at least first and second zones, each zone having a different spacing from the border, the method including:

acquiring a frame of image data, spanning a field of view angle corresponding to a portion of the region, from a position along the border;

applying object detection criteria to analyze an image of an object in the frame of data, including application of a template matching process to classify the object among a set of object types,

wherein a determination is made whether the object is present in the first zone based on an effective application of a first series of scan windows, effectively varying in size over a first range of window sizes relative to the size of the image frame which application of the first range of window sizes only permits detection of images limited in size to a first range of image sizes; and

wherein a determination is made whether the object is present in the second zone based on an effective application of a second series of scan windows effectively varying in size over a second range of window sizes relative to the size of the image frame which application of the second range of window sizes only permits detection of images limited in size to a second range of image sizes.

5. The method of claim 4 where the first range of scan window sizes and the second range of scan window sizes are each smaller in range than the range of a third possible series of scan window sizes which would permit detection of images varying in size over a third range of image sizes greater than each of the first range of image sizes and the second range of image sizes.

6. The method of claim 4 where the first range of scan window sizes and the second range of scan window sizes are identical.

7. The method of claim 4 where the frame of image data has a predetermined number of pixels and the effective application of the first series of scan windows that effectively varies in size over the first range of window sizes is had by applying one scan window of a fixed size relative to the frame of image data having the predetermined number of pixels by down-sampling the number of pixels to provide smaller frames of image data relative to the one scan window of the fixed size.

8. The method of claim 7 where the effective application of the second series of scan windows that effectively varies in size over the second range of window sizes is also had by applying one scan window of a fixed size relative to the frame of image data having the predetermined number of pixels by down-sampling the number of pixels to provide smaller frames of image data relative to the one scan window of the fixed size.

9. The method of claim 4 where the object is classified as an object type with parallel classifier cascades based on one or more of a Haar measure, local binary pattern features, a histogram of oriented gradients, and a census transform, and the object types include vehicles and pedestrians.

10. The method of claim 4 where the border is a peripheral surface along a vehicle.

11. The method of claim 4 where the frame of image data is acquired with an imaging system providing the field of view angle along an optical axis and the position of the object is determined based on position of the image of the object in the field of view relative to the optical axis and relative size of the image of the object to the frame of image data.

20. The method of claim 4, wherein:

the first series of scan windows are captured using an imaging device having a first focal length, and the second series of scan windows are captured using another imaging device having a second focal length, the second focal length being different from the first focal length; and

the first imaging device or the second imaging device provides programmably adjustable focal distances or a fixed focal length.

12. A method of displaying position information of objects, comprising:

classifying different identified objects with different image acquisition devices, each image acquisition device including a camera that captures an image of an object in a field of view having a field of view angle relative to an optical axis, and a processing unit performing the classification according to object type;

for each in a plurality of objects detected with at least one image acquisition device, processing acquired optical information to determine position information for the object, relative to said at least one image acquisition device with which the object is detected, including (i) distance between the object and said one image acquisition device based on approximate size of the object image and (ii) angle of object image position in the field of view of said one image acquisition device;

for each detected object, transforming the position information of the object as determined with said at least one image acquisition device, and relative to the position of the said one image acquisition device, to revised position information based on a common reference position so that the position of each in the plurality of detected objects is defined relative to the common reference position; and

transmitting the revised position information from each detected object from the plurality of image acquisition devices so as to simultaneously display the position of each detected object on a display screen.

13. The method of claim 12 including transforming the position information of each in the plurality of detected objects to the revised position information with computations performed by the image acquisition device which processes the acquired optical information.

14. The method of claim 12 including, prior to displaying the position of each detected object, transmitting the revised reference information acquired for each in the plurality of detected objects to a central control unit.

15. The method of claim 12 where the position of each detected object based on the common reference position is presented by display of a selected icon, selection of each icon based on application of a template matching engine to determine each object type.

16. The method of claim 12, wherein each of the plurality of image acquisition devices transmit the revised position information for the objects detected by the respective image acquisition device to a central control unit.

17. The method of claim 12, wherein each of the plurality of image acquisition devices determines a classification the object detected by the respective image acquisition device, and

wherein the image acquisition device transmits the classification of the object to the central control unit.