METHOD AND SYSTEM FOR CLASSIFYING OBJECTS IN A PERCEPTION SCENE GRAPH BY USING A SCENE-DETECTION-SCHEMA

• US 20180341822A1
• Filed: 05/26/2017
• Published: 11/29/2018
• Est. Priority Date: 05/26/2017
• Status: Abandoned Application

• Current Assignee: New Eagle LLC (Woodward Incorporated)
• Original Assignee: Dura Operating LLC (Dura Automotive Systems)
• Inventors: Hovis, Jeffrey Gene, Szczerba, Michael Bernhard
• Application Number:
  US15/607,070
• Publication Number:
  US 20180341822A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

A method and system is provided for using a scene detection schema for classifying objects in a perception scene graph for a motor vehicle. The scene detection schema includes collecting sensor information about an area surrounding the vehicle, including capturing an image of the area; processing the sensor information to generate a perception scene graph (PSG); detecting a plurality of objects by analyzing the captured image; comparing the detected objects with reference objects; classifying the detected objects based on matching reference objects; and assigning a priority to each of the classified objects. The steps of detecting and classifying objects having higher priorities are reiterated with a greater frequency than objects having lower priorities. An object is assigned a higher priority level if the object is in a focus region of the PSG. The fidelity of the focus region is increased by updating the classification of the objects after each reiteration.

Claims

1. A method of classifying objects for a perception scene graph, comprising the steps of:

collecting sensor information about an area adjacent a motor vehicle;

processing the sensor information to detect a plurality of objects and to generate a perception scene graph (PSG) comprising a virtual 3-dimensional model of the area adjacent the motor vehicle, wherein the PSG includes the detected plurality of objects;

assigning a classification level (n) to each of the detected plurality of objects;

comparing each of the detected plurality of objects with reference objects in a scene detection schema (SDS) tree having a plurality of classification levels; and

classifying each of the detected plurality of objects in the PSG based on the classification level in the SDS tree.

2. The method of claim 1, wherein:

the step of collecting sensor information about an area adjacent a motor vehicle, includes capturing an image by an image capturing device,

the step of processing the sensor information to detect a plurality of objects, includes analyzing the captured image to detect the plurality of objects, and

the step of comparing each of the detected plurality of objects with reference objects, includes comparing the captured image with reference images of the objects.

3. The method of claim 2 further comprising the step of assigning a priority level to each of the detected plurality of objects based on a predetermined importance of each of the plurality of object.

4. The method of claim 3 further comprising the step of increasing the fidelity of the PSG by classifying each of the detected objects based on the respective assigned priority levels and classification levels (n) for each object.

5. The method of claim 4 further comprising the steps of:

determining the number of instances (n) each of the plurality of objects have been previously classified;

assigning a classification level of n+1 for each of the plurality of objects based on the number instances (n) each of the plurality of objects have been previously classified; and

comparing each of the plurality of objects with reference objects within the respective n+1 classification level of each object in the SDS tree.

6. The method of claim 5, wherein the priority level of an object classified as an animate object is higher than the priority level of an object classified as an inanimate object.

7. The method of claim 6, wherein the priority level of an animate object of a pedestrian is higher than an animate object of a motor vehicle.

8. The method of claim 4, further comprising the steps of:

identifying a focus region in the virtual 3-D model of the area adjacent the motor vehicle; and

assigning a higher priority level to a plurality of objects detected in the focus region as compared to objects detected outside the focus region.

9. A method of a scene detection schema for classifying objects, comprising the steps of:

collecting sensor information about an area adjacent a motor vehicle, including capturing an image, by an image capturing device;

processing the sensor information to generate a perception scene graph (PSG) comprising a virtual 3-dimensional model of the area adjacent the motor vehicle;

detecting a plurality of objects in the virtual 3-dimensional model by analyzing the collected sensor information, including analyzing the captured image to detect the plurality of objects;

comparing each of the detected plurality of objects with reference objects;

classifying each of the detected plurality of objects based on matching reference objects;

assigning a priority level to each of the classified objects; and

reclassifying selected objects using a SDS tree based on the respective assigned priority levels for each object.

10. The method of claim 9, wherein the SDS tree includes a plurality of classification levels, and the method further comprising the steps of obtaining a higher classification level in the SDS tree for objects having a higher priority level than an object with a lower priority level.

11. The method of claim 10, further comprising the steps of:

increasing the fidelity of the PSG by reclassifying the objects by repeated iterations through the SDS tree.

12. The method of claim 11, further comprising the steps of assigning an object with a higher priority level if the object is in the path of the vehicle than an object that is not within the path of the vehicle.

13. The method of claim 12, further comprising the steps of assigning an object with a higher priority level if the object is in a defined focus region than an object that is not within a defined focus region.

14. The method of claim 13, wherein objects having a higher priority level corresponds with a higher classification level, and wherein the higher classification level includes greater details of the object.

15. A system for using a scene detection schema for classifying objects in a perception scene graph (PSG) in a motor vehicle, comprising:

at least one external sensor, including an image capturing device, having an effective sensor range configured to gather information on a surrounding of the motor vehicle; and

at least one perception controller in communication with the at least one external sensor, wherein the at least one perception controller is configured to generate the PSG comprising a virtual model of the surroundings of the motor vehicle based on the gathered information from the external sensor;

wherein the at least one perception controller is further configured to detect objects based on gathered information, compare detected objects with reference objects, and classify detected object based on matching reference objects in a scene detection schema (SDS) tree having a plurality of classification levels.

16. The system of claim 15, wherein the at least one perception controller is further configured to increase the fidelity of the virtual model by assigning a priority level to each of the detected objects based on a predetermined importance of each object.

17. The system of claim 16, wherein the at least one perception controller is further configured to reclassify higher priority objects with a greater frequency than lower priority objects.

18. The system of claim 17, wherein the at least one perception controller is further configured to determine the number of instances (n) each of the objects have been previously classified from information extracted from the PSG.

19. The system of claim 18, wherein the at least one perception controller is further configured to assign a level of classification of n+1 for each of the plurality of objects based on the number instances (n) each of the plurality of objects have been previously classified.

20. The system of claim 19, wherein the at least one perception controller is further configured to compare each of the plurality of objects with a respective n+1 level reference objects for classification of each of the plurality of objects.