Computer-assisted or autonomous driving with region-of-interest determination for traffic light analysis

• US 10,139,832 B2
• Filed: 01/26/2017
• Issued: 11/27/2018
• Est. Priority Date: 01/26/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Sarkar, Nandini, Leow, Teng Jun Nicholas
• Primary Examiner(s): Hilgendorf, Dale W
• Application Number:
  US15/416,893
• Publication Number:
  US 20180211530A1
• Time in Patent Office: 670 Days
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G08G 1/09623 Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence

Abstract

Apparatuses, methods and storage medium associated with determining a ROI for traffic light analysis for CA/AD are disclosed herein. In embodiments, a system may include a region-of-interest (ROI) for traffic light analysis component to receive a current pose of the ego vehicle, and to apply the current pose of the ego vehicle to a plurality of traffic light poses to identify a ROI for a determination of a current state of a traffic light to be taken into consideration in provision of assistance to, or autonomous operation of the ego vehicle; and a traffic light analysis component to determine the current state of the traffic light within the ROI. Other embodiments may be disclosed or claimed.

Claims

1. A system for computer assisted or autonomous driving of an ego vehicle, comprising:

a region-of-interest for traffic light analysis (ROIFTLA) identification component to receive a current pose of the ego vehicle, and to apply the current pose of the ego vehicle to a plurality of traffic light poses to identify a region-of-interest (ROI) in an image of a frontal view of the ego vehicle, for a determination of a current state of a traffic light to be taken into consideration in provision of assistance to, or autonomous operation of, the ego vehicle, the traffic light being within the ROI in the image, and is one of a plurality of traffic lights in the frontal view of the ego vehicle, wherein each traffic light pose comprises coordinates and orientation relative to an origin of a traffic light coordinate system that is different from a map'"'"'s coordinate system employed by the system to provide assistance to, or autonomously operate, the ego vehicle to traverse a route, and wherein the current pose of the ego vehicle comprises coordinates and yaw relative to an origin of the map'"'"'s coordinate system, employed by the system to provide assistance to, or autonomously operate the ego vehicle to traverse a route; and

a traffic light analysis component coupled to the ROIFTLA identification component to determine the current state of the traffic light within the ROI in the image, and output the current state of the traffic light within the ROI in the image for use to provide assistance to, or autonomously operate, the ego vehicle, wherein to apply the current pose of the ego vehicle to the plurality of traffic light poses to identify the ROI in the image of a horizon for the determination of the current state of the traffic light within the ROI in the image to be taken into consideration, the ROIFTLA identification component is to transform the coordinates and yaw of the current pose of the ego vehicle from relative to the origin of the map'"'"'s coordinate system to relative to the origin of the traffic light coordinate system, and apply a distance and an orientation constraint to the transformed coordinates and yaw of the current pose of the ego vehicle and the coordinates and orientations of the plurality of traffic light poses to identify the ROI in the image for the determination of the current state of the traffic light within the ROI in the image to be taken into consideration.

2. The system of claim 1, wherein to determine the current state of the traffic light within the identified ROI in the images, the traffic light analysis component is to apply shape, contour or color analysis to a region of an image that corresponds to the identified ROI in the image.

3. The system of claim 2, further comprising one or more sensors including a camera to capture the image.

4. The system of claim 1, further comprising a localization component coupled to the ROIFTLA identification component to periodically determine in real time the current pose of the ego vehicle, and provide the current pose of the ego vehicle to the ROIFTLA identification component.

5. The system of claim 1, further comprising a non-transitory computer readable storage medium coupled to the ROIFTLA identification component to store the plurality of traffic light poses.

6. The system of claim 1, wherein the system is disposed in the ego vehicle.

7. A method for computerized assist or autonomous driving of an ego vehicle, comprising:

receiving, by a computerized assist or autonomous driving (CA/AD) system of the ego vehicle, a current pose of the ego vehicle, wherein the current pose of the ego vehicle comprises coordinates and yaw relative to an origin of a map'"'"'s coordinate system, employed by the system to provide assistance to, or autonomously operate the ego vehicle to traverse a route;

applying, by the CA/AD system, the current pose of the ego vehicle to a plurality of traffic light poses to identify a region of interest (ROI) in an image of a front view of the ego vehicle as seen through a frontal view of the ego vehicle for determining a current state of a traffic light within the ROI in the image to be taken into consideration in providing assistance to, or autonomously operating the ego vehicle, the traffic light being one of a plurality of traffic lights in the front view of the ego vehicle, wherein each traffic light pose comprises coordinates and orientation relative to an origin of a traffic light coordinate system that is different from the map'"'"'s coordinate system employed by the system to provide assistance to, or autonomously operate the ego vehicle to traverse a route, wherein to apply the current pose of the ego vehicle to the plurality of traffic light poses to identify the ROI in the image of a horizon for the determination of the current state of the traffic light within the ROI in the image of the horizon to be taken into consideration, an ROIFTLA identification component is to transform the coordinates and yaw of the current pose of the ego vehicle from relative to the origin of the map'"'"'s coordinate system to relative to the origin of the traffic light coordinate system, and apply a distance and an orientation constraint to the transformed coordinates and yaw of the current pose of the ego vehicle and the coordinates and orientations of the plurality of traffic light poses to identify the ROI in the horizon for the determination of the current state of the traffic light within the ROI in the horizon to be taken into consideration;

determining, by the CA/AD system, the current state of the traffic light within the ROI in the image; and

providing, by the CA/AD system, assistance to, or autonomously operating the ego vehicle, based at least in part on the determined current state of the traffic light within the ROI in the image.

8. The method of claim 7, wherein determining the current state of the traffic light within the identified ROI in the image comprises applying shape, contour or color analysis to a region of an image that corresponds to the identified ROI in image.

9. The method of claim 7 further comprising periodically determining, by the CA/AD system, in real time the current pose of the ego vehicle.

10. The method of claim 7 further comprising receiving and storing the plurality of traffic light poses by the CA/AD system.

11. At least one non-transitory computer readable media (CRM) comprising a plurality of instructions configured to cause a computerized assist or autonomous driving (CA/AD) system of an ego vehicle, in response to execution of the instructions by CA/AD system, to:

receive a current pose of the ego vehicle, wherein the current pose of the ego vehicle comprises coordinates and yaw relative to an origin of a map'"'"'s coordinate system, employed by the system to provide assistance to, or autonomously operate the ego vehicle to traverse a route;

apply the current pose of the ego vehicle to a plurality of traffic light poses to identify a region of interest (ROI) in a image in front of the ego vehicle as seen through a frontal view of the ego vehicle for a determination of a current state of a traffic light within the ROI in the image to be taken into consideration in provision of assistance to, or autonomous operation of the ego vehicle, wherein each traffic light pose comprises coordinates and orientation relative to an origin of a traffic light coordinate system that is different from the map'"'"'s coordinate system employed by the system to provide assistance to, or autonomously operate, the ego vehicle to traverse a route, wherein to apply the current pose of the ego vehicle to the plurality of traffic light poses to identify the ROI in the image of a horizon for the determination of the current state of the traffic light within the ROI in the image of the horizon to be taken into consideration, an ROIFTLA identification component is to transform the coordinates and yaw of the current pose of the ego vehicle from relative to the origin of the map'"'"'s coordinate system to relative to the origin of the traffic light coordinate system, and apply a distance and an orientation constraint to the transformed coordinates and yaw of the current pose of the ego vehicle and the coordinates and orientations of the plurality of traffic light poses to identify the ROI in the horizon for the determination of the current state of the traffic light within the ROI in the horizon to be taken into consideration;

determine the current state of the traffic light within the ROI in the image; and

output the current state of the traffic light within the ROI in the image for use for provision of assistance to, or autonomous operation of, the ego vehicle.

12. The non-transitory CRM of claim 11, wherein to determine the current state of the traffic light within the identified ROI comprises to apply shape, contour or color analysis to a region of an image that corresponds to the identified ROI in the image.

13. The non-transitory CRM of claim 11, wherein the CA/AD system is further caused to periodically determine in real time the current pose of the ego vehicle, and provide the current pose of the ego vehicle to the ROIFTLA identification component.

14. The non-transitory CRM of claim 11, wherein the CA/AD system is further caused to receive and store the plurality of traffic light poses.

15. The non-transitory CRM claim 11, wherein the CA/AD system is disposed in the ego vehicle.