Early detection of turning condition identification using perception technology

• US 9,959,624 B2
• Filed: 01/15/2016
• Issued: 05/01/2018
• Est. Priority Date: 12/22/2014
• Status: Active Grant

• Current Assignee: Audi AG (Volkswagen AG), Volkswagen AG
• Original Assignee: Audi AG (Volkswagen AG), Volkswagen AG
• Inventors: Raghu, Kaushik, Beatrix, Schmidts
• Primary Examiner(s): ADAMS, EILEEN M
• Application Number:
  US14/996,628
• Publication Number:
  US 20160176358A1
• Time in Patent Office: 837 Days
• Field of Search: None
• CPC Class Codes: B60R 2300/308 by overlaying the real scene, e.g. through a head-up display on the windscreen
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An in-vehicle system for identifying turn-only and shared turn lanes and identifying turn rules according to visual indicia captured from turn signs. An image apparatus configured to obtain image data including graphic data and/or text data from a roadway sign, as well as street light image data for an intersection on the roadway. The image data may be processed to detect a graphic image and/or text from the image data relating to the roadway sign. Street light image data may also be captured and processed to determine a street light status. An identification module may determine rules for turning on the intersection of the roadway based on the processed image data relating to the roadway sign, wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the determined street light status.

Claims

1. An in-vehicle system for determining rules for a turn on a roadway, the system comprising:

a first image apparatus for obtaining lane marking image data from the roadway;

a lane marking detection module for detecting lane markings on the roadway from the lane marking image data;

a second image apparatus for obtaining at least one of image data and text data from a roadway sign;

a sign analyzer module for detecting at least one of a graphic image and text from the at least one of image data and text data respectively;

means for detecting lane markings as one of standard frequency dashed lane markings, high frequency dashed lane markings, and solid lane markings and for identifying a turn-only lane and a shared turn lane in response to the detected lane markings;

and means for determining rules for turning for an intersection of the roadway based on the at least one of the graphic image and recognition and interpretation of the detected text.

2. The in-vehicle system of claim 1, wherein the second image apparatus is configured to obtain street light image data for the intersection.

3. The in-vehicle system of claim 2, wherein the means for determining rules for turning for an intersection is configured to compare the street light image data with the determined rules.

4. The in-vehicle system of claim 3, further comprising communications for transmitting a warning signal if the means for determining rules for turning for an intersection determines that the street light image data does not comply with the determined rules.

5. The in-vehicle system of claim 3, further comprising communications for transmitting a turn engagement signal if the means for determining rules for turning for an intersection determines that the street light image data complies with the determined rules.

6. An in-vehicle system for determining rules for a turn on a roadway, the system comprising:

an image apparatus configured to obtain image data comprising (i) at least one of graphic data and text data from a roadway sign, and (ii) street light image data for an intersection on the roadway;

an analyzer module configured to process the image data to detect at least one of a graphic image and text from the image data relating to the roadway sign, and to process the street light image data to determine a street light status; and

an identification module for determining rules for turning on the intersection of the roadway based on the processed image data relating to the roadway sign, wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the determined street light status, and

wherein the identification module is further configured to determine the rules for turning on the intersection based at least in part on recognition and interpretation of the detected text data.

7. The in-vehicle system of claim 6, wherein the analyzer module is configured to perform optical character recognition (OCR) from the image data relating to the roadway sign to detect text data.

8. The in-vehicle system of claim 7, wherein the identification module is configured to determine the rules for turning on the intersection based at least in part on the detected text data.

9. The in-vehicle system of claim 8, further comprising communications for transmitting a warning signal if the identification module determines that the determined street light status does not comply with the determined rules.

10. The in-vehicle system of claim 6, wherein the analyzer module is configured to detect graphic images from the graphic data.

11. The in-vehicle system of claim 9, wherein the identification module is configured to compare the detected graphic image to one or more stored graphic images to determine a matching graphic image, wherein the identification module is configured to determine the rules for turning on the intersection based at least in part on the matching graphic image.

12. The in-vehicle system of claim 11, further comprising communications for transmitting a warning signal if the identification module determines that the determined street light status does not comply with the determined rules.

13. The in-vehicle system of claim 6, further comprising:

a second image apparatus for obtaining lane marking image data from the roadway; and

a lane marking detection module for detecting lane markings on the roadway from the lane marking image data,

wherein the lane marking detection module is configured to detect lane markings as one of standard frequency dashed lane markings, high frequency dashed lane markings, and solid lane markings and for identifying a turn-only lane and a shared turn lane in response to the detected lane markings.

14. The in-vehicle system of claim 13, wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the detected lane marking.

15. A method for determining rules for a turn on a roadway via an in-vehicle system, comprising:

obtaining image data via an image apparatus, the image data comprising (i) at least one of graphic data and text data from a roadway sign, and (ii) street light image data for an intersection on the roadway;

processing the image data via an analyzer module to detect at least one of a graphic image and text from the image data relating to the roadway sign, and to process the street light image data to determine a street light status; and

determining rules for turning on the intersection of the roadway, via an identification module, based on the processed image data relating to the roadway sign, wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the determined street light status;

wherein the identification module is further configured to determine the rules for turning on the intersection based at least in part on recognition and interpretation of the detected text data.

16. The method of claim 15, further comprising performing optical character recognition (OCR) from the image data relating to the roadway sign via the analyzer module to detect text data.

17. The method of claim 16, further comprising determining the rules for turning on the intersection, via the identification module, based at least in part on the detected text data.

18. The method of claim 17, further comprising transmitting, via communications, a warning signal if the identification module determines that the determined street light status does not comply with the determined rules.

19. The method of claim 15, further comprising detecting, via the analyzer module, graphic images from the graphic data.

20. The method of claim 19, further comprising comparing, via the identification module, the detected graphic image to one or more stored graphic images to determine a matching graphic image, and determining the rules for turning on the intersection based at least in part on the matching graphic image.

21. The method of claim 20, further comprising transmitting, via communications, a warning signal if the identification module determines that the determined street light status does not comply with the determined rules.

22. The method of claim 15, further comprising:

obtaining, via a second image apparatus, lane marking image data from the roadway; and

detecting, via a lane marking detection module, lane markings on the roadway from the lane marking image data,

wherein detecting lane markings comprise one of standard frequency dashed lane markings, high frequency dashed lane markings, and solid lane markings and for identifying a turn-only lane and a shared turn lane in response to the detected lane markings.

23. The method of claim 22, wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the detected lane marking.