Multi-modal speed limit assistant

• US 8,725,394 B2
• Filed: 10/07/2008
• Issued: 05/13/2014
• Est. Priority Date: 10/09/2007
• Status: Expired due to Fees

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG, Siemens Corp. (Siemens AG)
• Inventors: Bahlmann, Claus, Pellkofer, Martin, Giebel, Jan, Baratoff, Gregory
• Primary Examiner(s): SHAPIRO, JEFFREY ALAN
• Application Number:
  US12/246,726
• Publication Number:
  US 20100188288A1
• Time in Patent Office: 2,044 Days
• Field of Search: 340/936, 340/937, 340/907, 348/113, 382/104, 382/107, 701117-119, 701/507
• US Class Current: 701/117
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 30/146 Speed limiting
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages

Abstract

A speed limit assistant (SLA) system includes a camera based SLA, a map based SLA, and a fusion unit. The camera based SLA is configured to determine a first set of probabilities for an input image, wherein the probabilities indicate how likely the image includes a discrete set of speed limit signs. The map based SLA is configured to determine a second set of probabilities for an input coordinate, wherein the probabilities indicate how likely the coordinate is to correspond to one of a discrete set of speed limits. The fusion unit is configured to perform a Bayesian fusion on the first and second set of probabilities to determine a final speed limit from the discrete set of speed limits.

Claims

1. A method of estimating a speed limit of a speed limit sign, the method comprising:

receiving, by a processor, at least one first tuple from a system having a camera configured to capture an image of a speed limit sign and process the image to generate the first tuples, wherein each first tuple includes an estimated speed limit of the sign and a probability that the estimated speed limit is correct;

receiving, by a processor, at least one second tuple from a global positioning system (GPS), wherein each second tuple includes a speed limit associated with a coordinate and a probability that the speed limit is correct; and

estimating, by a processor, a final speed limit of the image of the speed limit sign by tuples performing a Bayesian fusion on the probabilities of the first and second tuples having the same speed limit,

wherein the image is captured when a vehicle is in a first street segment and the fusion is delayed until the vehicle enters a second adjacent segment from which the coordinate is acquired.

2. The method of claim 1, wherein each second tuple corresponds to one of a plurality of street types.

3. The method of claim 2, wherein the street type has upper and lower boundary speed limits, and the probabilities of the second tuples having a speed limit outside the boundaries are lower than the probabilities of the second tuples having a speed limit within the boundaries.

4. The method of claim 2, wherein the street type has a recommended speed limit, and the probabilities of the second tuples having a speed limit not equal to the recommended speed limit are lower than the probabilities of the second tuples having a speed limit equal to the limit.

5. The method of claim 1, wherein the performing the Bayesian fusion comprises:

multiplying probabilities of the first and second tuples for each of the corresponding speed limits to generate product values;

determining a highest one of the product values; and

setting the final speed limit to the speed limit corresponding to the highest determined product value.

6. The method of claim 5, wherein the product values are normalized before the highest one is determined.

7. A method of estimating a speed limit of a speed limit sign, the method comprising:

determining, by a processor, a type of a street that corresponds to a coordinate from a global positioning system (GPS);

determining, by a processor, at least one speed limit that corresponds to the determined street type;

determining, by a processor, first probabilities for each tuple comprising the determined street type and a corresponding one of the speed limits;

receiving second probabilities from a system having a camera configured to capture an image of the speed limit sign and process the image to generate the second probabilities, wherein each second probability is mapped to a corresponding one of the speed limits; and

estimating, by a processor, a final speed limit of the image of the speed limit sign by combining the first and second probabilities for each same speed limit,

wherein the image is captured when a vehicle is in a first street segment and the estimating is delayed until the vehicle enters a second adjacent segment from which the coordinate is acquired.

8. The method of claim 7, wherein the combining comprises performing a Bayesian fusion on the first and second probabilities for each same speed limit.

9. The method of claim 7, wherein the street type has a corresponding maximum and minimum speed limit values, and the probabilities of the first tuples including speed limits outside the minimum and maximum are set to zero.

10. The method of claim 7, wherein performing the Bayesian fusion comprises:

multiplying the first and second probabilities for each of the corresponding speed limits to generate product values;

determining a highest one of the product values; and

setting the final speed limit to the speed limit corresponding to the highest determined product value.

11. The method of claim 7, wherein the product values are normalized before the highest one is determined.

12. A non-transitory computer readable medium, tangibly embodying a program of instructions executable by a processor to perform method steps for determining a speed limit of a speed limit sign, the method steps comprising:

determining a recommended speed limit for a first coordinate of a global positioning system (GPS);

determining first probabilities for speed limits at and below the recommended speed limit, herein each first probability indicates a likelihood that the corresponding speed limit is correct for the first coordinate;

receiving second probabilities from a system including a camera that captures an image of the speed limit sign and processes the image to determine the second probabilities, wherein each second probability indicates a likelihood that each of the speed limits corresponds to the speed limit sign; and

estimating a speed limit of the speed limit sign by combining the first and second probabilities for each same speed limit,

wherein the image is captured when a vehicle is in a first street segment and the first coordinate is determined by projecting a second coordinate from the GPS along the first street segment to a point along a second street segment adjacent the first.

13. The program storage device of claim 12, wherein the determining of the recommended speed limit comprises:

determining a road segment from a table of road segments that corresponds to the coordinate;

determining a type of the road segment; and

determining the recommended speed limit from a table of recommended speed limits using the type.

14. The program storage device of claim 12, wherein the combining comprises performing a Bayesian Fusion on the first and second probabilities for each same speed limit.