Method and apparatus for identifying vehicle information from an image

• US 10,540,564 B2
• Filed: 06/20/2018
• Issued: 01/21/2020
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: Blinker Inc.
• Original Assignee: Blinker Inc.
• Inventors: Wilbert, Anthony Russell, Wach, Hans Brandon, Chung, David Ching-Chien, Rauker, Goran Matko, White, Solomon John
• Primary Examiner(s): Rudolph, Vincent
• Assistant Examiner(s): Schwartz, Raphael
• Application Number:
  US16/013,768
• Publication Number:
  US 20180300578A1
• Time in Patent Office: 580 Days
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/127 with the intervention of an operator
• CPC Class Codes: G08G 1/017 identifying vehicles G08G1/015, G08G1/054 take precedence
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/027 using movement velocity, acceleration information
• CPC Class Codes: H04W 4/40 for vehicles, e.g. vehicle-to-pedestrians [V2P]

Abstract

Some aspects of the invention relate to a mobile apparatus including an image sensor configured to convert an optical image into an electrical signal. The optical image includes an image of a vehicle license plate. The mobile apparatus includes a license plate detector configured to process the electrical signal to recover information from the vehicle license plate image.

Claims

1. A mobile apparatus, comprising:

an image sensor configured to capture at least one optical image into an electrical signal, the at least one optical image including an image of a vehicle license plate; and

a license plate detector configured to;

identify, by a processor, one or more object images from the at least one optical image based on the electrical signal, with each of said one or more object images comprising a candidate vehicle license plate image having a plurality of alphanumeric characters,

crop, by the processor, the at least one optical image to said identified one or more of the object images,

provide a rectangular fitting around each character of the plurality of alphanumeric characters in the one or more object images,

score, by the processor, each of said cropped one or more of the object images by applying a straight line to a center of the respective rectangular fittings of each pair of alphanumeric characters, summing a distance of all other respective rectangular fittings to each respective fitted line, selecting the respective pair of alphanumeric characters corresponding to the respective fitted line with a smallest summed distance of all of the fitted lines as a text baseline for the candidate vehicle license plate, and computing the score for candidate vehicle license plate having the selected text baseline based on the calculated slope of the fitted line for the selected pair of alphanumeric characters,

based on the computer score, obtain vehicle information generated from the candidate vehicle license plate image, with the obtained vehicle information including at least one physical feature of a possible vehicle,

detect vehicle identification information from the at least one optical image, the detected vehicle identification information including at least one identifying feature corresponding to a vehicle in the at least one optical image, and

compare the detected vehicle identification information to the obtained vehicle information generated from the candidate vehicle license plate image to determine an estimated level of accuracy for the candidate vehicle license plate image.

2. The mobile apparatus of claim 1, wherein the vehicle identification information comprises at least one of a make, model, year, and color of the vehicle in the at least one optical image.

3. The mobile apparatus of claim 1, wherein the vehicle identification information comprises at least one of size, shape, and taillight placement of the vehicle in the at least one optical image.

4. The mobile apparatus of claim 1, wherein the vehicle license plate includes identifying characteristics on the vehicle license plate itself to identify an issuing state of the vehicle license plate.

5. The mobile apparatus of claim 1, further comprising a logo detector configured to detect a vehicle manufacture logo of the vehicle in the at least one optical image.

6. The mobile apparatus of claim 1, wherein the license plate detector is further configured to recover the information from a portion of the electrical signal corresponding to the selected one of said identified one or more of the object images.

7. The mobile apparatus of claim 1, wherein the license plate detector is further configured to process the electrical signal to dewarp at least one of said one or more of the object images.

8. The mobile apparatus of claim 1, further comprising a display and a rendering module configured to render the at least one optical image to the display and overlay a detection indicator on each of said identified one or more of the object images in the at least one optical image.

9. The mobile apparatus of claim 8, wherein each of the cropped one or more object images is scored by:

detecting the plurality of alphanumeric characters in the object image, and

applying the score based on a position of one of the plurality of characters relative to a position of another one of the plurality of characters.

10. The mobile apparatus of claim 9, wherein the license plate detector is further configured to process the electrical signal to score each of the plurality of object images based on a probability that the each of the plurality of object images comprises the vehicle license plate image.

11. The mobile apparatus of claim 10, wherein the license plate detector is further configured to select said one of the plurality of object images based on the selected object image'"'"'s score.

12. The mobile apparatus of claim 11, wherein the rendering module is further configured to overlay a first detection indicator over said one of the plurality of object images and a second detection indicator over the other object images of said plurality of object images.

13. The mobile apparatus of claim 12, wherein the rendering module is further configured to cause the first detection indicator to appear visually different from the second detection indicator on the display.

14. The mobile apparatus of claim 8, further comprising an image filter configured to apply one or more filter parameters to the electrical signal based on at least one of color temperature of the image, ambient light, and motion of the mobile apparatus.

15. The mobile apparatus of claim 14,

wherein the rendering module is further configured to overlay a detection indicator on the displayed optical image to assist the user in positioning the apparatus with respect to the optical image in response to a signal from the image filter, and

wherein the rendering module is further configured to provide an alert to the display when the license plate detector fails to recover the vehicle license plate information if the estimated level of accuracy for the candidate vehicle license plate image is below a predefined threshold.

16. A method for detecting a vehicle, the method comprising:

converting, by an image sensor, at least one optical image into an electrical signal, the at least one optical image including an image of a vehicle license plate;

identifying, by a processor, one or more object images from the at least one optical image based on the electrical signal, with each of said one or more object images comprising a candidate vehicle license plate image having a plurality of alphanumeric characters;

cropping, by the processor, the at least one optical image to said identified one or more of the object images;

provide a rectangular fitting around each character of the plurality of alphanumeric characters in the one or more object images;

scoring, by the processor, each of said cropped one or more of the object images by applying a straight line to a center of the respective rectangular fittings of each pair of alphanumeric characters, summing a distance of all other respective rectangular fittings to each respective fitted line, selecting the respective pair of alphanumeric characters corresponding to the respective fitted line with a smallest summed distance of all of the fitted lines as a text baseline for the candidate vehicle license plate, and computing the score for candidate vehicle license plate having the selected text baseline based on the calculated slope of the fitted line for the selected pair of alphanumeric characters;

based on the computer score, obtaining, by the processor, vehicle information generated from the candidate vehicle license plate image, with the obtained vehicle information including at least one physical feature of a possible vehicle;

detecting, by the processor, vehicle identification information from the at least one optical image, the detected vehicle identification information including at least one identifying feature corresponding to a vehicle in the at least one optical image; and

comparing, by the processor, the detected vehicle identification information to the obtained vehicle information generated from the candidate vehicle license plate image to determine an estimated level of accuracy for the candidate vehicle license plate image.

17. The method of claim 16, wherein the vehicle identification information comprises at least one of a make, model, year, and color of the vehicle in the at least one optical image.

18. The method of claim 16, wherein the vehicle identification information comprises at least one of size, shape, and taillight placement of the vehicle in the at least one optical image.

19. The method of claim 16, wherein the vehicle license plate includes identifying characteristics on the vehicle license plate itself to identify an issuing state of the vehicle license plate.

20. The method of claim 16, further comprising detecting, by the processor, a vehicle manufacture logo of the vehicle in the at least one optical image.

21. The method of claim 16, further comprising recovering, by the processor, the information from a portion of the electrical signal corresponding to the selected one of said identified one or more of the object images.

22. The method of claim 16, further comprising processing the electrical signal to dewarp at least one of said one or more of the object images.

23. The method of claim 16, further comprising:

rendering the at least one optical image on a display; and

overlaying a detection indicator on each of said identified one or more of the object images in the at least one optical image.

24. The method of claim 23, further comprising scoring each of the cropped one or more object images by:

detecting the plurality of alphanumeric characters in the object image, and

applying the score based on a position of one of the plurality of characters relative to a position of another one of the plurality of characters.

25. The method of claim 24, further comprising scoring, by the processor, each of the plurality of object images based on a probability that the each of the plurality of object images comprises the vehicle license plate image.

26. The method of claim 25, further comprising selecting said one of the plurality of object images based on the selected object image'"'"'s score.

27. The method of claim 26, further comprising overlaying a first detection indicator over said one of the plurality of object images and a second detection indicator over the other object images of said plurality of object images.

28. The method of claim 27, further comprising causing the first detection indicator to appear visually different from the second detection indicator on the display.

29. The method of claim 16, further comprising applying, by an image filter, one or more filter parameters to the electrical signal based on at least one of color temperature of the image, ambient light, and motion of the mobile apparatus.

30. The method of claim 29, further comprising:

overlaying a detection indicator on the displayed optical image to assist the user in positioning the apparatus with respect to the optical image in response to a signal from the image filter; and

providing an alert to the display when the license plate detector fails to recover the vehicle license plate information if the estimated level of accuracy for the candidate vehicle license plate image is below a predefined threshold.