Method and apparatus for receiving listings of similar vehicles from an image

• US 10,169,675 B2
• Filed: 03/10/2017
• Issued: 01/01/2019
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: Blinker Inc.
• Original Assignee: Blinker Inc.
• Inventors: Wilbert, Anthony Russell, Wach, Hans Brandon, Chung, David Ching-Chien
• Primary Examiner(s): Woldemariam, Aklilu
• Application Number:
  US15/455,482
• Publication Number:
  US 20170185858A1
• Time in Patent Office: 662 Days
• Field of Search: 382104, 382105, 382176, 382182, 348148, 348149, 340445
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2210/22 Cropping
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Some aspects of the invention relate to a mobile apparatus including an image sensor configured to convert an optical image into an electrical signal. The optical image includes an image of a vehicle license plate. The mobile apparatus includes a license plate detector configured to process the electrical signal to recover information from the vehicle license plate image. The mobile apparatus includes an interface configured to transmit the vehicle license plate information to a remote apparatus and receive a listing corresponding to another vehicle in response to the transmission.

Claims

1. A mobile apparatus, comprising:

an image sensor configured to convert an optical image into an electrical signal, the optical image including an image of a vehicle license plate;

a storage for storing instructions;

a processor communicatively coupled to the storage, the processor configured to retrieve the instructions;

a license plate detector, which when executed by the processor;

processes the electrical signal to identify one or more object images from the image, each of said one or more images comprising a candidate vehicle license plate image,

processes the electrical signal to crop the image to said identified one or more of the object images,

processes the electrical signal to score each of the cropped one or more of the object images based on a probability that the cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by;

detecting a plurality of alphanumeric characters in the object image, and

applying a score based on a position of one of the plurality of characters relative a position of another one of the plurality of characters, and

based on the scored object images, processes, at the mobile apparatus, the electrical signal to recover information from the vehicle license plate image; and

an interface configured to transmit the vehicle license plate information to a remote apparatus and receive a listing corresponding to another vehicle in response to the transmission.

2. The mobile apparatus of claim 1, wherein the license plate detector further recovers the information from a portion of the electrical signal corresponding to the selected one of said identified one or more of the object images.

3. The mobile apparatus of claim 1, wherein the license plate detector further processes the electrical signal to dewarp at least one of said one or more of the object images.

4. The mobile apparatus of claim 1, further comprising a display and a rendering module, which when executed by the processor, renders the optical image to the display and overlay a detection indicator on each of said identified one or more of the object images in the optical image.

5. The mobile apparatus of claim 1, further comprising an image filter which when executed by the processor:

applies a set of filter parameters to the electrical signal; and

dynamically adjusts the parameters based on at least one of color temperature, ambient light, object image location, and the location of the apparatus.

6. A mobile apparatus, comprising:

an image sensor configured to convert an optical image into an electrical signal, wherein the optical image comprises a plurality of object images, and wherein one of the plurality of object images comprises an image of a vehicle license plate;

a storage for storing instructions;

a processor communicatively coupled to the storage, the processor configured to retrieve the instructions;

a license plate detector which when executed by the processor;

processes the electrical signal to identify one or more object images from the image, each of said one or more images comprising a candidate vehicle license plate image,

processes the electrical signal to crop the image to said identified one or more of the object images,

processes the electrical signal to score each of the cropped one or more of the object images based on a probability that the cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by;

detecting a plurality of alphanumeric characters in the object image, and

applying a score based on a position of one of the plurality of characters relative a position of another one of the plurality of characters, and

based on the scored object images, processes, at the mobile apparatus, the electrical signal to recover information from the vehicle license plate image from a portion of the electrical signal corresponding to said one of the plurality of object images; and

an interface configured to transmit the vehicle license plate information to a remote apparatus and receive a listing corresponding to another vehicle in response to the transmission.

7. The mobile apparatus of claim 6, wherein the license plate detector further processes the electrical signal to dewarp said one of the object images prior to recovering the information.

8. The mobile apparatus of claim 6, further comprising a display and a rendering module, which when executed by the processor, renders the optical image to the display and overlay a detection indicator on each of the plurality of object images.

9. The mobile apparatus of claim 6, wherein the license plate detector further selects said one of the plurality of object images based on the selected object image'"'"'s score.

10. The mobile apparatus of claim 8, wherein the rendering module further overlays a first detection indicator over said one of the plurality of object images and a second detection indicator over the other object images of said plurality of object images.

11. The mobile apparatus of claim 10, wherein the rendering module further causes the first detection indicator to appear visually different from the second detection indicator on the display.

12. A mobile apparatus, comprising:

an image sensor configured to convert an optical image into an electrical signal, the optical image comprising an image of a vehicle license plate;

a storage for storing instructions;

a processor communicatively coupled to the storage, the processor configured to retrieve the instructions;

a display;

a rendering module, which when executed by the processor, renders the optical image to the display;

an image filter configured to apply one or more filter parameters to the electrical signal based on at least one of color temperature of the image, ambient light, and motion of the apparatus; and

a license plate detector which when executed by the processor;

processes the electrical signal to identify one or more object images from the image, each of said one or more images comprising a candidate vehicle license plate image,

processes the electrical signal to crop the image to said identified one or more of the object images,

processes the electrical signal to score each of the cropped one or more of the object images based on a probability that the cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by;

detecting a plurality of alphanumeric characters in the object image, and

applying a score based on a position of one of the plurality of characters relative a position of another one of the plurality of characters, and

based on the scored object images, processes, at the mobile apparatus, the electrical signal to recover information from the vehicle license plate image,

wherein the rendering module further overlays a detection indicator on the displayed optical image to assist the user position of the apparatus with respect to the optical image in response to a signal from the image filter,

wherein the rendering module further provides an alert to the display when the license plate detector fails to recover the vehicle license plate information; and

an interface configured to transmit the vehicle license plate information to a remote apparatus and receive a listing corresponding to another vehicle in response to the transmission.

13. The mobile apparatus of claim 12, wherein the optical image comprises a plurality of object images, and wherein the license plate detector further processes the electrical signal to identify one or more of the plurality of object images from the image, each of said one or more of the plurality of object images comprising a candidate vehicle license plate image.

14. The mobile apparatus of claim 13, wherein the license plate detector further processes the electrical signal to dewarp at least one of said one or more of the plurality of object images.

15. A computer program product for a mobile apparatus having an image sensor configured to convert an optical image into an electrical signal, the optical image including a plurality of object images, wherein one of the object images comprises an image of a vehicle license plate, the computer program product comprising:

a non-transitory machine readable medium comprising code to;

process the electrical signal to select said one of the object images,

process the electrical signal to identify one or more object images from the image, each of said one or more images comprising a candidate vehicle license plate image,

process the electrical signal to crop the image to said identified one or more of the object images,

process the electrical signal to score each of the cropped one or more of the object images based on a probability that the cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by;

detecting a plurality of alphanumeric characters in the object image, and

applying a score based on a position of one of the plurality of characters relative a position of another one of the plurality of characters, and

based on the scored object images, process, at the mobile apparatus, a portion of the electrical signal corresponding to the selected said one of the object images to recover information from the vehicle license plate image;

transmit the vehicle license plate information to a remote apparatus; and

receive a listing corresponding to another vehicle in response to the transmission.

16. The computer program product of claim 15, wherein the code to process the electrical signal is further configured to select said one of the object images from said identified one or more of the object images based on the scores.

17. The computer program product of claim 16, wherein the code to process the electrical signal is further configured to recover the information from a portion of the electrical signal corresponding to the selected one of said identified one or more of the object images.

18. The computer program product of claim 15, wherein the code for processing the electrical signal is further configured to dewarp at least one of said one or more of the object images.

19. A mobile apparatus, comprising:

an image sensor configured to convert an optical image into an electrical signal, the optical image including an image of a vehicle license plate;

a timing circuit configured to sample the electrical signal at a frame rate;

a storage for storing instructions;

a processor communicatively coupled to the storage, the processor configured to retrieve the instructions;

a license plate detector, which when executed by the processor;

processes the electrical signal to identify one or more object images from the image, each of said one or more images comprising a candidate vehicle license plate image,

processes the electrical signal to crop the image to said identified one or more of the object images,

processes the electrical signal to score each of the cropped one or more of the object images based on a probability that the cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by;

detecting a plurality of alphanumeric characters in the object image, and

applying a score based on a position of one of the plurality of characters relative a position of another one of the plurality of characters, and

based on the scored object images, processes, at the mobile apparatus, the sampled electrical signal to recover information from the vehicle license plate image; and

an interface configured to transmit the vehicle license plate information to a remote apparatus and receive a listing corresponding to another vehicle in response to the transmission.

20. The apparatus of claim 19, wherein the sampled electrical signal comprises a plurality of samples, and wherein the license plate detector further identifies one or more object images from at least one of the plurality of samples.

21. The apparatus of claim 20, wherein at least one of said one or more of the object images comprising a candidate vehicle license plate image.

22. The apparatus of claim 21, wherein the license plate detector further processes said at least one of the plurality of samples to score each of said one or more of the object images based on a probability that the object image comprises the vehicle license plate image.

23. The apparatus of claim 22, wherein the license plate detector further processes at least first and second samples to score each of said one or more of the object images from each of said at least first and second samples.

24. The apparatus of claim 23, wherein each of said one or more object images detected in the first sample is associated with location information, and wherein the license plate detector further uses the location information to estimate the location of said one or more of the objects in the second sample.

25. The apparatus of claim 22, wherein the license plate detector further stops processing the sampled electrical signal when said score is above a threshold value.

26. The apparatus of claim 20, further comprising:

a display, wherein each sample represents a frame of a video image presented to the display; and

a rendering module, which when executed by the processor, renders each frame of the video image to the display and overlay a detection indicator on each of said identified one or more of the object images.

27. The apparatus of claim 26, wherein the overlay provides a visual indication of the object image associated with the score that is above a threshold value.