Method and apparatus for identifying objects depicted in a videostream

• US 20030016869A1
• Filed: 09/16/2002
• Published: 01/23/2003
• Est. Priority Date: 10/23/1998
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Facet Technology Corp.
• Inventors: Retterath, James Eugene, Laumeyer, Robert Anthony
• Granted Patent: US 6,625,315 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 20/582 of traffic signs

Abstract

The present invention relates to an apparatus for rapidly analyzing frame(s) of digitized video data which may include objects of interest randomly distributed throughout the video data and wherein said objects are susceptible to detection, classification, and ultimately identification by filtering said video data for certain differentiable characteristics of said objects. The present invention may be practiced on pre-existing sequences of image data or may be integrated into an imaging device for real-time, dynamic, object identification, classification, logging/counting, cataloging, retention (with links to stored bitmaps of said object), retrieval, and the like. The present invention readily lends itself to the problem of automatic and semi-automatic cataloging of vast numbers of objects such as traffic control signs and utility poles disposed in myriad settings. When used in conjunction with navigational or positional inputs, such as GPS, an output from the inventative systems indicates the identity of each object, calculates object location, classifies each object by type, extracts legible text appearing on a surface of the object (if any), and stores a visual representation of the object in a form dictated by the end user/operator of the system. The output lends itself to examination and extraction of scene detail, which cannot practically be successfully accomplished with just human viewers operating video equipment, although human intervention can still be used to help judge and confirm a variety of classifications of certain instances and for types of identified objects.

Claims

1. A method of segmenting at least two image frames depicting at least one common object of interest comprising the steps of:

receiving at least two image frames that each depict at least a single common object of interest;

applying a fuzzy logic color filter to said at least two images frames to segment as separate image portions from said at least two image frames each region that contains a group of pixels all having a common color set; and

identifying each of said separate image portions as potentially depicting said single common object of interest.

2. The method of claim 1, wherein the step of identifying comprises saving to a separate memory each of said separate image portions.

3. The method of claim 1, wherein the step of identifying comprises creating a record in a database of a pointer to a bitmap image representing each of said separate image portions.

4. The method of claim 1, further comprising, prior to completing the step of applying said fuzzy logic color filter, the step of converting said at least two image frames from a native color space to a single color space portion of a L*u*v color space.

5. The method of claim 4, wherein the step of apply in said fuzzy logic color filter operates on said image frame to generate a value output signal for each pixel that is at a maximum for only said common color set.

6. The method of claim 5, wherein the step of apply in said fuzzy logic color filter determines said value output signal by location in said L*u*v color space and wherein said value output signals are assigned to a minimal set of mathematically described colors representing all permissible colors and combinations for said common color set.

7. The method of claim 1, wherein the step of applying said fuzzy logic color filter segments the separate image portions such that each image portion contains a range of colors related to said common color set whereby variations between colors of individual pixels are allowed to exist in said group of pixels that comprise each image portion.

8. The method of claim 1, wherein the step of applying said fuzzy logic color filter is utilized with a plurality of common color sets, each color set representing a pre-selected color set associated with said common object of interest.

9. The method of claim 1, wherein said image frames are digitized image frames operably generated by a digital image capture device that is recording said digital image as an image selected from the set consisting of:

live images, a pre-recorded set of images, a series of still images, a digitized version of an original analog image sequence, or any combination thereof.

10. The method of claim 1, wherein said image frames comprise a large number of frames of digitized image data and the method is used as part of a graphic-based search engine to recognize a desired single object within said large number of frames of digitized image data.

11. In a computer system, a computer-readable storage media storing:

at least one computer program that operates to segment a plurality of digitized image frames that each contain a common object of interest by performing for each digitized image frame the steps of;

receiving at least two image frames that each depict at least a single common object of interest;

applying a fuzzy logic color filter to said at least two images frames to segment as separate image portions from said at least two image frames each region that contains a group of pixels all having a common color set; and

identifying each of said separate image portions as potentially depicting said single common object of interest.

12. The storage media of claim 11, wherein the step of identifying comprises saving to a separate memory each of said separate image portions.

13. The storage media of claim 11, wherein the step of identifying comprises creating a record in a database of a pointer to a bitmap image representing each of said separate image portions.

14. The storage media of claim 11, further comprising, prior to completing the step of applying said fuzzy logic color filter, the step of converting said at least two image frames from a native color space to a single color space portion of a L*u*v color space.

15. The storage media of claim 14, wherein the step of apply in said fuzzy logic color filter operates on said image frame to generate a value output signal for each pixel that is at a maximum for only said common color set.

16. The storage media of claim 15, wherein the step of apply in said fuzzy logic color filter determines said value output signal by location in said L*u*v color space and wherein said value output signals are assigned to a minimal set of mathematically described colors representing all permissible colors and combinations for said common color set.

17. The storage media of claim 11, wherein the step of applying said fuzzy logic color filter segments the separate image portions such that each image portion contains a range of colors related to said common color set whereby variations between colors of individual pixels are allowed to exist in said group of pixels that comprise each image portion.

18. The storage media of claim 11, wherein the step of applying said fuzzy logic color filter is utilized with a plurality of common color sets, each color set representing a pre-selected color set associated with said common object of interest.

19. The storage media of claim 11, wherein said image frames are digitized image frames operably generated by a digital image capture device that is recording said digital image as an image selected from the set consisting of:

live images, a pre-recorded set of images, a series of still images, a digitized version of an original analog image sequence, or any combination thereof.

20. The storage media of claim 11, wherein said image frames comprise a large number of frames of digitized image data and the method is used as part of a graphic-based search engine to recognize a desired single object within said large number of frames of digitized image data.

21. A computer system for segmenting a plurality of digitized image frames that each contain a common object of interest:

a frame buffer that stores at least two frames of a digital image, each of said at least two frames containing a common object of interest and being represented in a single color space;

a fuzzy logic color filter operably connected to said frame buffer to generate a value output signal in response to each pixel of each frame of said digital image in said frame buffer signals that is determined by location in said single color space; and

a computer processing system operably connected to said fuzzy logic color filter and said frame buffer that determines whether said value output signals correspond to a set of mathematically described colors representing a color set associated with said common object of interest and, in response, segmenting pixels into a group of pixels representing an image portion of each of said frames that is associated with said common object of interest.

22. The computer system of claim 21, wherein said computer processing system includes a separate memory into which each of said separate image portions for each frame is stored.

23. The computer system of claim 21, wherein said computer processing system includes a database into which a pointer to a bitmap image representing each of said separate image portions in said frame buffer is stored.

24. The computer system of claim 21, wherein said single color space is a L*u*v color space.

25. The computer system of claim 21, wherein said fuzzy logic color filter operates on said at least two frames to generate said value output signal for each pixel that is at a maximum for only said common color set.

26. The computer system of claim 21, wherein said set of mathematically described colors contains a range of colors related to said common color set whereby variations between colors of individual pixels are allowed to exist in said group of pixels that comprise each image portion.

27. The computer system of claim 21, wherein said image frames are digitized image frames operably generated by a digital image capture device that is recording said digital image as an image selected from the set consisting of:

live images, a pre-recorded set of images, a series of still images, a digitized version of an original analog image sequence, or any combination thereof.

28. The computer system of claim 21, wherein said image frames comprise a large number of frames of digitized image data and the computer system is part of a graphic-based search engine to recognize a desired single object within said large number of frames of digitized image data.