Image capture and identification system and process

• US 20020090132A1
• Filed: 11/05/2001
• Published: 07/11/2002
• Est. Priority Date: 11/06/2000
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Evryx Technologies Inc (Ipplex Holdings Corporation)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Granted Patent: US 7,016,532 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: H04N 1/00002 Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for error detection, error correction or monitoring in digital computers or digital computer components G06F11/00
• CPC Class Codes: H04N 1/00005 relating to image data
• CPC Class Codes: H04N 1/00031 Testing, i.e. determining the result of a trial H04N1/00029 takes precedence; using test signals for transmission mode changing H04N2201/33371
• CPC Class Codes: H04N 1/00045 using a reference pattern designed for the purpose, e.g. a test chart
• CPC Class Codes: H04N 1/00068 Calculating or estimating
• CPC Class Codes: H04N 1/00082 Adjusting or controlling interrupting an operation H04N1/00915; inhibiting an operation H04N1/00925
• CPC Class Codes: H04N 1/00827 Arrangements for reading an image from an unusual original, e.g. 3-dimensional objects
• CPC Class Codes: H04N 2201/3225 of data relating to an image, a page or a document

Abstract

An identification method and process for objects from digitally captured images thereof that uses image characteristics to identify an object from a plurality of objects in a database. The image is broken down into parameters such as a Shape Comparison, Grayscale Comparison, Wavelet Comparison, and Color Cube Comparison with object data in one or more databases to identify the actual object of a digital image.

Claims

1. A method for identifying to a user an object from a digitally captured image thereof whose image characteristics are present in at least one database including:

decomposing the digitally captured image into a group of parameters; and

comparing each of said group of parameters with identical parameters of known objects stored in the at least one database; and

producing the best matches from the at least one database to a user to identify the object.

2. The method as defined in claim 1 further including before decomposing the digitally captured image:

determining if the digitally captured image includes at least one symbolic image, and if such is included;

decoding the at least one symbolic image;

comparing the decoded at least one symbolic image to a database of decoded symbolic images; and

determining a level of confidence that the object has been identified.

3. The method as defined in claim 2 further including:

reducing the database choices in accordance with the determined level of confidence with which said comparing each of said group of parameters with identical parameters of known objects stored in the at least one database is performed.

4. The method as defined in claim 1 wherein the at least one database includes:

URLs associated with the objects, and wherein said method includes;

establishing a connection between an URL associated with the identified object and the user.

5. The method as defined in claim 1 wherein the at least one database includes:

operative devices for interaction with the user when identified as the object.

6. The method defined in claim 1 wherein the decomposing the digitally captured image into a group of parameters includes:

performing a radiometric correction of the digitally captured image by;

transforming the digitally captured image to one of a set of digital numbers per color plane, RGB representation; and

normalizing in all color planes with linear gain and offset transformations so that pixel values within each color channel span a full dynamic range of the set.

7. The method defined in claim 6 wherein the decomposing the digitally captured image into a group of parameters after a radiometric correction includes:

analyzing the radiometrically normalized RGB image for regions of similar color, and

mapping region boundaries formatted as an x, y binary image map of the same aspect ratio as the radiometrically normalized RGB image.

8. The method defined in claim 7 wherein the decomposing the digitally captured image into a group of parameters after mapping region boundaries of similar color includes:

grouping regions in increasing number to produce a plurality of groups.

9. The method defined in claim 8 wherein for each group of regions:

generating a bounding box by;

determinining the elliptical major axis of the segment group of an ellipse just large enough to include the entire segment group; and

determining a rectangle with long sides parallel to the elliptical major axis of a size just large enough to completely contain every pixel in the segment group.

10. The method defined in claim 9 wherein for each generated bounding box:

converting all pixels not included in the generated bounding box to a mid-level gray; and

remapping such that the corners of the bounding box are mapped into the corners of an output test image for comparison to the at least one data base.

11. The method defined in claim 10 wherein for each output test image:

producing a high grayscale thereof by;

adding proportionately each R, G, and B pixel of the output test image by;

using the following formula;

L_x,y=0.34*R_x,y+0.55*G_x,y+0.11*B_x,y; and

then

rounding to nearest integer value.

12. The method defined in claim 10 wherein for each output test image:

transforming the RGB image into a normalized Intensity (Y), In-phase (I) and Quadrature-phase (Q) color image;

averaging the Y, I, and Q values for each segment; and

saving

Y_avg=Average Intensity,

I_avg=Average In-phase,

Q_avg=Average Quadrature,

Y_sigma=Intensity standard deviation,

I_sigma=In-phase standard deviation,

Q_sigma=Quadrature standard deviation,

N_pixels=number of pixels in the segment, for each segment as a three-dimensional color space map.

13. The method defined in claim 12 wherein for each three-dimensional color space map:

extracting;

the region outer edge boundary;

the total area enclosed by the region outer edge boundary;

the area centroid of the total area enclosed by the region outer edge boundary; and

the net ellipticity of the closest fit ellipse to the region.

14. The method defined in claim 12 wherein for each output test image:

subsampling the output test image to produce a low resolution grayscale by;

combining a number of pixels in both x and y directions, and saving the brightness result at a reduced dynamic range.

15. The method defined in claim 11 wherein for each output test image:

transforming the RGB image into a normalized Intensity (Y), In-phase (I) and Quadrature-phase (Q) color image;

averaging the Y, I, and Q values for each segment; and

saving

Y_avg=Average Intensity,

I_avg=Average In-phase,

Q_avg=Average Quadrature,

Y_sigma=Intensity standard deviation,

I_sigma=In-phase standard deviation,

Q_sigma=Quadrature standard deviation,

N_pixels=number of pixels in the segment, for each segment as a three-dimensional color space map.

16. The method defined in claim 15 wherein for each three-dimensional color space map:

extracting;

the region outer edge boundary;

the total area enclosed by the region outer edge boundary;

the area centroid of the total area enclosed by the region outer edge boundary; and

the net ellipticity of the closest fit ellipse to the region.

17. The method defined in claim 16 wherein for each output test image:

subsampling the output test image to produce a low resolution grayscale by;

combining a number of pixels in both x and y directions, and saving the brightness result at a reduced dynamic range.

18. The method defined in claim 17 wherein each determined output:

comparing each against the at least one database; and

producing best matches from the at least one database.

19. The method defined in claim 18 wherein each determined output are compared against the at least one database in parallel.

20. The method defined in claim 18 wherein each determined output are compared against the at least one database in combinations.

21. The method defined in claim 18 wherein the at least one database is indexed and each determined output are compared against the at least one database in indexed areas thereof which contain similar data to the determined output.

22. A method for identifying to a user a multi-dimensional object from at least one digitally captured image thereof whose image characteristics are present in at least one database including:

decomposing the at least one digitally captured image into a group of parameters of different types; and

comparing the parameters of each type of the group of different parameter types with parameters of matching types of the different parameter types of known objects stored in the at least one database; and

producing a parameter match to a user from at least one parameter type, with respect to the at least one database for a high probability identification of the object.

23. The method defined in claim 22 wherein the producing a parameter match to a user from at least one parameter type, with respect to the at least one database for a high probability identification of the object includes:

producing the best parameter match from a plurality of parameters.

24. The method defined in claim 22 wherein the producing a parameter match to a user from at least one parameter type, with respect to the at least one database for a high probability identification of the object includes:

producing the best match from at least one group of a plurality of parameter types.

25. The method defined in claim 22 wherein the decomposing the digitally captured image into a group of parameters of different types includes:

performing a radiometric correction of the at least one digitally captured image by;

transforming the at least one digitally captured image to at least one of a set of digital numbers per color plane, RGB representation; and

normalizing in all color planes with linear gain and offset transformations so that pixel values within each color channel span a full dynamic range of the set.

26. The method defined in claim 25 wherein the decomposing the at least one digitally captured image into a group of parameters of different types after a radiometric correction includes:

analyzing the radiometrically normalized RGB image for regions of similar color, and

mapping region boundaries formatted as an x, y binary image map of the same aspect ratio as the radiometrically normalized RGB image.

27. The method defined in claim 26 wherein the decomposing the at least one digitally captured image into a group of parameters of different types after mapping region boundaries of similar color includes:

grouping regions in increasing number to produce a plurality of groups.

28. The method defined in claim 27 wherein for each group of regions:

generating a bounding box by;

determinining the elliptical major axis of the segment group of an ellipse just large enough to include the entire segment group;

determining a rectangle with long sides parallel to the elliptical major axis of a size just large enough to completely contain every pixel in the segment group;

converting all pixels not included in the generated bounding box to a mid-level gray; and

remapping such that the corners of the bounding box are mapped into the corners of an output test image for comparison to the at least one data base.

29. The method defined in claim 28 wherein for each output test image:

producing a high grayscale thereof by;

adding proportionately each R, G, and B pixel of the output test image by;

using the following formula;

L_x,y=0.34*R_x,y+0.55*G_x,y+0.11*B_x,y; and

then

rounding to nearest integer value.

30. The method defined in claim 28 wherein for each output test image:

transforming the RGB image into a normalized Intensity (Y), In-phase (I) and Quadrature-phase (Q) color image;

averaging the Y, I, and Q values for each segment; and

saving

Y_avg=Average Intensity,

I_avg=Average In-phase,

Q_avg=Average Quadrature,

Y_sigma=Intensity standard deviation,

I_sigma=In-phase standard deviation,

Q_sigma=Quadrature standard deviation,

N_pixels=number of pixels in the segment, for each segment as a three-dimensional color space map.