Object location and span determination method and apparatus which determines a location and span of an object in an image

• US 6,028,956 A
• Filed: 04/04/1997
• Issued: 02/22/2000
• Est. Priority Date: 04/04/1997
• Status: Expired due to Term

• Current Assignee: Ip Ot Sub Ulc
• Original Assignee: Kofile, Inc. (Eastman Software, Inc.)
• Inventors: Shustorovich, Alexander, Thrasher, Christopher W.
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US08/833,372
• Time in Patent Office: 1,054 Days
• Field of Search: 382/155, 382/156, 382/158, 382/159, 382/161, 382/102, 382/173, 382/178, 382/181-188, 382/224-228, 382/190-195, 706/20, 706/179-180, 706/10, 706/15-16, 706/25, 706/31, 706/42, 706/43
• US Class Current: 382/156
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/147 Determination of region of interest
• CPC Class Codes: G06V 30/166 Normalisation of pattern dimensions

Abstract

Apparatus and method for determining a location and span of an object in an image. The determined location and span of the object are used to process the image to simplify a subsequent classification process.

Claims

1. A method for processing an image containing an object to be subsequently classified, the method comprising steps of:

a) generating windows;

b) applying the generated windows to the image to form windowed images;

c) determining a position of the object by providing the windowed images to a trained learning mechanism;

d) determining a span of the object by providing the windowed images to the trained learning mechanism;

e) normalizing the object based on the determined span to form a normalized object; and

f) centering the normalized object within a classification window based on the determined position,

wherein a subsequent classification of the object may be performed on the classification window.

2. The method of claim 1 wherein the step of generating windows includes sub-steps of:

i) determining a first horizontal position at a first horizontal side of the image;

ii) applying a window having a predetermined size at the first horizontal position of the image;

iii) determining a next horizontal position;

iv) applying a window having the predetermined size at the next horizontal position; and

v) repeating sub-steps (iii) and (iv).

3. The method of claim 2 wherein the image is defined by a two-dimensional array of pixels, and

wherein the sub-step of determining a next horizontal position includes a step of moving the window a predetermined number of pixels in a first horizontal direction away from the first horizontal side of the image.

4. The method of claim 1 wherein the the trained learning mechanism is trained to determine the position and span of an object.

5. The method of claim 1 wherein the steps of (c) determining a position of the object by providing the windowed images to a trained learning mechanism, and (d) determining a span of the object by providing the windowed images to a trained learning mechanism, collectively, includes

the step of post-processing outputs of the trained learning mechanism.

6. The method of claim 5 wherein the sub-step of post-processing outputs of the trained learning mechanism includes steps of:

A) filtering the outputs of the trained learning mechanism to generate filtered outputs;

B) thresholding the filtered outputs to generate thresholded, filtered outputs;

C) determining the position the object based on a center of mass of the thresholded, filtered outputs; and

D) determining the span of the object based on the thresholded, filtered outputs.

7. The method of claim 1 further comprising steps of:

a1) detecting sub-images within the image;

a2) normalizing the detected sub-images; and

a3) reassembling the normalized detected sub-images.

8. The method of claim 1 further comprising a step of:

a1) iteratively applying finer resolution sub-images of the image, as an input, until the position of the object can be determined.

9. The method of claim 1 wherein the position of the object determined is a horizontal coordinate of a center of the object.

10. A method for classifying an object contained in an image, the method comprising steps of:

a) generating windows;

b) applying the generated windows to the image to form windowed images;

c) determining a position of the object by providing the windowed images to a trained learning mechanism;

d) determining a span of the object by providing the windowed images to the trained learning mechanism;

e) normalizing the object based on the determined span to form a normalized object;

f) centering the normalized object within a classification window based on the determined position; and

g) classifying the object based on the classification window to generate a classification result.

11. The method of claim 10 further comprising a step of:

h) postprocessing the classification result based on at least one of (i) a confidence measure, and (ii) the determined position of the object.

12. The method of claim 10 further comprising steps of:

a1) detecting sub-images within the image;

a2) normalizing the detected sub-images;

a3) reassembling the normalized detected sub-images; and

h) postprocessing the classification result based on at least one of (i) a confidence measure, (ii) the determined position of the object, and (iii) a sub-image boundary.

13. The method of claim 10 wherein the position of the object determined is a horizontal coordinate of a center of the object.

14. A device for processing an image having an object to be classified, the device comprising:

a) means for generating windows and for applying the generated windows to the image to form windowed images;

b) means for determining a position and a span of the object by providing the windowed images to a trained learning mechanism;

c) means for normalizing the object based on the determined span to form a normalized object; and

d) means for centering the normalized object within a classification window based on the determined position,

wherein the normalized object centered within the classification window may be classified.

15. The device of claim 14 wherein the trained leaning mechanism is a neural network.

16. The device of claim 15 wherein the neural network includes an input layer of neurons, a Gabor layer of neurons having inputs coupled with outputs of the neurons of the input layer, at least one hidden layer of neurons, and an output layer of neurons.

17. The device of claim 15 wherein the neural network includes:

i) an input layer of neurons,

ii) a Gabor layer of neurons having inputs coupled with outputs of the neurons of the input layer,

iii) a first hidden layer of neurons having inputs coupled with outputs of the neurons of the Gabor layer,

iv) a second hidden layer of neurons having inputs coupled with outputs of the neurons of the first hidden layer,

v) a third hidden layer of neurons having inputs coupled with outputs of the neurons of the second hidden layer, and

vi) an output layer of neurons having inputs coupled with outputs of the neurons of the third hidden layer.

18. The device of claim 14 wherein the means for determining a position and a span of the object by providing the windowed images to a trained learning mechanism includes:

i) the trained learning mechanism providing outputs based on the windowed images; and

ii) means for post-processing the outputs of the trained learning mechanism.

19. The device of claim 18 wherein the means for post-processing includes:

A) means for filtering the outputs of the trained learning mechanism to generate filtered outputs;

B) means for thresholding the filtered outputs to generate thresholded, filtered outputs;

C) means for determining the position of the object based on a center of mass of the thresholded, filtered outputs; and

D) means for determining the span of the object based on the thresholded, filtered outputs.

20. The device of claim 14 further comprising:

a1) means for detecting sub-images within the image;

a2) means for normalizing the detected sub-images; and

a3) means for reassembling the normalized detected sub-images.

21. The device of claim 14 further comprising:

e) means for classifying the object based on the classification window to generate a classification result.

22. The device of claim 21 further comprising:

f) means for postprocessing the classification result based on at least one of (i) a confidence measure, and (ii) the determined position of the object.

23. The device of claim 21 further comprising:

a1) means for detecting sub-images within the image;

a2) means for normalizing the detected sub-images;

a3) means for reassembling the normalized detected sub-images; and

f) means for postprocessing the classification result based on at least one of (i) a confidence measure, (ii) the determined position of the object, and (iii) a sub-image boundary.

24. The device of claim 14 wherein the position of the object is a horizontal coordinate of a center of the object.