Computer vision system for subject characterization

• US 6,031,934 A
• Filed: 10/15/1997
• Issued: 02/29/2000
• Est. Priority Date: 10/15/1997
• Status: Expired due to Term

• Current Assignee: Hanger Solutions, LLC (IP Investments Group LLC)
• Original Assignee: Planet Electric Incorporated
• Inventors: Ahmad, Subutai, Hunter, Kevin L.
• Primary Examiner(s): Au, Amelia
• Assistant Examiner(s): Bali, Vikkram
• Application Number:
  US08/950,404
• Time in Patent Office: 867 Days
• Field of Search: 382/284, 382/294, 382/201, 382/203, 382/195, 382/282, 382/283, 382/154, 395/435, 395/116, 395/467, 348/584, 348/586, 348/489, 348/594, 348/595
• US Class Current: 382/154
• CPC Class Codes: G06T 7/66 of image moments or centre of gravity
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

Methods and apparatuses are disclosed for determining one or more characteristics of a sub-image within an electronic image.

In one embodiment, the present invention includes an alpha image generator and a characterizer. The alpha image generator provides an alpha image of the subject being characterized, separated from a background of the input image. In another embodiment, the alpha image generator also may provide a summed area table. The characterizer determines a characteristic of the subject from the alpha image.

In another embodiment, a first characteristic of the subject is derived from the alpha image. The characterizer determines a second characteristic based upon the first characteristic. In yet another embodiment, the determination of the second characteristic is simplified by eliminating all the unlikely estimates of the second characteristic in the overall potential estimates of the second characteristic. In one embodiment, simplification is accomplished by segmenting the alpha image and eliminating portions of the alpha image that do not correspond to the second characteristic. In a further embodiment, the remaining potential estimates are scored based upon the correlation of the alpha image with the estimate, the best estimate having the best score.

Claims

1. A computer implemented method for characterizing a sub-image within an image, wherein an alpha image of the sub-image and a first characteristic of the sub-image is provided, comprising determining a second characteristic based upon the first characteristic, wherein the second characteristic is determined by way of a plurality of bounding boxes, the location of the sub-image is compared to the locations of the plurality of bounding boxes and eliminating a subset of the plurality of bounding boxes that are not substantially co-located with the sub-image, wherein a plurality of potential estimates of the second characteristic based in part upon the first characteristic exists such that a subset of the plurality of potential estimates are eliminated from the plurality of potential estimates because the subset is not capable of being an accurate estimate of the second characteristic, whereby the determination of a best estimate is simplified.

2. A method as recited in claim 1, wherein the second characteristic is determined by a best estimate of the second characteristic based in part upon the first characteristic, wherein a plurality of potential estimates of the second characteristic based in part upon the first characteristic exist such that a subset of the plurality of potential estimates are eliminated from the plurality of potential estimates because the subset is not capable of being an accurate estimate of the second characteristic, whereby the determination of the best estimate is simplified.

3. A method as recited in claim 1, wherein the second characteristic is determined by making a first estimate of the second characteristic based upon the first characteristic and the reduced plurality of potential estimates, such that the reduced plurality of potential estimates is further reduced, whereby the determination of the best estimate is simplified.

4. A method as recited in claim 1, wherein the first and second characteristics are interrelated, the second characteristic is determined based upon the interrelationship between the first and second characteristics.

5. A method as recited in claim 1, wherein a best estimate of the second characteristic is determined from a plurality of potential estimates by eliminating a subset of the plurality of potential estimates that are improbable estimates of the second characteristics from the plurality of potential estimates.

6. A method as recited in claim 5, further comprising searching through the remaining estimates in the plurality of potential estimates and determining the best estimate.

7. A method as recited in claim 6, wherein the alpha image is comprised of a plurality of pixels such that the best estimate is chosen from the plurality of potential estimates based upon a status of a portion of the plurality of pixels of the alpha image associated with the second characteristic.

8. A computer readable medium including a computer readable code device configuring a computer to execute as recited in claim 1.

9. A computer implemented method for characterizing a subject comprising:

electronically imaging a scene including a subject, whereby an electronic image of the scene and an alpha image of the subject is created, wherein the alpha image is comprised of a plurality of pixels;

determining a first characteristic of from the alpha image, the first characteristic being indicative of the subject; and

determining a second characteristic indicative of the subject, wherein a plurality of potential estimates exists for the second characteristic, the operation of eliminating the subset of potential estimates including,

eliminating a subset of potential estimates from the plurality of potential estimates, whereby a reduced plurality of potential estimates is provided,

bounding a portion of the alpha image associated with the second characteristic with a plurality of bounding boxes,

determining that a first selected bounding box is empty bounding if the first selected bounding box does not contain any pixels of the portion of the alpha image associated with the second characteristic, wherein the subset of potential estimates eliminated from the plurality of potential estimates includes each estimate that corresponds to the first selected bounding box,

determining that a second selected bounding box is partially empty if the second selected bounding box contains a number of pixels of the portion of the alpha image associated with the second characteristic that is less than a lower threshold, wherein the subset of potential estimates eliminated from the plurality of potential estimates includes each estimate that corresponds to the second selected bounding box,

determining a score for a first estimate of the second characteristic from the reduced plurality of potential estimates, the score being indicative of the probability that the first estimate of the second characteristic is a best estimate, and repeating for each estimate of the reduced plurality of potential estimates, and

comparing the scores of the reduced plurality of potential estimates and choosing the best estimate from the reduced plurality of potential estimates having the best score as the second characteristic.

10. A computer-implemented digital video subject characterizing method comprising:

creating a background model by examining a plurality of frames of an average background image;

obtaining a frame of an input image that contains a subject to be characterized;

creating an alpha image such that a value of each pixel in the alpha image is set according to a value corresponding to a pixel from the input image and a value corresponding to a pixel from the average background image;

deriving a set of templates such that the set of templates comprises the shape of the subject being characterized;

setting the values of pixels in the alpha image according to the set of templates such that portions of the alpha image contained within the templates are adjusted so that the pixels do not represent holes in the subject;

deriving a first characteristic from a selected one of the set of templates of the alpha image; and

blending the subject in the input image onto the destination image.

11. A method as recited in claim 10, wherein the first characteristic is derived from the selected template based in part upon a first predetermined ratio between the first characteristic and the selected template.

12. A method as recited in claim 10, the method further comprising:

deriving a second characteristic from the first characteristic.

13. A method as recited in claim 12, wherein the second characteristic is derived from the first characteristic based in part upon a second predetermined ratio between the first characteristic and the second characteristic.

14. A computer implemented method for characterizing a subject such that the subject is represented in an electronic image, the method comprising:

electronically imaging a scene including a subject, wherein the electronic image of the scene and an alpha image of the subject is created, the alpha image being comprised of a plurality of pixels;

determining a first characteristic from the alpha image, the first characteristic being indicative of the subject;

deriving a plurality of potential estimates of a second characteristic indicative of the subject, based in part on the first characteristic; and

determining a second characteristic indicative of the subject from the plurality of potential estimates, wherein determining the second characteristic includes;

bounding a portion of the alpha image associated with the second characteristic with a plurality of bounding boxes;

determining that a first selected bounding box is empty if the selected bounding box does not contain any pixels of the portion of the alpha image associated with the second characteristic;

eliminating a subset of potential estimates from the plurality of potential estimates, wherein the subset of potential estimates eliminated from the plurality of potential estimates includes each estimate that corresponds to the first selected bounding box, whereby a reduced plurality of potential estimates is provided; and

determining a best estimate from the reduced plurality of potential estimates.

15. A method as recited in claim 14, wherein the operation of deriving the plurality of potential estimates of the second characteristics is based in part upon an interrelationship between the first and second characteristics.

16. A method as recited in claim 15, wherein the interrelationship between the first and second characteristics is a ratio.

17. A computer readable medium including a computer readable code device configuring a computer to perform the method as recited in claim 14.

18. A method as recited in 14, wherein the determination of whether the first selected bounding box is empty is made by utilizing a summed area table.

19. A computer implemented method for characterizing a subject such that the subject is represented in an electronic image, the method comprising:

electronically imaging a scene including a subject, wherein the electronic image of the scene and an alpha image of the subject is created, the alpha image being comprised of a plurality of pixels;

determining a first characteristic from the alpha image, the first characteristic being indicative of the subject;

deriving a plurality of potential estimates of a second characteristic indicative of the subject, based in part on the first characteristic;

determining a second characteristic indicative of the subject from the plurality of potential estimates wherein determining the second characteristic includes;

bounding a portion of the alpha image associated with the second characteristic with a plurality of bounding boxes;

determining that a first selected bounding box is partially empty if the selected bounding box contains a number of pixels of the portion of the alpha image associated with the second characteristic that is less than a lower threshold;

eliminating a subset of potential estimates from the plurality of potential estimates, wherein the subset of potential estimates eliminated from the plurality of potential estimates includes each estimate that corresponds to the second selected bounding box, whereby a reduced plurality of potential estimates is provided, and

determining a best estimate from the reduced plurality of potential estimates.

20. A method as recited in claim 19, wherein the determination of whether the first selected bounding box is partially empty is made by utilizing a summed area table.

21. A computer implemented method for characterizing a subject such that the subject is represented in an electronic image the method comprising:

electronically imaging a scene including a subject, wherein the electronic image of the scene and an alpha image of the subject is created, the alpha image being comprised of a plurality of pixels;

determining a first characteristic from the alpha image, the first characteristic being indicative of the subject;

deriving a plurality of potential estimates of a second characteristic indicative of the subject, based in part on the first characteristic;

determining a second characteristic indicative of the subject from the plurality of potential estimates, wherein determining the second characteristic includes;

determining a score for a first estimate of the second characteristic from the reduced plurality of potential estimates, the score being indicative of the probability that the first estimate of the second characteristic is a best estimate, and repeating for each estimate of the reduced plurality of potential estimates;

comparing the scores of the reduced plurality of potential estimates; and

choosing the best estimate having the best score as the second characteristic from the reduced plurality of potential estimates.

22. A method as recited in claim 21, wherein the operation of determining the second characteristic further includes:

eliminating a selected estimate from the reduced plurality of potential estimates, if the selected estimate is incapable of obtaining a score greater than a score for a previous estimate.

23. A method as recited in claim 21, wherein the operation of determining the second characteristic further includes:

choosing a selected estimate as the best estimate if the score of the selected estimate is greater than a threshold score.

24. A method as recited in claim 23, wherein the threshold score is set such that if the score of the selected estimate is greater than the threshold score then the probability that the selected estimate is the best estimate is greater than about 90%.