Computer-implemented system and method for automated and highly accurate plaque analysis, reporting, and visualization

• US 7,324,661 B2
• Filed: 04/30/2004
• Issued: 01/29/2008
• Est. Priority Date: 04/30/2004
• Status: Active Grant

• Current Assignee: Colgate-Palmolive Company
• Original Assignee: Colgate-Palmolive Company
• Inventors: Talukder, Ashit, Kemp, James Herbert, Lambert, James, Lam, Raymond
• Primary Examiner(s): Ahmed; Samir
• Assistant Examiner(s): Bhatnagar; Anand
• Application Number:
  US10/836,567
• Publication Number:
  US 20050244794A1
• Time in Patent Office: 1,369 Days
• Field of Search: 382/100, 382/128, 382/181, 382/190, 128/922, 424/9.71
• US Class Current: 382/100
• CPC Class Codes: G01N 33/4833 of solid biological material, e.g. tissue samples, cell cultures tissue in vivo A61B5/00; cell suspensions G01N33/48735
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: Y10S 128/922 including image analysis

Abstract

A computer-implemented system and method of intra-oral analysis for measuring plaque removal is disclosed. The system includes hardware for real-time image acquisition and software to store the acquired images on a patient-by-patient basis. The system implements algorithms to segment teeth of interest from surrounding gum, and uses a real-time image-based morphing procedure to automatically overlay a grid onto each segmented tooth. Pattern recognition methods are used to classify plaque from surrounding gum and enamel, while ignoring glare effects due to the reflection of camera light and ambient light from enamel regions. The system integrates these components into a single software suite with an easy-to-use graphical user interface (GUI) that allows users to do an end-to-end run of a patient record, including tooth segmentation of all teeth, grid morphing of each segmented tooth, and plaque classification of each tooth image.

Claims

1. A method for automatically evaluating the efficacy of plaque removal and whitening capabilities for different toothbrushes and toothpastes on a tooth-by-tooth basis in real-time, comprising:

determining a central portion of each tooth with a watershed segmentation algorithm using a center of each tooth as a seed point;

applying a reverse watershed segmentation algorithm using an external boundary surrounding each tooth and an internal portion of each tooth as two seed regions, wherein the point at which the two seed regions intersect correspond to the actual tooth boundary;

applying a line-based morphing procedure to automatically overlay a grid on each segmented tooth; and

classifying plaque regions of each tooth.

2. A method as recited in claim 1, wherein prior to the central portion determination, the method further comprises:

preprocessing data about each tooth to handle variations in lighting conditions and in the color and contrast of each tooth.

3. A method as recited in claim 2, wherein the preprocessing data comprises:

filtering an input color image of the data using edge kernels in each color channel to output edge images; and

normalizing each edge image based upon a variance of each edge image.

4. A method as recited in claim 1, wherein the central portion determination comprises:

locating a few pixels on each tooth by determining a cumulative histogram for color channels;

finding the brightest regions in the color channels; and

extracting the largest color region closest to the central region of each tooth.

5. A method as recited in claim 1, wherein the applying a reverse watershed segmentation algorithm comprises:

growing each tooth region and a background region simultaneously; and

labeling as true tooth boundaries the intersections of the tooth and background regions.

6. A method as recited in claim 1, wherein the applying a line-based morphing procedure comprises:

determining a center of gravity of each segmented tooth;

yielding a line representation of a template tooth and each tooth with an automatic correspondence between each; and

applying a line-based warping rule to produce a grid warped onto each segmented tooth.

7. A method as recited in claim 1, wherein classifying plaque regions of each tooth comprises using a three-class Fisher Linear Discriminant linear pattern recognition classifier.

8. A computer-implemented method for automatically evaluating the efficacy of plaque removal and whitening capabilities for different toothbrushes and toothpastes on a tooth-by-tooth basis in real-time, comprising:

determining a central portion of each tooth with a watershed segmentation algorithm using a center of each tooth as a seed point;

applying a reverse watershed segmentation algorithm using an external boundary surrounding each tooth and an internal portion of each tooth as two seed regions, wherein the point at which the two seed regions intersect correspond to the actual tooth boundary;

applying a line-based morphing procedure to automatically overlay a grid on each segmented tooth; and

classifying plaque regions of each tooth.

9. A computer-implemented method as recited in claim 8, wherein prior to the central portion determination, the method further comprises:

preprocessing data about each tooth to handle variations in lighting conditions and in the color and contrast of each tooth.

10. A computer-implemented method as recited in claim 9, wherein the preprocessing data comprises:

filtering an input color image of the data using edge kernels in each color channel to output edge images; and

normalizing each edge image based upon a variance of each edge image.

11. A computer-implemented method as recited in claim 8, wherein the central portion determination comprises:

locating a few pixels on each tooth by determining a cumulative histogram for color channels;

finding the brightest regions in the color channels; and

extracting the largest color region closest to the central region of each tooth.

12. A computer-implemented method as recited in claim 8, wherein the applying a reverse watershed segmentation algorithm comprises:

growing each tooth region and a background region simultaneously; and

labeling as true tooth boundaries the intersections of the tooth and background regions.

13. A computer-implemented method as recited in claim 8, wherein the applying a line-based morphing procedure comprises:

determining a center of gravity of each segmented tooth;

yielding a line representation of a template tooth and each tooth with an automatic correspondence between each; and

applying a line-based warping rule to produce a grid warped onto each segmented tooth.

14. A computer-implemented method as recited in claim 8, wherein classifying plaque regions of each tooth comprises using a three-class Fisher Linear Discriminant linear pattern recognition classifier.

15. A system for automatically evaluating the efficacy of plaque removal and whitening capabilities for different toothbrushes and toothpastes on a tooth-by-tooth basis in real-time, comprising:

a memory configured to store instructions; and

a processor configured to execute the instructions stored in the memory, the instructions being for;

determining a central portion of each tooth with a watershed segmentation algorithm using a center of each tooth as a seed point,

applying a reverse watershed segmentation algorithm using an external boundary surrounding each tooth and an internal portion of each tooth as two seed regions, wherein the point at which the two seed regions intersect correspond to the actual tooth boundary,

applying a line-based morphing procedure to automatically overlay a grid on each segmented tooth, and

classifying plaque regions of each tooth.

16. A system as recited in claim 15, wherein prior to the central portion determination instructions, the processor executes further instructions for:

preprocessing data about each tooth to handle variations in lighting conditions and in the color and contrast of each tooth.

17. A system as recited in claim 16, wherein the preprocessing data instructions comprise:

filtering an input color image of the data using edge kernels in each color channel to output edge images; and

normalizing each edge image based upon a variance of each edge image.

18. A system as recited in claim 15, wherein the central portion determination instructions comprise:

locating a few pixels on each tooth by determining a cumulative histogram for color channels;

finding the brightest regions in the color channels; and

extracting the largest color region closest to the central region of each tooth.

19. A system as recited in claim 15, wherein the applying a reverse watershed segmentation algorithm instructions comprise:

growing each tooth region and a background region simultaneously; and

labeling as true tooth boundaries the intersections of the tooth and background regions.

20. A system as recited in claim 15, wherein the applying a line-based morphing procedure instructions comprise:

determining a center of gravity of each segmented tooth;

yielding a line representation of a template tooth and each tooth with an automatic correspondence between each; and

applying a line-based warping rule to produce a grid warped onto each segmented tooth.

21. A system as recited in claim 15, wherein classifying plaque regions of each tooth instructions comprise using a three-class Fisher Linear Discriminant linear pattern recognition classifier.

22. A computer readable medium stores instructions executable by at least one processor to perform a method for automatically evaluating the efficacy of plaque removal and whitening capabilities for different toothbrushes and toothpastes on a tooth-by-tooth basis in real-time, comprising:

instructions for determining a central portion of each tooth with a watershed segmentation algorithm using a center of each tooth as a seed point;

instructions for applying a reverse watershed segmentation algorithm using an external boundary surrounding each tooth and an internal portion of each tooth as two seed regions, wherein the point at which the two seed regions intersect correspond to the actual tooth boundary;

instructions for applying a line-based morphing procedure to automatically overlay a grid on each segmented tooth; and

instructions for classifying plaque regions of each tooth.

23. A computer readable medium as recited in claim 22, wherein prior to the central portion determination instructions, the computer readable medium further comprises:

instructions for preprocessing data about each tooth to handle variations in lighting conditions and in the color and contrast of each tooth.

24. A computer readable medium as recited in claim 23, wherein the preprocessing data instructions comprise:

instructions for filtering an input color image of the data using edge kernels in each color channel to output edge images; and

instructions for normalizing each edge image based upon a variance of each edge image.

25. A computer readable medium as recited in claim 22, wherein the central portion determination instructions comprise:

instructions for locating a few pixels on each tooth by determining a cumulative histogram for color channels;

instructions for finding the brightest regions in the color channels; and

instructions for extracting the largest color region closest to the central region of each tooth.

26. A computer readable medium as recited in claim 22, wherein the applying a reverse watershed segmentation algorithm instructions comprise:

instructions for growing each tooth region and a background region simultaneously; and

instructions for labeling as true tooth boundaries the intersections of the tooth and background regions.

27. A computer readable medium as recited in claim 22, wherein the applying a line-based morphing procedure instructions comprise:

instructions for determining a center of gravity of each segmented tooth;

instructions for yielding a line representation of a template tooth and each tooth with an automatic correspondence between each; and

instructions for applying a line-based warping rule to produce a grid warped onto each segmented tooth.

28. A computer readable medium as recited in claim 22, wherein classifying plaque regions of each tooth instructions comprise instructions for using a three-class Fisher Linear Discriminant linear pattern recognition classifier.