APPARATUS, METHOD AND ARTICLE FOR EVALUATING A STACK OF OBJECTS IN AN IMAGE

• US 20110052049A1
• Filed: 06/22/2010
• Published: 03/03/2011
• Est. Priority Date: 08/26/2009
• Status: Active Grant

• Current Assignee: Sg Gaming, Inc. (Scientific Games Corporation)
• Original Assignee: Bally Gaming Incorporated (Scientific Games Corporation)
• Inventors: Rajaraman, Arun, Arbogast, Christopher P.
• Granted Patent: US 8,285,034 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06F 2218/00 Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

An object evaluation system may determine a value for a stack of objects that appear in a pixelated color image. To determine the value of the stack of objects, the evaluation system preprocess at least a portion of the pixelated color image to produce a set of two color contour data, processes the two color contour data to identify a location of a top and a bottom of the stack (if any), and locates, for each of the objects in the stack, a respective set of color pixels from the pixelated color image corresponding to each object based on the identified locations of the top and bottom of the stack. Each of the objects in the stack are then classified into a color classification based on the object'"'"'s respective set of color pixels, and the value of the object is determined based on a known correspondence between the color classification and a value. The cumulative value of the stack is determined by summing the determined values for each of the objects in the stack.

Claims

1. A method of computationally processing images of areas which may contain objects, the method comprising:

acquiring a pixelated color image of an area as a set of pixilated color image data at least temporarily stored in at least one processor-readable medium;

computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data that represents at least the portion of the pixilated color image, wherein a first color of the two color contour image data corresponds to contour pixels and a second color of the two color contour image corresponds to non-contour pixels;

computationally processing by the at least one processor the set of two color contour image data to find a location of a bottom of a stack of objects in the set of two color contour image data and to find a location of a top of the stack of objects in the set of two color contour image data, if any objects are in the at least portion of the area, wherein the stack of objects includes at least one object if any objects are in the at least portion of the area;

computationally determining by the at least one processor a total number of objects in the stack, if any based at least in part on a knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects;

computationally processing by the at least one processor a portion of the set of pixelated color image data based on the knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes; and

computationally determining by the at least one processor a cumulative total value of the stack of objects based at least in part on the color class to which each of the objects in the stack of objects is classified.

2. The method of claim 1 wherein acquiring a pixilated color image of an area includes capturing a color image by at least one image acquisition device of at least a portion of a gaming table from a position spaced at a non-zero degree and non-perpendicular angle out of a plane of a playing surface of the gaming table.

3. The method of claim 1 wherein computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data includes:

converting at least a portion of the set of the pixelated color image data to a set of gray scale image data; and

edge detection filtering the set of gray scale image data to produce a set of contour image data.

4. The method of claim 3 wherein computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data further includes:

converting by the at least one processor at least a portion of the set of contour image data to a set of black and white contour image data.

5. The method of claim 1 wherein computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data further includes:

eliminating extraneous pixels from the set of two color contour image data.

6. The method of claim 4 wherein converting by the at least one processor at least a portion of the set of contour image data to a set of black and white contour image data includes:

converting by the at least one processor a set of gray scale contour image data to a set of hue, saturation, and brightness (HSB) contour image data; and

converting by the at least one processor the set of HSB contour image data to the set of black and white contour image data.

7. The method of claim 1 wherein computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data includes:

comparing a set of pixilated color image data representing a pixelated reference image of the area to the set of two color contour image data, said reference image having no stack of objects; and

removing irregularities from the set of two color contour image data around a stack of objects in the two color contour image data based on the comparing.

8. The method of claim 7 wherein the removing irregularities from the set of two color contour image data includes:

subtracting the set of pixilated color image data representing a pixelated reference image of the area from the set of two color contour image data; and

adding back to the set of two color contour image data an element of average background color of the area.

9. The method of claim 1 wherein computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data includes:

color balancing a set of pixilated color image data representing a pixelated reference image of the area to have it more closely match the set of two color contour image data, said reference image having no stack of objects;

rotating the set of two color contour image data to compensate for camera distortion for one or more stacks of objects that are near edges of the two color contour image data;

aligning the set of pixilated color image data representing the pixelated reference image and the set of two color contour image data to account for movement of an image acquisition device used to acquire the pixelated color image;

subtracting the set of pixilated color image data representing a pixelated reference image of the area from the set of two color contour image data;

adding back to the set of two color contour image data image data representing an element of average background color of the area; and

either before the subtracting or after the subtracting, converting the set of pixilated color image data representing the pixelated reference image of the area and the set of two color contour image data to gray scale if the converting to gray scale had been determined likely to improve accuracy of the determining of the total number of objects in the stack.

10. The method of claim 1 wherein computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data further includes:

eliminating extraneous pixels from the set of two color contour image data that are outside the at least portion of the area; and

eliminating extraneous pixels that are not part of groups that at least approximately form the shape of a stack object based on a minimum threshold number of pixels.

11. The method of claim 10 wherein the eliminating extraneous pixels that are not part of groups comprises:

passing a template shaped as a stack object over the set of two color contour image data;

counting a number of contour pixels within an area defined by the template for each location of the template on the set of two color contour image data; and

eliminating all pixels outside the area defined by the template at each location if the number of contour pixels within the area defined by the template contains the minimum threshold number of pixels.

12. The method of claim 1 further comprising:

before computationally determining by the at least one processor a total number of objects in the stack, determining a horizontal center of the stack of objects based on an overall concentration of contour pixels in the set of two color contour image data; and

determining an estimated height of individual objects in the stack of objects based on a mean/average height of an individual object of the stack rationalized for a distance between an image acquisition device used to acquire the pixelated color image and an expected location of the stack, said distance adjusted based on the location of the bottom of the stack at the determined horizontal center of the stack relative to the expected location of the stack of objects.

13. The method of claim 1 wherein computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data further includes:

logically segmenting the two color contour image data into a set of target pixel areas, each of the target pixel areas including a set of n by m pixels.

14. The method of claim 13 wherein computationally preprocessing by at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data further includes eliminating extraneous pixels from the set of two color contour image data by:

for each target pixel area of the set of target pixel areas, eliminating all contour pixels in the target pixel area from the set of two color contour image data if a total number of the contour pixels in the target pixel area is less that a threshold number of pixels per target pixel area; and

for each target pixel area of at least some of the set of target pixel areas, eliminating all contour pixels in the target pixel area from the set of two color contour image data if the target pixel area has less than a nearest neighboring threshold number of nearest neighboring target pixel areas.

15. The method of claim 13 wherein computationally processing by the at least one processor the set of two color contour image data to find a location of a bottom and a top of a stack of objects in the set of two color contour image data includes:

applying a sample bounding box at successive intervals to groups of target pixel areas of the set of target pixel areas to find a location of the bottom of the stack of objects, each of the groups of target pixel areas including at least some target pixel areas with contour pixels of the set of two color contour image data that are below a determined upper location in the set of two color contour image data;

determining a bottom quadrant to be the sample bounding box that includes the group of target pixel areas with the highest number of target pixel areas having contour pixels;

determining whether a width of the target pixel areas included in the bottom quadrant fits a face ellipse of an object;

determining whether a width of the target pixel areas included bottom quadrant fits a face ellipse of an object;

determining that there is no stack present when the width does not fit the face ellipse; and

determining a location of the bottom of the bottom quadrant when the width fits the face ellipse.

16. The method of claim 15 wherein computationally processing by the at least one processor the set of two color contour image data to find a location of a bottom and a top of a stack of objects in the set of two color contour image data includes:

determining a location of a bottom of the bottom quadrant by traversing downwardly starting at a location offset from the determined upper location until a last one or more target pixel areas found within a left and right boundary of the bottom quadrant.

17. The method of claim 13 wherein computationally processing by the at least one processor the set of two color contour image data to find a location of a bottom and a top of a stack of objects in the set of two color contour image data includes:

successively applying a sample bounding box to groups of target pixel areas of the set of target pixel areas at successive intervals from a determined bottom of the stack of objects until a top streak of target pixel areas contained within the sample bounding box is empty.

18. The method of claim 17 wherein computationally processing by the at least one processor the set of two color contour image data to find a location of a bottom and a top of a stack of objects in the set of two color contour image data includes:

determining a top bounding box based on the values of the sample bounding box with the sample bounding box positioned such that the top streak of target quadrants contained within the sample bounding box is empty;

successively applying the sample bounding box at successive intervals to find a sample quadrant that fits an elliptical pattern;

determining a list of coordinates of the target quadrants that intersect a top arc and a bottom arc of an imaginary ellipse the size of the sample bounding box;

determining a total number of target quads that overlap the target quadrants of the imaginary ellipse from the sample bounding box; and

determining the sample bounding box position with the most number of overlaps as the top quadrants.

19. The method of claim 18 wherein computationally determining by the at least one processor a total number of objects in the stack includes determining a difference in the bottom of the top quadrant and a bottom of a bottom quadrant and a mean/average height of an individual object rationalized for a distance between the image acquisition device and the stack of objects.

20. The method of claim 1 wherein computationally processing by the at least one processor a portion of the set of pixelated color image data based on a knowledge of a location of a bottom of the stack of objects and a location of a top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes includes:

for each of the objects, classifying a number of pixels in a color pixel streak to respective hue, saturation, and brightness (HSB) values that represent the hue, saturation and brightness of the number of pixels, the color pixel streak composed of a subset of the pixels between a top arc and a bottom arc of an ellipse of the object.

21. The method of claim 20 wherein computationally processing by the at least one processor a portion of the set of pixelated color image data based on the knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes further includes:

for each of the objects, classifying the number of pixels in the color pixel streak to respective fundamental colors based on defined relationships between HSB values and a defined set of fundamental colors.

22. The method of claim 21 wherein computationally processing by the at least one processor a portion of the set of pixelated color image data based on the knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes further includes:

for each of the objects, determining whether the classified fundamental colors of the color pixel streak is in a set of color drift colors.

23. The method of claim 22 wherein computationally processing by the at least one processor a portion of the set of pixelated color image data based on the knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes further includes:

for each of the objects, determining whether the classified fundamental colors of the color pixel streak are a defined primary fundamental color and a number of defined secondary fundamental colors.

24. The method of claim 23 wherein computationally processing by the at least one processor a portion of the set of pixelated color image data based on the knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes further includes:

for each of the objects, if the classified fundamental colors of the color pixel streak are not in the set of color drift colors and the classified fundamental colors of the color pixel streak are not one of the defined primary fundamental color or the defined secondary fundamental colors, determining a fundamental color based on a most common color shared by the pixels of the color pixel streak.

25. The method of claim 23 wherein computationally processing by the at least one processor a portion of the set of pixelated color image data based on the knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes further includes:

for each of the objects, determining a value of the object based on a determined fundamental color and a defined set of fundamental color to object value relationships.

26. The method of claims 24 wherein computationally processing by the at least one processor a portion of the set of pixelated color image data based on the knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes further includes:

summing the determined values of each of the objects.

27. The method of claim 1 further comprising:

reclassifying an object into a different color class based on a determination that a color of the object is obscured by a shadow in the pixelated color image.

28. The method of claim 27 wherein the reclassifying an object into a different color class includes:

determining whether pixels of an object previously classified as black surpass a threshold regarding (HSB) values of the pixels; and

reclassifying the object into a different color class if the threshold is surpassed; and

the method further includes;

re-determining a value of the object based on the reclassification.

29. A system, comprising:

at least one image capturing device configured to acquire a pixelated color image of an area as a set of pixelated color image data; and

at least one object evaluation computing system having at least one processor and at least one storage medium that at least temporarily stores a the set of pixelated color image data, the object evaluation computing system configured to;

preprocess using the at least one processor the set of pixelated color image data for at least a portion of the area to produce a set of two color contour image data that represents at least the portion of the pixilated color image;

process using the at least one processor the set of two color contour image data to find a location of a bottom of a stack of objects in the set of two color contour image data and to find a location of a top of the stack of objects in the set of two color contour image data, if any objects are in the at least portion of the area, wherein the stack of objects includes at least one object if any objects are in the at least portion of the area;

determine using the at least one processor a total number of objects in the stack, if any based at least in part on a knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects;

process using the at least one processor a portion of the set of pixelated color image data based on the knowledge of the location of the bottom of the stack of objects and the location of the top of the stack of objects to respectively classify each object in the stack of objects into one of a defined set of color classes; and

determine using the at least one processor a cumulative total value of the stack of objects based at least in part on the color class to which each of the objects in the stack of objects is classified.

30. The system of claim 29, wherein the at least one object evaluation computing system is further configured to convert at least a portion of the set of the pixelated color image data to a set of gray scale image data and perform edge detection filtering of the set of gray scale image data to produce a set of contour image data.

31. The system of claim 29, wherein the at least one object evaluation computing system is further configured to logically segment the two color contour image data into a set of target pixel areas, each of the target pixel areas including a set of n by m pixels.

32. The system of claim 29 wherein the at least one object evaluation computing system is configured to preprocess using the at least one processor the set of pixelated color image data by:

comparing a set of pixilated color image data representing a pixelated reference image of the area to the set of two color contour image data, said reference image having no stack of objects; and

removing irregularities from the set of two color contour image data around a stack of objects in the two color contour image data based on the comparing.

33. The system of claim 32 wherein the removing irregularities comprises:

subtracting the set of pixilated color image data representing a pixelated reference image of the area from the set of two color contour image data; and

adding back to the set of two color contour image data an element of average background color of the area.

34. The system of claim 29 wherein the at least one object evaluation computing system is configured to preprocess using the at least one processor the set of pixelated color image data by:

color balancing a set of pixilated color image data representing a pixelated reference image of the area to have it more closely match the set of two color contour image data, said reference image having no stack of objects;

rotating the set of two color contour image data to compensate for camera distortion for one or more stacks of objects that are near edges of the two color contour image data;

aligning the set of pixilated color image data representing the pixelated reference image and the set of two color contour image data to account for movement of an image acquisition device used to acquire the pixelated color image;

subtracting the set of pixilated color image data representing a pixelated reference image of the area from the set of two color contour image data;

adding back to the set of two color contour image data image data representing an element of average background color of the area; and

either before the subtracting or after the subtracting, converting the set of pixilated color image data representing the pixelated reference image of the area and the set of two color contour image data to gray scale if the converting to gray scale had been determined likely to improve accuracy of the determining of the total number of objects in the stack.

35. The system of claim 29 wherein the at least one object evaluation computing system is configured to preprocess using the at least one processor the set of pixelated color image data by:

eliminating extraneous pixels from the set of two color contour image data that are outside the at least portion of the area; and

eliminating extraneous pixels that are not part of groups that at least approximately form the shape of a stack object based on a minimum threshold number of pixels.

36. The system of claim 35 the eliminating extraneous pixels that are not part of groups comprises:

passing a template shaped as a stack object over the set of two color contour image data;

counting a number of contour pixels within an area defined by the template for each location of the template on the set of two color contour image data; and

eliminating all pixels outside the area defined by the template at each location if the number of contour pixels within the area defined by the template contains the minimum threshold number of pixels.

37. The system of claim 29 wherein the at least one object evaluation computing system is further configured to:

before computationally determining using the at least one processor a total number of objects in the stack, determine a horizontal center of the stack of objects based on an overall concentration of contour pixels in the set of two color contour image data; and

determine an estimated height of individual objects in the stack of objects based on a mean/average height of an individual object of the stack rationalized for a distance between an image acquisition device used to acquire the pixelated color image and an expected location of the stack, said distance adjusted based on the location of the bottom of the stack at the determined horizontal center of the stack relative to the expected location of the stack of objects.

38. The system of claim 37, wherein the at least one object evaluation computing system is further configured to eliminate extraneous pixels from the set of two color contour image data by:

for each target pixel area of the set of target pixel areas, eliminating all contour pixels in the target pixel area from the set of two color contour image data if a total number of the contour pixels in the target pixel area is less that a threshold number of pixels per target pixel area; and

for each target pixel area of at least some of the set of target pixel areas, eliminating all contour pixels in the target quadrant from the set of two color contour image data if the target pixel area has less than a nearest neighboring threshold number of nearest neighboring target pixel areas.

39. The system of claim 37, wherein the at least one object evaluation computing system is further configured to:

apply a sample bounding box at successive intervals to groups of target pixel areas of the set of target pixel areas to find a location of the bottom the stack of objects, each of the groups of target pixel areas including at least some target pixel areas with contour pixels of the set of two color contour image data that are below a determined upper location in the set of two color contour image data;

determine a bottom quadrant to be the sample bounding box that includes the group of target pixel areas with the highest number of target pixel areas having contour pixels;

determine whether a width of the target pixel areas included in the bottom quadrant fits a face ellipse of an object;

determine whether a width of the target pixel areas included bottom quadrant fits a face ellipse of an object;

determine that there is no stack present when the width does not fit the face ellipse; and

determine a location of the bottom of the bottom quadrant when the width fits the face ellipse.

40. The system of claim 37, wherein the at least one object evaluation computing system is further configured to:

successively apply a sample bounding box to groups of target pixel areas of the set of target pixel areas at successive intervals from a determined bottom of the stack of objects until a top streak of target pixel areas contained within the sample bounding box is empty.

41. The system of claim 40, wherein the at least one object evaluation computing system is further configured to:

determine a top bounding box based on the values of the sample bounding box with the sample bounding box positioned such that the top streak of target quadrants contained within the sample bounding box is empty;

successively apply the sample bounding box at successive intervals to find a sample quadrant that fits an elliptical pattern;

determine a list of coordinates of the target quadrants that intersect a top arc and a bottom arc of an imaginary ellipse the size of the sample bounding box;

determine a total number of target quads that overlap the target quadrants of the imaginary ellipse from the sample bounding box; and

determine the sample bounding box position with the most number of overlaps as the top quadrants.

42. The system of claim 41, wherein the at least one object evaluation computing system is further configured to:

determine a total number of objects in the stack by determining a difference in the bottom of the top quadrant and a bottom of a bottom quadrant and a mean/average height of an individual object rationalized for a distance between the image acquisition device and the stack of objects.

43. The system of claim 29, wherein the at least one object evaluation computing system is further configured to:

classify for each of the objects a number of pixels in a color pixel streak to respective hue, saturation, and brightness (HSB) values that represent the hue, saturation and brightness of the number of pixels, the color pixel streak composed of a subset of the pixels between a top arc and a bottom arc of an ellipse of the object.

44. The system of claim 43, wherein the at least one object evaluation computing system is further configured to:

classify for each of the objects the number of pixels in the color pixel streak to respective fundamental colors based on defined relationships between HSB values and a defined set of fundamental colors.

45. The system of claim 44, wherein the at least one object evaluation computing system is further configured to:

determine for each of the objects whether the classified fundamental colors of the color pixel streak is in a set of color drift colors.

46. The system of claim 43, wherein the at least one object evaluation computing system is further configured to:

determine for each of the objects a value of the object based on a determined fundamental color and a defined set of fundamental color to object value relationships.

47. The system of claim 29 wherein the at least one object evaluation computing system is further configured to:

reclassify an object into a different color class based on a determination that a color of the object is obscured by a shadow in the pixelated color image.

48. The system of claim 47 wherein the at least one object evaluation computing system is configured to reclassify an object by:

determining whether pixels of an object previously classified as black surpass a threshold regarding (HSB) values of the pixels; and

reclassifying the object into a different color class if the threshold is surpassed; and

the at least one object evaluation computing system may be further configured to;

re-determine a value of the object based on the reclassification.