Size recognition system with method for determining price of a commodity

• US 6,434,257 B1
• Filed: 07/09/2001
• Issued: 08/13/2002
• Est. Priority Date: 02/17/1995
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Connell, Jonathan Hudson, Mohan, Rakesh, Bolle, Rudolf Maarten
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Bali, Vikkram
• Application Number:
  US09/900,139
• Time in Patent Office: 400 Days
• Field of Search: 382/110, 382/100, 382/141-143, 382/152, 382/199, 382/170, 382/286, 705/16, 705/400, 209/538, 399/1, 348/86, 348/89, 348/91, 377/14, 178/18.01
• US Class Current: 382/110
• CPC Class Codes: G01B 11/024 by means of diode-array scanning
• CPC Class Codes: G01G 19/4144 for controlling weight of goods in commercial establishments, e.g. supermarket, P.O.S. systems
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30128 Food products
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/64 of convexity or concavity
• CPC Class Codes: G07G 1/0054 with control of supplementary check-parameters, e.g. weight or number of articles

Abstract

The present system and apparatus use image processing to recognize object size with a scene. The system includes novel image processing apparatus and method to segment one or more object images from a background image of the scene. A processed image (that can be used to characterize size features) of the object(s) is then compared to stored reference images. The object size is recognized when a match occurs. The system can recognize object sizes independent of number of objects present and the objects may be touching each other and overlapping. The system can be trained to recognize object sizes that it was not originally programmed to recognize.

Claims

1. A method for determining price of a commodity, said method comprising the of:

scanning at least one target object, said target objects being commodities of a same variety, or type, wherein a visual input device is used for receiving a scanned scene image, the scene image including the image of the at least one target object and a background image;

determining size characteristics associated with said at least one target object;

calculating, by an automated system, a price for said at least one target object, said calculation using the determined size characteristics and priced data to determine a calculated price; and

providing a user with the calculated price, said calculated price is displayed on an interactive output device, wherein

said user interacts with the interactive output device to assist in decision making or to train the automated system to recognize new target object characteristics, such as size, or to input price data associated with the target object commodity type, and wherein

the step of determining size characteristics associated with said at least one target object further comprises the steps of

determining a boundary of an image of a target object, the boundary being a plurality of transition points between the target object image and a background image;

determining a plurality of intermediate sizes of the target object, each of said intermediate sizes determined from three of said transition points on the boundary of the target object, and an image point of the target object, and each of said intermediate sizes being along a substantially same direction with respect to a predetermined scan direction, and

determining a size of the target object from said plurality of determined intermediate sizes.

2. The method as recited in claim 1, wherein the target object is of a roughly spherical shape.

3. The method as recited in claim 1, wherein the interactive output device displays the determined target object size characteristics.

4. The method as recited in claim 1, further comprising the step of determining weight characteristics of the at least one target object, wherein the weight characteristics are also used with the size characteristics in the calculating step to determine a calculated price.

5. The method as recited in claim 1, wherein the user is a check-out clerk at a retail store, the commodity is produce, such as apples, pears, lemons, or limes, and the automated system for calculating price interfaces with a cash register, or other payment or inventory keeping system as is used by a check-out clerk.

6. The method as recited in claim 4, wherein the weight characteristics of the at least one target object are determined by weighing the at least one target object.

7. The method as recited in claim 1, wherein the user is a check-out clerk at a retail store, the commodity is produce, such as apples, pears, lemons, or limes, and the automated system for calculating price interfaces with a cash register or other payment or inventory keeping system as is used by a checkout clerk.

8. The method as recited in claim 1, wherein the price data is prestored by a user.

9. The method as recited in claim 1, wherein the step of calculating, by an automated system, further comprises the steps of:

determining the weight of the at least one target object from the size characteristics and volume function of the target object, wherein the size characteristics include information about density, i.e., weight per volume, and

calculating the price by multiplying the determined weight by price per weight of the at least one target object.

10. The method as recited in claim 9, wherein the step of calculating, by an automated system, further comprises the step of calculating a price by multiplying price per unit by a determined number of units of the at least one target object.

11. The method as recited in claim 10, wherein the determined number of units is input by a human operator.

12. The method as recited in claim 11, wherein the human operator inputs information by using a method selected from the group of touch, voice, mouse and keyboard entry.

13. The method as recited in claim 10, wherein the determined number of units is calculated automatically.

14. A method for determining price of a commodity, said method comprising the steps of:

scanning at least one target object, said target objects being commodities of a same variety, or type, wherein a visual input device is used for receiving a scanned scene image, the scene image including the image of the at least one target object and a background image;

determining size characteristics associated with said at least one target object;

calculating, by an automated system, a price for said at least one target object, said calculation using the determined size characteristics and price data to determine a calculated price; and

providing a user with the calculated price, wherein

the step of determining size characteristics associated with said at least one target object further comprises the steps of;

determining a boundary of an image of a target object, the boundary being a plurality of transition points between the target object image and a background image;

determining a plurality of intermediate sizes of the target object, each of said intermediate sizes determined from three of said transition points on the boundary of the target object, and an image point of the target object, and each of said intermediate sizes being along a substantially same direction with respect to a predetermined scan direction; and

determining a size of the target object from said plurality of the determined intermediate sizes.

15. The method as recited in claim 14, wherein the step of determining a size further comprises the step of determining an average of a plurality of said intermediate sizes and wherein the size is determined from the average.

16. The method as recited in claim 14, wherein the step of determining a size further comprises the step of for determining a median of a plurality of said intermediate sizes and wherein the size is determined from the median.

17. The method recited in claim 14, wherein the target object is of a roughly spherical shape.

18. The method as recited in claim 14, further comprising the step of determining weight characteristics of the at least one target object, wherein the weight characteristics are also used with the size characteristics in the calculating step to determine a calculated price.

19. The method as recited in claim 18, wherein the user is a check-out clerk at a retail store, the commodity is produce, such as apples, pears, lemons, or limes, and the automated system for calculating price interfaces with a cash register, or other payment or inventory keeping system as is used by a check-out clerk.

20. The method as recited in claim 18, wherein the weight characteristics of the at lease one target object are determined by weighing the at least one target object.

21. The method as recited in claim 14, wherein tie user is a check-out clerk at a retail store, the commodity is produce, such as apples, pears, lemons, or limes, and the automated system for calculating price interfaces with a cash register, or other payment or inventory keeping system as is used by a check-out clerk.

22. The method as recited in claim 14, wherein the price data is prestored by a user.

23. The method as recited in claim 14, wherein the step of calculating, by an automated system, further comprises the steps of;

determining the weight of the at least one target object from the size characteristics and volume function of the target object, wherein the size characteristics including information about density, i.e., weight per volume; and

calculating the price by multiplying the determined weight by price per weight of the at least one target object.

24. The method as recited in claim 23, wherein the step of calculating, by an automated system, further comprises the step of calculating a price by multiplying price per unit by a determined number of unites of the at least one target object.

25. The method as recited in claim 24, wherein the determined number of units is input by a human operator.

26. The method as recited in claim 25, wherein the human operator inputs information by using a method selected from the group of touch, voice, mouse and keyboard entry.

27. The method as recited in claim 24, wherein the determined number of units is calculated automatically.

28. A method for determining price of a commodity, said method comprising the steps of:

scanning at least one target object, said target objects being commodities of a same variety, or type, wherein a visual input device is used for receiving a scanned scene image, the scene image including the image of the at least one target object and a background image;

determining size characteristics associated with said at least one target object;

calculating, by an automated system, a price for said at least one target object, said calculation using the determined size characteristics and price data to determine a calculated price; and

providing a user with the calculated price, wherein

the step of determining size characteristics associated with said at least one target object also determines a size of a target object having a boundary, the boundary having a radius of curvature, and further comprises the steps of;

determining a boundary of the image of a target object, the boundary being a plurality of transition points between a target object image and the background image;

determining a plurality of intermediate sizes corresponding to the target object, each intermediate size determined from three points on the boundary, and each of the intermediate sizes being along a same direction with respect to a predetermined scan direction;

storing a plurality of predetermined reference object size characterizations;

comparing the intermediate sizes with at least one of said stored reference object size characterizations and generating a result of said comparing; and

determining a size of the target object based on said result.

29. A method as recited in claim 28, further comprising the step of selecting a plurality of sets of three points on the object boundary, wherein said the step of comparing compares said selected sets with said reference object size characterizations, and

wherein said the step of determining a size of the target object determines the size based on results from said comparing step.

30. A method as recited in claim 29, wherein the intermediate size represents a determined diameter of said target object and the reference size represents a reference diameter.

31. A method as recited in claim 30, wherein the step of determining a size further comprises the steps of:

determining a chord between a first and second point on the image boundary;

selecting a plurality of reference points on the chord;

determining a first distance, a, for each of said reference points, defined as being the distance between the first point and the reference point

determining a second distance, b, for each of said reference points, defined as being the distance between the second point and the reference point;

calculating a third distance c, for each of said reference points, being the distance between from the reference point orthogonal to a point on the boundary;

determining a plurality of intermediate target diameters, D, by the equation;

32. The method recited in claim 28, wherein the target object is of a roughly spherical shape.

33. The method as recited in claim 28, further comprising the step of determining weight characteristics of the at least one target object, wherein the weight characteristics are also used with the size characteristics in the calculating step to determine a calculated price.

34. The method as recited in claim 33, wherein the user is a check-out clerk at a retail store, the commodity is produce, such as apples, pears, lemons, or limes, and the automated system for calculating price interfaces with a cash register, or other payment or inventory keeping system as is used by a check-out clerk.

35. The method as recited in claim 33, wherein the weight characteristics of the at lease one target object are determined by weighing the at least one target object.

36. The method as recited in claim 28, wherein the user is a check-out clerk at a retail store, the commodity is produce, such as apples, pears, lemons, or limes and the automated system for calculating price interfaces with a cash register, or other payment or inventory keeping system as is used by a check-out clerk.

37. The method as recited in claim 28, wherein the price data is prestored by a user.

38. The method as recited in claim 28, wherein the step of calculating, by an automated system, further comprises the steps of:

determining the weight of the at least one target object from the size characteristics and volume function of the target object, wherein the size characteristics including information about density, i.e., weight per volume; and

calculating the price by multiplying the determined weight by price per weight of the at least one target object.

39. The method as recited in claim 38, wherein the step of calculating, by an, automated system, further comprises the step of calculating a price by multiplying price per unit by a determined number of unites of the at least one target object.

40. The method as recited in claim 38, wherein the determined number of units is input by a human operator.

41. The method as recited in claim 40, wherein the human operator inputs information by using a method selected from the group of touch, voice, mouse and keyboard entry.

42. The method as recited in claim 38, wherein the determined number of units is calculated automatically.