Vision-enabled household appliances

• US 20050196046A1
• Filed: 01/20/2005
• Published: 09/08/2005
• Est. Priority Date: 01/30/2004
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: Evolution Robotics Inc. (iRobot Corporation)
• Inventors: Gross, William, Hudnut, Alec
• Granted Patent: US 7,903,838 B2
• Time in Patent Office: Days
• US Class Current: 382/218
• CPC Class Codes: A47J 37/00 Baking; Roasting; Grilling; Frying bakers' ovens, non-domestic baking apparatus or equipment A21B; domestic stoves or ranges F24B, F24C
• CPC Class Codes: A47L 15/0063 using remote monitoring or controlling of the dishwasher operation, e.g. networking systems
• CPC Class Codes: A47L 15/4293 Arrangements for programme selection, e.g. control panels; Indication of the selected programme, programme progress or other parameters of the programme, e.g. by using display panels
• CPC Class Codes: A47L 2401/03 Operation mode, e.g. delicate washing, economy washing, reduced time, sterilizing, water softener regenerating, odor eliminating or service
• CPC Class Codes: A47L 2401/04 Crockery or tableware details, e.g. material, quantity, condition
• CPC Class Codes: A47L 2401/34 Other automatic detections
• CPC Class Codes: A47L 2501/26 Indication or alarm to the controlling device or to the user
• CPC Class Codes: A47L 2501/30 Regulation of machine operational steps within the washing process, e.g. performing an additional rinsing phase, shortening or stopping of the drying phase, washing at decreased noise operation conditions
• CPC Class Codes: A47L 2501/34 Change machine operation from normal operational mode into special mode, e.g. service mode, resin regeneration mode, sterilizing mode, steam mode, odour eliminating mode or special cleaning mode to clean the hydraulic circuit
• CPC Class Codes: D06F 2103/02 Characteristics of laundry or load
• CPC Class Codes: D06F 2105/10 Temperature of washing liquids; Heating means therefor
• CPC Class Codes: D06F 2105/58 Indications or alarms to the control system or to the user
• CPC Class Codes: D06F 2105/60 Audible signals
• CPC Class Codes: F25D 2400/361 Interactive visual displays

Abstract

The invention is generally related to the systems and methods through which household appliances can provide intuitive and enhanced interactivity through visual inputs. In particular, the disclosure presents methods relating to control of a household appliance, such as a microwave or a refrigerator, via recognition of visual indicia, such as graphics, text, and the like, using a visual sensor such as a camera.

Claims

1. A method of automating an appliance through a sensing device, the method comprising:

receiving a visual image;

determining that one or more features in the visual image match with one or more features from at least one previously identified household object, where the determination of the matching one or more features is performed by a visually-based automated process;

analyzing one or more quality metrics associated with the match between the one or more features of the visual image and the at least one previously identified household object; and

automatically determining that the match is potentially unreliable at least partly based on the analysis of the one or more quality metrics.

2. The method as defined in claim 1, wherein automatically determining that the match is potentially unreliable further comprises determining whether to automate the appliance at least partly based on the one or more quality metrics.

3. The method as defined in claim 1, wherein the method is used by a household appliance for a household duty.

4. The method as defined in claim 1, wherein determining that one or more features in the image match further comprises receiving an indicia from a training database that one or more features in the visual image match.

5. The method as defined in claim 1, wherein analyzing one or more quality metrics further comprises:

receiving an indication related to a quality of the match; and

comparing the indication to a predetermined threshold.

6. The method as defined in claim 5, wherein the indication relates to a count of a number of features that match between the visual image and the previously identified household object, and wherein automatically determining that the match is potentially unreliable further comprises determining that the match is potentially unreliable when the number of features that match does not exceed a predetermined number.

7. A system for automating an appliance, the system comprising:

a sensor configured to receive a visual image;

a circuit configured to determine that one or more features in the visual image match with one or more features from at least one previously identified household object, where the determination of the matching one or more features is performed by a visually-based automated process;

a circuit configured to analyze one or more quality metrics associated with the match between the one or more features of the visual image and the at least one previously identified household object; and

a circuit configured to automatically determine that the match is potentially unreliable at least partly based on the analysis of the one or more quality metrics.

8. The system as defined in claim 7, wherein the circuit configured to automatically determine that the match is potentially unreliable is further configured to determine whether to use the match to automate the appliance at least partly based on the one or more quality metrics.

9. The system as defined in claim 77, wherein the circuit is embodied in a household appliance for a household duty.

10. A computer program embodied in a tangible medium for identifying a potentially unreliable match for automating an appliance, the computer program comprising:

a module with instructions configured to determine that one or more features in a visual image match with one or more features from at least one previously identified household object, where the determination of the matching one or more features is performed by a visually-based automated process;

a module with instructions configured to analyze one or more quality metrics associated with the match between the one or more features of the visual image and the at least one previously identified household object; and

a module with instructions configured to automatically determine that the match is potentially unreliable at least partly based on the analysis of the one or more quality metrics.

11. The computer program as defined in claim 10, wherein the module with instructions configured to automatically determine that the match is potentially unreliable further comprises instructions configured to determine whether to use the match to automate the appliance at least partly based on the one or more quality metrics.

12. The computer program as defined in claim 10, wherein the computer program is embodied in a household appliance for a household duty.

13. A method of filtering potentially unreliable measurements from automating an appliance, the method comprising:

receiving an indication that a visual image matches with features observed in a previously identified household object, where the visual image is provided by a visual sensor;

computing a measure of a quality of a measurement;

comparing the measure of the quality of the visual image to at least one predetermined criterion to determine at least one of two states, where a first state that indicates a higher quality measurement than a second state;

avoiding use of the measurement for automating the appliance at least partly in response to a determination of the second state indicating the lower quality match; and

using the measurement for automating the appliance at least partly in response to a determination of the first state indicating the higher quality match.

14. The method as defined in claim 13, further comprising using the appliance for a household duty.

15. The method as defined in claim 13, wherein the measure of the quality corresponds to a count of a number of features that match between the visual image and the previously identified household object, and wherein the second state that indicates the lower quality is determined at least partly when the number of features that match does not exceed a predetermined number.

16. The method as defined in claim 13, further comprising executing an instruction in the appliance, wherein the appliance is configured to execute the instruction, wherein the measure of the quality corresponds to an analysis result of a quality matrix, wherein the quality matrix corresponds to a measurement of match quality, and wherein the second state that indicates the lower quality is determined at least partly when the match exceeds a predetermined number.

17. A circuit for filtering potentially unreliable measurements from automating an appliance, the method comprising:

a circuit configured to receive an indication that a visual image matches with features observed in a previously identified household object, where the visual image is provided by a visual sensor;

a circuit configured to compute a measure of a quality of a measurement;

a circuit configured to compare the measure of the quality of the measurement to at least one predetermined criterion to determine at least one of two states, where a first state that indicates a higher quality measurement than a second state;

a circuit configured to avoid use of the measurement for automating the appliance at least partly in response to a determination of the second state indicating the lower quality match; and

a circuit configured to use the measurement for automating the appliance at least partly in response to a determination of the first state indicating the higher quality match.

18. The circuit as defined in claim 17, wherein the circuit is embodied in a household appliance for a household duty.

19. The circuit as defined in claim 17, wherein the measure of the quality corresponds to a count of a number of features that match between the image and the previously identified household object, and wherein the second state that indicates the lower quality is determined at least partly when the number of features that match does not exceed a predetermined number.

20. The circuit as defined in claim 17, further comprising a circuit configured to execute an instruction in the appliance, wherein the appliance is configured to execute an instruction, wherein the measure of the quality corresponds to a analysis of a match, wherein the analysis corresponds to a measurement of number of matched features, and wherein the second state that indicates the lower quality is determined at least partly when the number of matches exceeds a predetermined number.

21. A computer program embodied in a tangible medium for filtering potentially unreliable measurements from automating an appliance, the computer program comprising:

a module with instructions configured to receive an indication that a previously identified household object matches with features observed in an image, where the image is provided by a visual sensor;

a module with instructions configured to compute a measure of a quality of a measurement;

a module with instructions configured to compare the measure of the quality of the measurement to at least one predetermined criterion to determine at least one of two states, where a first state that indicates a higher quality measurement than a second state;

a module with instructions configured to avoid use of the measurement for automating the appliance at least partly in response to a determination of the second state indicating the lower quality match; and

a module with instructions configured to use the measurement for automating the appliance at least partly in response to a determination of the first state indicating the higher quality match.

22. The computer program as defined in claim 21, wherein the measure of the quality corresponds to a count of a number of features that match between the image and the previously identified household object, and wherein the second state that indicates the lower quality is determined at least partly when the number of features that match does not exceed a predetermined number.

23. The computer program as defined in claim 21, further comprising a module with instructions configured to automate the appliance, wherein the appliance is configured to automatically execute the instruction, wherein the measure of the quality corresponds to an analysis of a match, wherein the analysis corresponds to a measurement of matched features, and wherein the second state that indicates the lower quality is determined at least partly when the number of matched features exceeds a predetermined number.

24. A circuit for filtering potentially unreliable measurements from automating an appliance, the circuit comprising:

a means for receiving an indication that a previously identified household object matches with features observed in an image, where the image is provided by a visual sensor;

a means for computing a measure of a quality of a measurement;

a means for comparing the measure of the quality of the measurement to at least one predetermined criterion to determine at least one of two states, where a first state that indicates a higher quality measurement than a second state;

a means for avoiding use of the measurement for automating the appliance at least partly in response to a determination of the second state indicating the lower quality match; and

a means for using the measurement for automating household appliance at least partly in response to a determination of the first state indicating the higher quality match.

25. The circuit as defined in claim 24, wherein the circuit is embodied in a household appliance.

26. A method of filtering potentially unreliable measurements from automating an appliance, the method comprising:

receiving a list of one or more household objects that have been preliminarily identified as matching with features observed in an image, where the image is provided by a visual sensor coupled to a household appliance for which the automation applies;

computing a measure of a quality of a measurement corresponding to each household object in the list;

comparing the measure of the quality of each of the measurements to at least one predetermined criterion to determine at least one of two states, where a first state indicates a higher-quality measurement and a second state indicates a lower-quality measurement;

generating a modified list by selectively removing a measurement from the list of the one or more measurements at least partly in response to a determination that the measure of the quality of the measurement associated with the household object corresponds to the second state indicating the lower-quality measurement; and

providing the modified list for use in automation.

27. The method as defined in claim 26, further comprising using the automation for a household appliance.

28. The method as defined in claim 26, wherein selectively removing a measurement comprises providing an indication of whether a measurement in the list is to be used.

29. The method as defined in claim 26, wherein the measure of the quality corresponds to a count of a number of features that match between the image and the household object, and wherein the second state that indicates the lower-quality measurement is determined at least partly when the number of features that match does not exceed a predetermined number.

30. The method as defined in claim 26, further comprising automating a household appliance, wherein the household appliance is configured to execute an instruction, wherein the measure of the quality corresponds to an analysis of at least a matched feature, wherein the at least a matched feature corresponds to a number of matched features, and wherein the second state that indicates the lower-quality measurement is determined at least partly when the number of matched features exceeds a predetermined number.

31. A computer program embodied in a tangible medium for filtering potentially unreliable measurements from automating an appliance, the computer program comprising:

a module with instructions configured to receive a list of one or more household objects that have been preliminarily identified as matching with features observed in an image, where the image is provided by a visual sensor coupled to an household appliance for which the automation applies;

a module with instructions configured to compute a measure of a quality of a measurement corresponding to each household object in the list;

a module with instructions configured to compare the measure of the quality of each of the measurements to at least one predetermined criterion to determine at least one of two states, where a first state indicates a higher-quality measurement and a second state indicates a lower-quality measurement;

a module with instructions configured to generate a modified list by selectively removing a measurement from the list of the one or more measurements at least partly in response to a determination that the measure of the quality of the measurement associated with the household object corresponds to the second state indicating the lower-quality measurement; and

a module with instructions configured to provide the modified list for use in automating the appliance.