Systems and methods for boil detection

• US 9,330,469 B2
• Filed: 04/08/2014
• Issued: 05/03/2016
• Est. Priority Date: 04/08/2014
• Status: Active Grant

• Current Assignee: Haier US Appliance Solutions, Inc. DBA GE Appliances (Haier Group Company)
• Original Assignee: General Electric Company
• Inventors: Guan, Li
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US14/247,381
• Publication Number:
  US 20150286891A1
• Time in Patent Office: 756 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: F24C 7/083 on tops, hot plates
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30128 Food products
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters

Abstract

Systems and methods for boil detection are provided. One example system includes a vision sensor positioned to collect imagery depicting the cooktop and one or more processors implementing instructions to perform operations. The operations include obtaining a first frame of imagery and a second frame of imagery from the vision sensor. The first frame of imagery and the second frame of imagery respectively depict the cooktop. The operations include identifying a plurality of motion vectors based on the first and second frames of imagery. The operations include generating a histogram describing a characteristic of the plurality of motion vectors. The operations include calculating a dissimilarity score. The dissimilarity score describes a difference between the histogram and a pre-learned histogram. The pre-learned histogram describes the characteristic for imagery depicting boiling liquid. The operations include determining whether a boiling event is occurring at the cooktop based at least in part on the dissimilarity score.

Claims

1. A system for detecting a boiling event at a cooktop, the system comprising:

a vision sensor positioned so as to collect imagery depicting the cooktop;

one or more processors; and

one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising;

obtaining a first frame of imagery and a second frame of imagery from the vision sensor, wherein the first frame of imagery and the second frame of imagery respectively depict the cooktop;

identifying a first plurality of motion vectors based on the first and second frames of imagery, wherein each of the first plurality of motion vectors describes a change in position of one or more pixels from the first frame of imagery to the second frame of imagery;

generating a motion vector orientation histogram that describes a first distribution of orientations associated with the first plurality of motion vectors;

calculating a first dissimilarity score for the motion vector orientation histogram, wherein the first dissimilarity score describes a first difference between the motion vector orientation histogram and a first pre-learned histogram, wherein the first pre-learned histogram describes a second distribution of orientations associated with a second plurality of motion vectors identified from imagery depicting boiling liquid, and wherein the second distribution of orientations indicates that the second plurality of motion vectors identified from the imagery depicting the boiling liquid have orientations distributed in a relatively uniform manner;

generating a motion vector magnitude histogram that describes a first distribution of magnitudes associated with the first plurality of motion vectors;

calculating a second dissimilarity score for the motion vector magnitude histogram, wherein the second dissimilarity score describes a second difference between the motion vector magnitude histogram and a second pre-learned histogram, wherein the second pre-learned histogram describes a second distribution of magnitudes associated with the second plurality of motion vectors identified from the imagery depicting the boiling liquid, and wherein the second distribution of magnitudes indicates that a relatively large number of the second plurality of motion vectors identified from the imagery depicting the boiling liquid are clustered about a particular magnitude having a relatively small value; and

determining whether a boiling event is occurring at the cooktop based at least in part on the first dissimilarity score and based at least in part on the second dissimilarity score.

2. The system of claim 1, wherein determining whether a boiling event is occurring at the cooktop based at least in part on the first dissimilarity score and based at least in part on the second dissimilarity score comprises:

incrementing a counter value when each of the first dissimilarity score and the second dissimilarity score is less than a respective threshold dissimilarity value; and

detecting the boiling event at the cooktop when the counter value exceeds a threshold counter value.

3. The system of claim 2, wherein determining whether a boiling event is occurring at the cooktop based at least in part on the first dissimilarity score and based at least in part on the second dissimilarity score further comprises clearing the counter value to zero when each of the first dissimilarity score and the second dissimilarity is greater than the respective threshold dissimilarity value.

4. The system of claim 1, wherein the vision sensor comprises a wide angle VGA resolution camera.

5. The system of claim 1, wherein the one or more processors and the one or more non-transitory computer-readable media comprise a single System-On-Chip.

6. The system of claim 1, wherein the first pre-learned histogram and the second pre-learned histogram are stored in the one or more non-transitory computer-readable media.

12. The system of claim 1, wherein the vision sensor captures frames of images at between 0.3 to 1 frames per second.

13. The system of claim 5, wherein the single System-On-Chip and the vision sensor are located on a same single circuit board.

14. The system of claim 1, wherein at least one of the motion vector orientation histogram and the motion vector magnitude histogram provides a continuous density function with kernel methods.

15. The system of claim 1, wherein calculating the first dissimilarity score comprises performing at least one of an earth mover'"'"'s distance (EMD) technique, a bin-to-bin comparison technique, an L-1 distance technique, an L-2 distance technique, a Wasserstein metric technique, a Kolmogorov-Smirnov test, an f-divergence technique, a total variation distance technique, and an absolute difference value technique.

16. The system of claim 1, wherein at least one of the first dissimilarity score and the second dissimilarity score comprises a weighted sum of a plurality of distance values respectively obtained using a plurality of histogram comparison techniques.

7. A method for detecting a boiling event at a cooktop, the method comprising:

collecting, by a vision sensor, a plurality of frames comprising imagery depicting the cooktop;

determining, by one or more computing devices, a plurality of motion vectors for each of the plurality of frames, wherein each motion vector for each frame describes a change in position of one or more pixels included in such frame with respect to a previous sequential frame of the plurality of frames;

generating, by the one or more computing devices, a motion vector orientation histogram for each of the plurality of frames, wherein the motion vector orientation histogram for each frame describes a first distribution of orientations associated with the plurality of motion vectors determined for such frame;

calculating, by the one or more computing devices a first dissimilarity score for the motion vector orientation histogram of each frame, wherein the first dissimilarity score for each frame describes a first difference between the motion vector orientation histogram for such frame and a first pre-learned histogram, wherein the first pre-learned histogram describes a second distribution of orientation associated with a second plurality of motion vectors identified from imagery depicting a boiling event, and wherein the second distribution of orientations indicates that the second plurality of motion vectors identified from the imagery depicting the event have orientations distributed in a uniform manner;

generating, by the one or more computing devices, a motion vector magnitude histogram for each of the plurality of frames, wherein the motion vector magnitude histogram for each frame describes a first distribution of magnitudes associated with the plurality of motion vectors determined for such frame;

calculating, by the one or more computing devices, a second dissimilarity score for the motion vector magnitude histogram of each frame, wherein the second dissimilarity score for each frame and a second difference between the motion vector magnitude histogram for such frame and a second pre-learned histogram, wherein the second pre-learned histogram describes a second distribution of magnitudes associated with the second plurality of motion vectors identified from the imagery depicting the boiling event, and wherein the second distribution of magnitudes indicates that a number of the second plurality of motion vectors identified from the imagery depicting the boiling event are clustered about a particular magnitude having a small value; and

detecting, by the one or more computing devices, a boiling event at the cooktop based on the first dissimilarity score and the second dissimilarity score calculated for at least one of the plurality of frames.

8. The method of claim 7, wherein detecting, by the one or more computing devices, the boiling event at the cooktop based at least in part on the first dissimilarity score and the second dissimilarity score for at least one the plurality of frames comprises:

determining, by the one or more computing devices, a number of consecutive frames for which the dissimilarity score is less than a threshold similarity value; and

when the number of consecutive frames exceeds a threshold counter value, detecting, by the one or more computing devices, the boiling event.

9. The method of claim 7, further comprising providing an alarm when the boiling event is detected.

17. The method of claim 9, wherein providing an alarm when the boiling event is detected comprises communicating with a mobile device of a user.

18. The method of claim 9, wherein providing an alarm when the boiling event is detected comprises communicating with the cooktop.

10. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations, the operations comprising:

obtaining a plurality of frames of imagery from a vision sensor, wherein the plurality of frames of imagery respectively depict a cooktop, and wherein the plurality of frames of imagery form a plurality of pairs of consecutive frames;

identifying a plurality of motion vectors for each pair of consecutive frames of imagery;

generating, for each pair of consecutive frames of imagery, a motion vector orientation histogram that describes a first distribution of orientations associated with the plurality of motion vectors determined for such pair of consecutive frames;

calculating, for each pair of consecutive frames of imagery, a first dissimilarity score for the corresponding motion vector orientation histogram, wherein the first dissimilarity score for each consecutive pair of frames describes a first difference between the motion vector orientation histogram for such pair of consecutive frames and a first pre-learned histogram, wherein the first pre-learned histogram describes a second distribution of orientations associated with a second plurality of motion vectors identified from imagery depicting a boiling event, and wherein the second distribution of orientations indicates that the second plurality of motion vectors identified from the imagery depicting the boiling event have orientations distributed in a uniform manner;

generating, for each pair of consecutive frames of imagery, a motion vector magnitude histogram, wherein the motion vector magnitude histogram for each pair of consecutive frames describes a first distribution of magnitudes associated with the plurality of motion vectors determined for such pair of consecutive frames;

calculating, for each pair of consecutive frames of imagery, a second dissimilarity score for the corresponding motion vector magnitude histogram, wherein the second dissimilarity score for each pair of consecutive frames describes a second difference between the motion vector magnitude histogram for such pair of consecutive frames and a second pre-learned histogram, wherein the second pre-learned histogram describes a second distribution of magnitudes associated with the second plurality of motion vectors identified from the imagery depicting the boiling event, and wherein the second distribution of magnitudes indicates that a number of the second plurality of motion vectors identified from the imagery depicting the boiling event are clustered about a particular magnitude having a small value; and

determining whether a boiling event is occurring at the cooktop based at least in part on the dissimilarity scores respectively calculated for the plurality of pairs of consecutive frames.

11. The non-transitory computer-readable medium of claim 10, wherein determining whether a boiling event is occurring at the cooktop based at least in part on the dissimilarity scores respectively calculated for the plurality of pairs of consecutive frames comprises:

determining a number of consecutive pairs for Which both the first dissimilarity score and the second dissimilarity score are less than a respective threshold similarity value; and

detecting the boiling event when the number of consecutive pairs exceeds a threshold counter value.