Setting different background model sensitivities by user defined regions and background filters

• US 10,366,509 B2
• Filed: 03/29/2016
• Issued: 07/30/2019
• Est. Priority Date: 03/31/2015
• Status: Active Grant

• Current Assignee: Thermal Imaging Radar LLC
• Original Assignee: Thermal Imaging Radar LLC
• Inventors: Jones, Lawrence Richard, Lembke, Bryce Hayden, Sablak, Sezai, Dortch, Michael D., Price, Larry J.
• Primary Examiner(s): Woldemariam, Aklilu K
• Application Number:
  US15/562,798
• Publication Number:
  US 20180286075A1
• Time in Patent Office: 1,218 Days
• Field of Search: 348239, 348 77, 348135, 348142, 348143, 348148, 382103, 382131, 382190, 382266
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Creating a background model for image processing to identify new foreground objects in successive video frames. A method includes providing a background image in a user interface. The method further includes receiving a first user input in the user interface that comprises an identification of one or more different regions within the background image. The method further includes receiving a second user input in the user interface that comprises a selection of an image change tolerance for each of the identified different regions. The method further includes providing the background image, information identifying the different regions, and the image change tolerances to an image processor. The background image, the information identifying the different regions, and the image change tolerances are used by the image processor to create a background model to thereby compare a successive image with the background model in order to identify foreground objects within the successive image.

Claims

1. A method of creating a background model for image processing to identify new foreground objects in successive video frames, the method comprising:

providing a background image in a user interface;

receiving a first user input in the user interface, the first user input comprising identifying one or more different regions within the background image;

receiving a second user input in the user interface, the second user input comprising selecting an image change tolerance for each of the one or more identified different regions, wherein the image change tolerance for each of the one or more identified different regions defines a threshold amount of change that is required to be detected as between (i) the one or more identified different regions within the background image and (ii) one or more corresponding identified different regions within a successive image in order for one or more objects within the successive image to be classified as one or more foreground objects;

providing the background image, information identifying the one or more different regions, and the image change tolerances to an image processor, wherein the background image, the information identifying the one or more different regions, and the image change tolerances are used by the image processor to create a background model to thereby compare the successive image with the background model in order to identify the one or more foreground objects within the successive image; and

in response to determining that a particular foreground object included in the one or more foreground objects is an object of interest, sending an alert that includes information describing the object of interest, wherein the information includes a confidence level indicating a computed certainty that the object of interest is a foreground object within the successive image as opposed to being a background object within the successive image such that the confidence level provided by the alert facilitates identification and classification of the object of interest.

2. The method of claim 1, wherein the background image is a 360 degree panorama image.

3. The method of claim 1, wherein receiving the first user input comprises receiving a drag and select user input that identifies a region boundary.

4. The method of claim 1, wherein the information identifying the one or more different regions comprises Cartesian coordinates derived from the user input identifying the one or more different regions within the background image.

5. The method of claim 1, wherein identifying one or more foreground objects within the successive image comprises determining that a change from the background image to the successive image in a region exceeds an image change tolerance for the region such that the foreground object is identified in the region when a score for the foreground object exceeds the image change tolerance for the region in which the foreground object is located.

6. The method of claim 1, wherein the image processor further applies one or more filters to the foreground object to filter out certain foreground objects to identify one or more foreground objects of interest in the successive video frame.

7. The method of claim 6, wherein one of the one or more filters comprises an edge detection filter.

8. The method of claim 6, wherein one of the one or more filters comprises a confidence level filter.

9. A method of identifying a foreground object in a video frame by comparing the video frame to a background model, the method comprising:

obtaining a background model, wherein the background model comprises a background image and identifies one or more user defined regions, each of the one or more user defined regions including an image change tolerance, and wherein the image change tolerance for each of the one or more user defined regions defines a threshold amount of change that is required to be detected as between (i) the one or more user defined regions within the background image and (ii) one or more corresponding user defined regions within a successive image in order for one or more objects within the successive image to be classified as one or more foreground objects;

obtaining a video frame that includes the successive image;

evaluating the successive image against the background model such that a foreground object is identified in a region of the successive image when a score for the foreground object exceeds the image change tolerance for the region in which the foreground object is located; and

in response to determining that the foreground object is an object of interest, sending an alert that includes information describing the object of interest, wherein the information includes a confidence level indicating a computed certainty that the object of interest is a foreground object within the successive image as opposed to being a background object within the successive image such that the confidence level provided by the alert facilitates identification and classification of the object of interest.

10. The method of claim 9, wherein the background image is a 360 degree panorama image.

11. The method of claim 9, wherein the score comprises an amount of change between pixels when comparing the background model to the video frame containing the foreground object.

12. The method of claim 9, further comprising using one or more filters to identify foreground objects.

13. The method of claim 9, wherein one of the one or more filters comprises an edge detection filter.

14. A method, implemented at a computer system, of identifying foreground objects of interest in a video frame, the method comprising:

obtaining a background model, wherein the background model comprises a background image and a user defined region, wherein the user defined region is associated with an image change tolerance, and wherein the image change tolerance defines a threshold amount of change that is required to be detected as between (i) the user defined region within the background image and (ii) a corresponding user defined region within a successive image in order for one or more objects within the successive image to be classified as one or more foreground objects;

obtaining a video frame that includes the successive image;

evaluating the successive image against the background model to identify objects, including identifying differences between the successive image and the background image;

applying one or more filters to the identified differences to identify a foreground object of interest in the successive image; and

in response to identifying the foreground object of interest, sending an alert that includes information describing the object of interest, wherein the information includes a confidence level indicating a computed certainty that the object of interest is a foreground object within the successive image as opposed to being a background object within the successive image such that the confidence level provided by the alert facilitates identification and classification of the object of interest.

15. The method of claim 14, wherein the background image and video frame are 360 degree panoramic images.

16. The method of claim 14, wherein the one or more filters includes an aspect ratio filter that filters out objects based on their aspect ratios.

17. The method of claim 14, wherein the one or more filters includes a correlation filter that filters out an object based on the object'"'"'s similarity to a background model object that is contained within the background image.

18. The method of claim 14, wherein the one or more filters includes an edge detection filter that filters out one or more objects based on a number of edges that the one or more objects have.

19. The method of claim 14, wherein the one or more filters includes a size filter that filters out one or more objects based on corresponding sizes of the one or more objects.

20. The method of claim 14, wherein the one or more filters includes a thermal filter that filters out one or more objects based on their thermal characteristics.