Method and apparatus for mapping a location from a video image to a map

• US 6,816,184 B1
• Filed: 04/15/1999
• Issued: 11/09/2004
• Est. Priority Date: 04/30/1998
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Texas Instruments, Inc.
• Inventors: Flinchbaugh, Bruce E., Brill, Frank Z.
• Primary Examiner(s): Kelly, Chris
• Assistant Examiner(s): SENFI, BEHROOZ M
• Application Number:
  US09/292,499
• Time in Patent Office: 2,035 Days
• Field of Search: 348/143, 348/147, 348/161, 348/169, 382/103, 382/209, 382/276, 382/277, 382/289, 382/291, 382/293, 382/294, 382/295, 382/305, 382/282, 345/629
• US Class Current: 348/143
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A monitoring system (10) includes a video camera (12) which generates images of a monitored area. A computer (16) receives the images from the video camera, and contains a digitized map (85) of the monitored area. One of the detected images (84) is saved as a reference image. An operator defines a first region (86) corresponding to a selected portion of the monitored area, as viewed in the reference image, and defines a second region (87) which corresponds to the selected portion of the area as viewed on the map. Subsequent images from the camera are compared to the reference image, in order to identify an object of interest, and a first point associated with the object is identified. If the first point is within the first region, a warp transformation of the first point from the first region to the second region is carried out, in order to identify within the second region a second point which corresponds to the first point, and which identifies the location on the map of the object of interest.

Claims

1. A method of monitoring, comprising the steps of:

providing a map of a monitored area;

detecting a reference image of the monitored area;

receiving operator input identifying a first region which corresponds to a selected portion of the monitored area as viewed in the reference image;

receiving operator input identifying a second region which corresponds to the selected portion as viewed on the map;

defining warp transformation parameters for translating a selected point in the first region into a corresponding point in the second region;

detecting a further image of the area;

identifying a portion of the further image corresponding to an object of interest;

selecting a first point near a lower end of the portion of the further image which corresponds to the object of interest, the first point being within the first region; and

carrying out a warp transformation of the first point from the first region to the second region according to the defined warp transform parameters in order to identify within the second region a second point which corresponds to the first point and which identifies a position on the map of the object of interest.

2. A method according to claim 1, including the step of providing on the map a visible indicator corresponding to the position of the object of interest.

3. A method according to claim 1, wherein said selecting step is carried out by determining a bounding box just large enough to contain the entire portion of the further image corresponding to the object of interest, and selecting as the first point a midpoint of a lower side of the bounding box.

4. A method according to claim 1, wherein:

said step of identifying a portion of the further image corresponding to the object of interest includes the step of comparing the further image to the reference image and identifying on a pixel-by-pixel basis the difference between the further image and the reference image.

5. A method according to claim 1, wherein the reference image and the further image are video images.

6. A method according to claim 1, including the steps of identifying a third region on the map, detecting whether the second point satisfies a predetermined condition with respect to the third region, and taking a predetermined action if the second point satisfies the predetermined condition with respect to the third region.

7. A method according to claim 1, wherein:

said step of identifying the first region includes;

generating a visual display of the reference image;

receiving operator input defining a plurality of first corners on the visual display of the reference image;

identifying the first region as a polygon having the plurality of first corners; and

generating a visual display having a first outline corresponding to the first region overlaid on the reference image; and

said step of identifying the second region includes;

generating a visual display of the map;

receiving operator input defining a plurality of second corners on the visual display of the map;

identifying the second region as a polygon having the plurality of second corners; and

generating a visual display having a second outline corresponding to the second region overlaid on the map.

8. A method according to claim 7, wherein:

said first outline consists of broken lines connecting said first corners; and

said second outline consists of broken lines connecting said second corners.

9. A method according to claim 7, wherein:

said first outline consists of solid lines connecting said first corners; and

said second outline consists of solid lines connecting said second corners.

10. A method according to claim 7, wherein:

said first outline consists of displaying said first region in a predetermined color; and

said second outline consists of displaying said second region in said predetermined color.

11. A method according to claim 7, wherein:

said step of identifying the first region further includes receiving operator input adjusting the first region; and

said step of identifying the second region further includes receiving operator input adjusting the second region.

12. A method according to claim 11, wherein:

said step of receiving operator input adjusting the first region includes receiving operator input via a pointing device to drag and drop a corner of the first region; and

said step of receiving operator input adjusting the second region includes receiving operator input via a pointing device to drag and drop a corner of the second region.

13. A method according to claim 11, wherein:

said step of receiving operator input adjusting the first region includes receiving operator input via a pointing device to drag and drop a side of the first region; and

said step of receiving operator input adjusting the second region includes receiving operator input via a pointing device to drag and drop a side of the second region.

14. A method of monitoring, comprising the steps of:

providing a map of a monitored area;

detecting a reference image of the area;

identifying a first region which corresponds to a first portion of the area as viewed in the reference image;

identifying a second region which corresponds to the first portion as viewed on the map;

identifying a third region which corresponds to a second portion of the area as viewed in the reference image, and which has a portion that overlaps a portion of the first region;

identifying a fourth region which corresponds to the second portion as viewed on the map;

selecting one of the first and third regions to have priority over the other thereof;

detecting a further image of the area;

identifying a portion of the further image corresponding to an object of interest;

selecting a first point near a lower end of the portion of the further image which corresponds to the object of interest;

determining whether the first point is within the selected one of the first and third regions, and carrying out a warp transformation of the first point from the selected one of the first and third regions to the corresponding one of the second and fourth regions if the first point is within the selected one of the first and third regions; and

determining whether the first point is outside the selected one of the first and third regions and within the other thereof, and carrying out a warp transformation of the first point from the nonselected one of the first and third regions to the corresponding one of the second and fourth regions.

15. A method according to claim 14, wherein said step of defining priority includes the step of automatically selecting the smaller of the first and third regions to have priority over the larger thereof.

16. A method according to claim 14, including the step of providing on the map a visible indicator which corresponds to the position of the object of interest.

17. A method according to claim 14, wherein said steps of generating the first, second, third and fourth regions include the steps of overlaying outlines of the first and third regions on the reference image, and overlaying outlines of the second and fourth regions on the map.

18. A monitoring apparatus, comprising:

an image detector which outputs a succession of images of a monitored area; and

a computer system which receives the images from the image detector, which saves one of the detected images as a reference image, which has information therein representing a map of the monitored area, which has an operator input/output section, and which is operative to;

permit an operator to define a first region via the operator input/output section which corresponds to a selected portion of the area as viewed in the reference image;

permit the operator to define a second region via the operator input/output section which corresponds to the selected portion as viewed on the map;

computing warp transformation parameters for translating a selected point in the first region into a corresponding point in the second region;

compare the reference image with a subsequent image to identify an object of interest in the monitored area;

select a first point near a lower end of the portion of the subsequent image which corresponds to the object of interest, the first point being in the first region; and

carry out a warp transformation of the first point from the first region to the second region according to the computed warp transform parameters in order to identify within the second region a second point which corresponds to the first point and which identifies the position on the map of the object of interest.

19. A monitoring apparatus according to claim 18, wherein said computer system is further operative to display on the map a visible indicator corresponding to the position of the object of interest.

20. A monitoring apparatus according to claim 18, wherein said computer system is operative to facilitate the definition of the first region by visually displaying the reference image, by permitting the operator to identify the first region on the reference image, and by overlaying an outline of the first region on the displayed reference image.

21. A monitoring apparatus according to claim 18, wherein said computer system is operative to facilitate the definition of the second region by visually displaying the map, by permitting the operator to identify the second region on the map, and by overlaying an outline of the second region on the displayed map.

22. A method of monitoring, comprising the steps of:

providing a map of a monitored area;

detecting a reference image of the monitored area;

receiving operator input identifying a first region which corresponds to a selected portion of the monitored area as viewed in the reference image;

receiving operator input identifying a second region which corresponds to the selected portion as viewed on the map;

defining warp transformation parameters for translating a selected point in the first region into a corresponding point in the second region;

detecting a further image of the area;

identifying a portion of the further image corresponding to an object of interest;

selecting a first point near a lower end of the portion of the further image which corresponds to the object of interest, the first point being within the first region;

carrying out a warp transformation of the first point from the first region to the second region according to the defined warp transform parameters in order to identify within the second region a second point which corresponds to the first point and which identifies a position on the map of the object of interest; and

receiving operator input identifying a third region which corresponds to a selected portion of the monitored area as viewed in the reference image, the third region being at least partially within the first region; and

inhibiting said step of carrying out the warp transformation if the portion of the further image corresponding to the object of interest is completely within the third region.