Image monitor apparatus and a method

• US 6,205,242 B1
• Filed: 09/24/1998
• Issued: 03/20/2001
• Est. Priority Date: 09/29/1997
• Status: Expired due to Term

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Onoguchi, Kazunori
• Primary Examiner(s): Tran, Phuoc
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US09/161,428
• Time in Patent Office: 908 Days
• Field of Search: 382/103, 382/104, 382/128, 382/154, 382/190, 382/218, 382/276, 382/283, 345/139, 356/12
• US Class Current: 382/154
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

In the image monitor apparatus, a plurality of images are respectively inputted through a plurality of TV camera from each view position. An image projection section projects each of the plurality of images onto a predetermined plane in three-dimensional space. A comparison section compares the projected image data of two images by unit of predetermined area on the plane, and extracts the predetermined areas, each of which two projected image data are different, from the plane. An extraction section extracts an existence position of a monitored object from the predetermined areas extracted. An integration section integrates the existence positions of the monitored object extracted for the projected image data of all of two images in the plurality of images as all existence positions of the monitored objects on the plane.

Claims

1. An image monitor apparatus, comprising:

a plurality of image input means for respectively inputting an image based on each different view position in order to input the images of a predetermined plane, an existence area of a monitored object on the predetermined plane being commonly viewed by two of the plurality of image input means;

image projection means for projecting each of the plurality of images onto a plane memory corresponding to the predetermined plane;

comparison means for comparing projected image data of two images from the two image input means by unit of a predetermined area on the plane memory, and for extracting a plurality of predetermined areas, each area including two different projected image data, from the plane memory;

extraction means for detecting two wedge-shaped areas extended from a common starting point along a respective direction from the plurality of predetermined areas, and for extracting the common starting point as a position of the monitored object; and

integration means for integrating a plurality of the positions of the monitored objects respectively extracted from each two wedge-shaped areas including two different projected image data of two images in the plurality of images as the existence area of the monitored objects on the predetermined plane.

2. The image monitor apparatus according to claim 1,

wherein the plurality of image input means are previously located at each different view position in order to input the images of the predetermined plane from the view position.

3. The image monitor apparatus according to claim 1,

further comprising a plurality of image memory means for respectively storing the image inputted through the plurality of image input means.

4. The image monitor apparatus according to claim 1,

further comprising a selection means for selecting two images from the plurality of inputted images, the two images being respectively inputted through two image input means located adjacent the existence area of the monitored object, and for supplying the two images to said comparison means.

5. The image monitor apparatus according to claim 4,

wherein the predetermined plane is a flat area of a road surface commonly viewed by the two image input means, and

wherein the projected image data includes projected (x, y) coordinates and a density value by unit of original (x, y) coordinate on the input image.

6. The image monitor apparatus according to claim 5,

wherein said comparison means selects a maximum value and a minimum value of the projected (x, y) coordinates of the two images, assigns each projected image data of the two images to a plane memory corresponding to the predetermined plane,

the plane memory being equally divided into the predetermined area between the maximum value and the minimum value along (x, y) coordinate axis, and

the density values of the two images being stored in the predetermined area of the plane memory including the projected (x, y) coordinate.

7. The image monitor apparatus according to claim 6,

wherein said comparison means compares the density values of the two images stored in each predetermined area of the plane memory, and extracts the predetermined area if a difference between the density values of the two images is above a threshold.

8. The image monitor apparatus according to claim 1,

wherein the respective direction along which the two wedge-shaped areas extend is the same as a direction of a vector from the view position of said image input means to a projection position onto the plane.

9. An image monitor apparatus, comprising:

at least two image input means for respectively inputting an image based on each different view position in order to input two images of a predetermined plane at the same timing;

transformation means for projecting the image inputted through one image input means onto the predetermined plane, and for transforming the projected image according to a view position of the other image input means;

comparison means for comparing image data of the transformed image with image data of the image inputted through the other image input means by unit of the same position on the image, and for generating a mask area representing positions of different image data according to the comparison result; and

extraction means for extracting an area of a moving object from the mask area by using the mask area in a plurality of the images inputted through the other image input means.

10. The image monitor apparatus according to claim 9,

further comprising at least two image memory means for respectively storing the image inputted through the at least two image input means.

11. The image monitor apparatus according to claim 9,

wherein the two image input means are previously located at different view positions in order to input the image of a predetermined plane commonly viewed by the two image input means, a focal length of the two image input means being equal.

12. The image monitor apparatus according to claim 11,

wherein said transformation means transforms the image by using transformation parameters, the transformation parameters are previously calculated by correspondence of a plurality of position on the predetermined plane between two images inputted through the one image input means and the other image input means.

13. The image monitor apparatus according to claim 9,

wherein said comparison means compares a density value of the transformed image of the one image input means with a density value of the input image of the other image input means by unit of the same pixel position, and extracts the pixel positions wherein a correlation value representing a difference of the density values between the transformed image and the input image is above a threshold as the mask area.

14. The image monitor apparatus according to claim 13,

wherein said extraction means extracts a first area of the mask area from a background image without a monitored object inputted through the other image input means, extracts a second area of the mask area from the input image of the other image input means, and extracts a difference area between the first area and the second area as the moving object.

15. The image monitor apparatus according to claim 13,

wherein said extraction means creates a merged mask area of the mask area of time (t) and the mask area of time (t+Δ

t), extracts a first area of the merged mask area from the input image of time (t) of the other image input means, extracts a second area of the merged mask area from the input image of time (t+Δ

t) of the other image input means, and extracts a difference area between the first area and the second area as the moving object.

16. A method for monitoring an object in an image, comprising the steps of:

inputting a plurality of images from each different view position in order to input the images of a predetermined plane, an existence area of a monitored object on the predetermined plane being commonly viewed by two of the plurality of images;

projecting each of the plurality of images onto a plane memory corresponding to the predetermined plane;

comparing projected image data of two images by unit of a predetermined area on the plane memory;

extracting a plurality of predetermined areas, each area including two different projected image data, from the plane memory;

detecting two wedge-shaped areas extended from a common starting point along a respective direction from the plurality of predetermined areas;

extracting the common starting point as a position of the monitored object; and

integrating a plurality of the positions of the monitored objects respectively extracted from each two wedge-shaped areas including two different projected image data of two images in the plurality of images as the existence area of the monitored objects on the predetermined plane.

17. A method for monitoring an object in an image, comprising the steps of:

inputting two images from each different view position in order to input two images of a predetermined plane at the same timing;

projecting one image inputted from one view position onto the predetermined plane;

transforming the projected image according to the other view position;

comparing image data of the transformed image with image data of the other image inputted from the other view position by unit of the same position on the image;

generating a mask area representing positions of different image data according to the comparison result; and

extracting an area of a moving object from the mask area by using the mask area in a plurality of the other images.

18. A computer readable memory containing computer readable instructions, comprising:

instruction means for causing a computer to input a plurality of images from each different view position in order to input the images of a predetermined plane, an existence area of a monitored object on the predetermined plane being commonly viewed by two of the plurality of images;

instruction means for causing a computer to project each of the plurality of images onto a plane memory corresponding to the predetermined plane;

instruction means for causing a computer to compare projected image data of two images by unit of a predetermined area on the plane memory;

instruction means for causing a computer to extract a plurality of predetermined areas, each area including two different projected image data, from the plane memory;

instruction means for causing a computer to detect two wedge-shaped areas extended from a common starting point along a respective direction from the plurality of predetermined areas;

instruction means for causing a computer to extract the common starting point as a position of the monitored object; and

instruction means for causing a computer to integrate a plurality of positions of the monitored objects respectively extracted from each two wedge-shaped areas including two different projected image data of two images in the plurality of images as the existence area of the monitored objects on the predetermined plane.

19. A computer readable memory containing computer readable instructions, comprising:

instruction means for causing a computer to input two images from each different view position in order to input two images of a predetermined plane at the same timing;

instruction means for causing a computer to project one image inputted from one view position onto the predetermined plane;

instruction means for causing a computer to transform the projected image according to the other view position;

instruction means for causing a computer to compare image data of the transformed image with image data of the other image inputted from the other view position by unit of the same position on the image;

instruction means for causing a computer to generate a mask area representing positions of different image data according to the comparison result; and

instruction means for causing a computer to extract an area of a moving object from the mask area by using the mask area in a plurality of the other images.

20. An image monitor apparatus, comprising:

a plurality of image input mechanisms configured to respectively input an image based on each different view position in order to input the images of a predetermined plane, an existence area of a monitored object on the predetermined plane being commonly viewed by two of the plurality of image input mechanisms;

an image projection section configured to project each of the plurality of images onto a plane memory corresponding to the predetermined plane;

a comparison section configured to compare projected image data of two images from the two image input mechanisms by unit of a predetermined area on the plane memory, and to extract a plurality of predetermined areas, each area including two different projected image data, from the plane memory;

an extraction section configured to detect two wedge-shaped areas extended from a common starting point along a respective direction from the plurality of predetermined areas, and to extract the common starting point as a position of the monitored object; and

an integration section configured to integrate a plurality of the positions of the monitored objects respectively extracted from each two wedge-shaped areas including two different projected image data of two images in the plurality of images as the existence area of the monitored objects on the predetermined plane.

21. The image monitor apparatus according to claim 20,

wherein the plurality of image input mechanisms are previously located at each different view position in order to input the images of the predetermined plane from the view position.

22. The image monitor apparatus according to claim 20, further comprising a plurality of image memories configured to respectively store the image inputted through the plurality of image input mechanisms.

23. The image monitor apparatus according to claim 20, further comprising a selection unit configured to select images from the plurality of inputted images, the two images being respectively inputted through two image input mechanisms located adjacent the existence area of the monitored object, and to supply the two images to said comparison section.

24. The image monitor apparatus according to claim 23,

wherein the predetermined plane is a flat area of a road surface commonly viewed by the two image input mechanisms, and

wherein the projected image data includes projected (x, y) coordinates and a density value by unit of original (x, y) coordinate on the input image.

25. The image monitor apparatus according to claim 24,

wherein said comparison section selects a maximum value and a minimum value of the projected (x, y) coordinates of the two images, assigns each projected image data of the two images to a plane memory corresponding to the predetermined plane,

the plane memory being equally divided into the predetermined area between the maximum value and the minimum value along (x, y) coordinate axis, and

the density values of the two images being stored in the predetermined area of the plane memory including the projected (x, y) coordinate.

26. The image monitor apparatus according to claim 25,

wherein said comparison section compares the density values of the two images stored in each predetermined area of the plane memory, and extracts the predetermined area if a difference between the density values of the two images is above a threshold.

27. The image monitor apparatus according to claim 20,

wherein the respective direction along which the two wedge-shaped areas extend is the same as a direction of a vector from the view position of said image input mechanism to a projection position onto the plane.

28. An image monitor apparatus, comprising:

at least two image input mechanisms configured to respectively input an image based on each different view position in order to input two images of a predetermined plane at the same timing;

a transformation section configured to project the image inputted through one image input mechanism onto the predetermined plane, and to transform the projected image according to a view position of the other image input mechanism;

a comparison section configured to compare image data of the transformed image with image data of the image inputted through the other image input mechanism by unit of the same position on the image, and to generate a mask area representing positions of different image data according to the comparison result; and

an extraction section configured to extract an area of a moving object from the mask area by using the mask area in a plurality of the images inputted through the other image input mechanism.

29. The image monitor apparatus according to claim 28, further comprising at least two image memory memories configured to respectively store the image inputted through the at least two image input mechanisms.

30. The image monitor apparatus according to claim 28,

wherein the two image input mechanisms are previously located at different view positions in order to input the image of a predetermined plane commonly viewed by the two image input mechanisms, a focal length of the two image input mechanisms being equal.

31. The image monitor apparatus according to claim 30,

wherein said transformation section transforms the image by using transformation parameters, the transformation parameters are previously calculated by correspondence of a plurality of position on the predetermined plane between two images inputted through the one image input mechanism and the other image input mechanism.

32. The image monitor apparatus according to claim 28,

wherein said comparison section compares a density value of the transformed image of the one image input mechanism with a density value of the input image of the other image input mechanism by unit of the same pixel position, and extracts the pixel positions wherein a correlation value representing a difference of the density values between the transformed image and the input image is above a threshold as the mask area.

33. The image monitor apparatus according to claim 32,

wherein said extraction section extracts a first area of the mask area from a background image without a monitored object inputted through the other image input mechanism, extracts a second area of the mask area from the input image of the other image input mechanism, and extracts a difference area between the first area and the second area as the moving object.

34. The image monitor apparatus according to claim 32,

wherein said extraction mechanism creates a merged mask area of the mask area of time (t) and the mask area of time (t+Δ

t), extracts a first area of the merged mask area from the input image of time (t) of the other image input mechanism, extracts a second area of the merged mask area from the input image of time (t+Δ

t) of the other image input mechanism, and extracts a difference area between the first area and the second area as the moving object.