Multiple image processing and synthesis using background image extraction

• US 20040008773A1
• Filed: 06/13/2003
• Published: 01/15/2004
• Est. Priority Date: 06/14/2002
• Status: Active Grant

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Itokawa, Osamu
• Granted Patent: US 7,292,264 B2
• Time in Patent Office: Days
• US Class Current: 375/240.08
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A technique of efficiently extracting and synthesizing an object image from a plurality of images captured at different image magnifications. An image processing method of the present invention includes the steps of generating a first background image corresponding to a first image captured at a predetermined image magnification, extracting a first object image from the first image, generating a second background image corresponding to a second image captured at an image magnification higher than the predetermined image magnification, based on the first background image and the first object image, and extracting a second object image from the second image and the second background image.

Claims

1. An image processing apparatus comprising:

first background image generating unit for generating a first background image corresponding to a first image captured at a predetermined image magnification;

first object image extracting unit for extracting a first object image from the first image;

second background image generating unit for generating a second background image corresponding to a second image captured at an image magnification higher than the predetermined image magnification, based on the first background image and the first object image; and

second object image extracting unit for extracting a second object image from the second image and the second background image.

2. An image processing apparatus according to claim 1 further comprising:

first background image encoding unit for encoding the first background image generated by the first background image generating unit;

first object image encoding unit for encoding the first object image extracted by the first object image extracting unit; and

second object image encoding unit for encoding the second object image extracted by the second object image extracting unit.

3. An image processing apparatus according to claim 2, further comprising:

multiplexing unit for multliplexing output signals from the first background image encoding unit, the first object image encoding unit, the second object image encoding unit, and transmitting a multiplexed output therefrom.

4. An image processing apparatus according to claim 2, wherein the first object image extracting unit determines a rectangular area containing a shape of the first object image therewithin.

5. An image processing apparatus according to claim 4, wherein the rectangular area has a rectangular shape that circumscribes the shape of the first object image, horizonal and vertical coordinates of the top left corner of the rectangular shape are even numbers in a coordinates system with the origin of the coordinates system placed at the top left corner of a screen, and each of a width and a height of the rectangular area is an integer multiple of a size of a macro block in the first object image encoding unit.

6. An image processing apparatus according to claim 4, wherein the second background image generating unit determines an expansion rate from the height and width of the rectangular area and the height and width of a screen, and expands the first background image based on the expansion rate to generate the second background image.

7. An image processing apparatus according to claim 6, wherein the second background image generating unit determines, as the expansion rate, a maximum image magnification at which the rectangular area, when expanded, fills the entire screen.

8. An image processing apparatus according to claim 4, wherein each of the first and second object image encoding units encodes a position of and the height and width of the rectangular area.

9. A surveillance camera system comprising an image processing apparatus, the image processing apparatus comprising:

first background image generating unit for generating a first background image corresponding to a first image captured at a predetermined image magnification;

first object image extracting unit for extracting a first object image from the first image;

second background image generating unit for generating a second background image corresponding to a second image captured at an image magnification higher than the predetermined image magnification, based on the first background image and the first object image; and

second object image extracting unit for extracting a second object image from the second image and the second background image.

10. An image processing apparatus comprising:

input unit for inputting a first background image captured at a predetermined image magnification, a first object image extracted from image data captured at the predetermined image magnification, and a second object image extracted from image data captured at an image magnification higher than the predetermined image magnification;

background image generating unit for generating a second background image corresponding to the second object image based on the first background image and the first object image; and

image synthesizing unit for synthesizing the second background image and the second object image.

11. An image processing apparatus according to claim 10, wherein the image synthesizing unit synthesizes the first background image and the first object image.

12. An image processing apparatus according to claim 10, wherein the first object image is a rectangular shaped image containing a shape of an object within, and further wherein the input unit also inputs information representing a position and a height and width of the rectangular area within a screen.

13. An image processing apparatus according to claim 12, wherein the background image generating unit determines an expansion rate from the height and width of the rectangular area and the size of the screen, and generates the second background image by expanding the first background image at the expansion rate.

14. An image processing apparatus according to claim 13, wherein the background image generating unit determines, as the expansion rate, a maximum image magnification at which the rectangular area, when expanded, fills the screen.

15. An image processing apparatus according to claim 10, wherein the first background image, the first object image, and the second object image, each input by the input unit, are encoded, and further wherein the input unit comprises decoding unit for decoding each of the first background image, the first object image, and the second object image.

16. An image processing apparatus according to claim 10, further comprising display unit for displaying an image synthesized by the image synthesizing unit.

17. An image processing method comprising the steps of:

generating a first background image corresponding to a first image captured at a predetermined image magnification;

extracting a first object image from the first image;

generating a second background image corresponding to a second image captured at an image magnification higher than the predetermined image magnification, based on the first background image and the first object image; and

extracting a second object image from the second image and the second background image.

18. An image processing method comprising the steps of:

inputting a first background image captured at a predetermined image magnification, a first object image extracted from image data captured at the predetermined image magnification, and a second object image extracted from image data captured at an image magnification higher than the predetermined image magnification;

generating a second background image corresponding to the second object image based on the first background image and the first object image; and

synthesizing the second background image and the second object image.

19. Computer-executable process steps stored on a computer-readable medium, said computer-executable process steps for processing an image, said computer-executable process steps comprising the steps for:

generating a first background image corresponding to a first image captured at a predetermined image magnification;

extracting a first object image from the first image;

generating a second background image corresponding to a second image captured at an image magnification higher than the predetermined image magnification, based on the first background image and the first object image; and

extracting a second object image from the second image and the second background image.

20. Computer-executable process steps stored on a computer-readable medium, said computer-executable process steps for processing an image, said computer-executable process steps comprising the steps for:

inputting a first background image captured at a predetermined image magnification, a first object image extracted from image data captured at the predetermined image magnification, and a second object image extracted from image data captured at an image magnification higher than the predetermined image magnification;

generating a second background image corresponding to the second object image based on the first background image and the first object image; and

synthesizing the second background image and the second object image.

21. A computer-readable medium which stores computer-executable process steps, the computer-executable process steps for processing an image, said computer executable process steps comprising the steps for:

generating a first background image corresponding to a first image captured at a predetermined image magnification;

extracting a first object image from the first image;

generating a second background image corresponding to a second image captured at an image magnification higher than the predetermined image magnification, based on the first background image and the first object image; and

extracting a second object image from the second image and the second background image.

22. A computer-readable medium which stores computer-executable process steps, the computer-executable process steps for processing an image, said computer executable process steps comprising the steps for:

inputting a first background image captured at a predetermined image magnification, a first object image extracted from image data captured at the predetermined image magnification, and a second object image extracted from image data captured at an image magnification higher than the predetermined image magnification;

generating a second background image corresponding to the second object image based on the first background image and the first object image; and

synthesizing the second background image and the second object image.

23. An image processing apparatus comprising:

object image extracting unit for extracting an image of an object from a captured image;

calculating unit for calculating a direction and a zoom magnification of a camera by determining a position and a height and width of the object of the object image extracted by the object image extracting unit; and

capturing unit for capturing the image of the object in the direction and the zoom magnification calculated by the calculating unit.

24. An image processing method comprising the steps of:

extracting an image of an object from a captured image;

calculating a direction and a zoom magnification of a camera by determining a position and a height and width of the object in the captured image; and

capturing an image of the object using the direction and the zoom magnification.