Image processing apparatus for performing object recognition focusing on object motion, and image processing method therefor

• US 9,489,747 B2
• Filed: 05/27/2014
• Issued: 11/08/2016
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kuchiki, Hiroaki
• Primary Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US14/288,136
• Publication Number:
  US 20150003676A1
• Time in Patent Office: 896 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

An image processing apparatus includes a detection unit configured to detect areas that have changed between a plurality of images, a calculation unit configured to calculate movement vectors for respective areas on the plurality of images, an estimation unit configured to estimate areas where a moving object exists in the plurality of images by using the movement vectors for respective areas calculated by the calculation unit, and information about the motion of an imaging unit, which captured the plurality of images, in the plurality of images, and an identification unit configured to identify moving object areas based on the changed areas detected by the detection unit, and the areas where a moving object exists estimated by the estimation unit.

Claims

1. An image processing apparatus comprising;

one or more processors; and

a memory storing instructions which, when executed by the processors, cause the information processing apparatus to function as;

a calculation unit configured to calculate movement vectors for respective areas on a plurality of images;

a detection unit configured to detect areas that have changed between the plurality of images, based on their differences;

an estimation unit configured to estimate areas where a moving object exists, in the plurality of images based on movement vectors for respective areas calculated by the calculation unit; and

an identification unit configured to identify moving object areas based on the changed areas detected by the detection unit and the areas estimated by the estimation unit;

wherein the detection unit identifies a background of the plurality of images and detects the changed areas from among the plurality of images having undergone position adjustment with reference to a position of the background.

2. The image processing apparatus according to claim 1, wherein the identification unit identifies, as moving object areas, the changed areas detected by the detection unit and the areas estimated by the estimation unit.

3. The image processing apparatus according to claim 1, wherein a size of areas for obtaining differences between images by the detection unit is smaller than a size of areas for obtaining movement vectors by the calculation unit.

4. The image processing apparatus according to claim 1, wherein the detection unit identifies a background of the plurality of images by using the movement vectors for respective areas calculated by the calculation unit.

5. The image processing apparatus according to claim 4, wherein the calculation unit calculates a histogram for the movement vectors for respective areas,

wherein the detection unit identifies the background based on the histogram, and

wherein the estimation unit estimates the areas where the moving object exists based on the histogram.

6. The image processing apparatus according to claim 5, wherein the detection unit identifies the background by using motion information based on information from a unit for detecting the motion of an imaging unit which captured the plurality of images.

7. The image processing apparatus according to claim 6, wherein the unit for detecting the motion of the imaging unit is a gyro sensor or an acceleration sensor.

8. The image processing apparatus according to claim 4, further comprising:

an imaging unit; and

a unit configured to detect motion of the imaging unit.

9. The image processing apparatus according to claim 1, wherein the estimation unit detects from among the movement vectors for respective areas calculated by the calculation unit, as movement vectors corresponding to moving object areas, movement vectors existing in a range having a size determined by motion information corresponding to information received from a sensor for detecting the motion of an imaging unit which captured the plurality of images and estimates the moving object areas.

10. The image processing apparatus according to claim 1, wherein the estimation unit estimates the moving object areas based on a histogram of the movement vectors.

11. The image processing apparatus according to claim 1, wherein, for continuously captured three images, the detection unit generates difference images formed of differences between an intermediate image and two other images, and performs an AND operation on two relevant difference images to detect the changed areas.

12. The image processing apparatus according to claim 1, wherein the estimation unit, after estimating the areas where the moving object exists, by using the movement vectors obtained by the calculation unit, detects movement vectors for smaller areas in the areas where the estimated moving object exists, and estimates the moving object areas.

13. The image processing apparatus according to claim 1, further comprising a combination unit configured to perform position adjustment on the plurality of images with reference to the moving object areas, and combine the plurality of images.

14. The image processing apparatus according to claim 13, wherein the image combined by the combination unit generates a combined image in which areas other than the identified moving object areas are blurred.

15. The image processing apparatus according to claim 1, wherein the detection unit identifies the background by using the movement for respective areas calculated by the calculation unit.

16. A method for controlling an image processing apparatus, the method comprising:

causing a calculation unit to calculate movement vectors for respective areas on a plurality of images;

causing a detection unit to detect areas that have changed between a plurality of images based on differences between the plurality of images;

causing an estimation unit to estimate areas where a moving object exists in the plurality of images based on the movement vectors for respective areas calculated by the calculation unit; and

causing an identification unit to identify moving object areas based on the changed areas detected by the detection unit and the areas estimated by the estimation unit;

wherein the detection unit identifies a background of the plurality of images and detects the changed areas from among the plurality of images having undergone position adjustment with reference to a position of the background.

17. A non-transitory computer-readable storage medium storing computer executable instructions for causing a computer to execute each step of the image processing method according to claim 16.

18. An image processing apparatus comprising;

a calculation unit configured to calculate movement vectors for respective areas on a plurality of images;

one or more processors; and

a memory storing instructions which, when executed by the processors, cause the information processing apparatus to function as;

a detection unit configured to detect areas that have changed between the plurality of images, based on their differences;

an estimation unit configured to estimate areas where a moving object exists, in the plurality of images based on movement vectors for respective areas calculated by the calculation unit; and

an identification unit configured to identify moving object areas based on the changed areas detected by the detection unit and the areas estimated by the estimation unit;

wherein a size of areas for obtaining differences between images by the detection unit is smaller than a size of areas for obtaining movement vectors by the calculation unit.

19. A method for controlling an image processing apparatus, the method comprising:

causing a calculation unit to calculate movement vectors for respective areas on a plurality of images;

causing a detection unit to detect areas that have changed between a plurality of images based on differences between the plurality of images;

causing an estimation unit to estimate areas where a moving object exists in the plurality of images based on the movement vectors for respective areas calculated by the calculation unit; and

causing an identification unit to identify moving object areas based on the changed areas detected by the detection unit and the areas estimated by the estimation unit;

wherein a size of areas for obtaining differences between images by the detection unit is smaller than a size of areas for obtaining movement vectors by the calculation unit.