Image processing apparatus and image processing method

• US 9,699,419 B2
• Filed: 04/01/2013
• Issued: 07/04/2017
• Est. Priority Date: 04/06/2012
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Katano, Yasuo, Kuroki, Tsuyoshi, Iwamoto, Takayuki
• Primary Examiner(s): Vo, Tung
• Assistant Examiner(s): Jiang, Zaihan
• Application Number:
  US13/854,479
• Publication Number:
  US 20130265434A1
• Time in Patent Office: 1,555 Days
• Field of Search: 348157
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An image of each frame output from an image sensing device is sequentially acquired. A situation appearing in the image is recognized through recognition processing performed on the image, and, if the recognized situation is a prescribed situation, an object of a type related to the prescribed situation is detected from the image and from an image of each frame containing the image. A detection position, in the image, of each object is used to determine whether the objects come in contact with each other in a physical space.

Claims

1. An image processing apparatus comprising:

a holding unit that holds, for each event which may occur in an image, a type of an object to be determined whether or not to come in contact with another object in a physical space, wherein the holding unit is configured to hold in advance, for each event, an object type which is preset in advance as a type of an object to be determined whether or not to come in contact with another object in the physical space;

a unit that sequentially acquires an image of each frame output from an image sensing device;

a first detection unit that detects, before performing an object detection on an image of interest, an event which occurs in the image of interest;

a specifying unit that specifies, from object types each of which is held in advance in the holding unit and is preset in advance as a type of an object to be determined whether or not to come in contact with another object in the physical space, the object type which corresponds to the event detected by the first detection unit;

a second detection unit that detects an object of the object type specified by the specifying unit from the image of interest; and

a determination unit that determines whether objects detected by the second detection unit come in contact with each other in the physical space,

wherein the first detection unit detects a first event, and the specifying unit specifies, from the holding unit, a first object type and a second object type as the object types which correspond to the first event, and

the determination unit;

sets an image, among the images of the frames, in which a distance between respective detection positions, in the image, of a first object corresponding to the first object type and a second object corresponding to the second object type is equal to or less than a threshold, as a target image, and

determines that the first object and the second object come in contact with each other in the physical space, if a magnitude of acceleration, in the target image, of the first object is greater than or equal to a threshold, and an angle formed by a vector representing the acceleration and a vector representing velocity, in the target image, of the second object is less than a threshold.

2. The image processing apparatus according to claim 1,

wherein the first detection unit performs an event detection with respect to an image output from each of a plurality of image sensing devices, the specifying unit specifies, from the holding unit, the object type corresponding to the event detected by the first detection unit, and the second detection unit detects the object of the object type specified by the specifying unit, from an image having a highest event occurrence probability calculated at a time of the detection among the images output from the plurality of image sensing devices.

3. The image processing apparatus according to claim 1, wherein the detection unit detects a person who abandons or removes an object.

4. The image processing apparatus according to claim 1, wherein the determination unit determines whether there is a violent action.

5. The image processing apparatus according to claim 1, further comprising a reporting unit that reports information corresponding to a result in which the determination unit determines that the objects come in contact.

8. The apparatus according to claim 1, wherein the holding unit holds, for each event which may occur in the image, the first object type of the first object whose acceleration and velocity are to be calculated and the second object type of the second object whose velocity is to be calculated.

9. The apparatus according to claim 1, wherein the first detection unit detects, before performing the object detection on the image of interest, the event which occurs in the image of interest using image features of the image of interest.

6. An image processing method that is performed by an image processing apparatus, comprising:

holding, for each event which may occur in an image, a type of an object to be determined whether or not to come in contact with another object in a physical space, wherein the holding holds in advance, for each event, an object type which is preset in advance as a type of an object to be determined whether or not to come in contact with another object in the physical space;

sequentially acquiring an image of each frame output from an image sensing device;

detecting, before performing an object detection on an image of interest, an event which occurs in the image of interest;

specifying, from object types each of which is held in advance in the holding step and is preset in advance as a type of an object to be determined whether or not to come in contact with another object in the physical space, the object type which corresponds to the event detected by the first detection step;

detecting an object of the object type specified by the specifying step from the image of interest; and

determining whether objects come in contact with each other in the physical space, based on a result of the detection,

wherein the first detection step detects a first event, and the specifying step specifies, from the holding step, a first object type and a second object type as the object types which correspond to the first event, and

the determining step;

sets an image, among the images of the frames, in which a distance between respective detection positions, in the image, of a first object corresponding to the first object type and a second object corresponding to the second object type is equal to or less than a threshold, as a target image, and

determines that the first object and the second object come in contact with each other in the physical space, if a magnitude of acceleration, in the target image, of the first object is greater than or equal to a threshold, and an angle formed by a vector representing the acceleration and a vector representing velocity, in the target image, of the second object is less than a threshold.

7. A non-transitory computer-readable storage medium storing a computer program for a computer having a holding unit that holds, for each event which may occur in an image, a type of an object to be determined whether or not to come in contact with another object in a physical space, wherein the holding unit is configured to hold in advance, for each event, an object type which is preset in advance as a type of an object to be determined whether or not to come in contact with another object in the physical space, the program for causing the computer to function as:

an acquisition unit that sequentially acquires an image of each frame output from an image sensing device;

a first detection unit that detects, before performing an object detection on an image of interest, an event which occurs in the image of interest;

a specifying unit that specifies, from object types of each of which is held in advance in the holding unit and is preset in advance as a type of an object to be determined whether or not to come in contact with another object in the physical space, the object type which corresponds to the event detected by the first detection unit;

a second detection unit that detects an object of the object type specified by the specifying unit from the image of interest; and

a determination unit that determines whether objects detected by the second detection step come in contact with each other in the physical space,

wherein the first detection unit detects a first event, and the specifying unit specifies, from the holding unit, a first object type and a second object type as the object types which correspond to the first event, and

the determination unit;

sets an image, among the images of the frames, in which a distance between respective detection positions, in the image, of a first object corresponding to the first object type and a second object corresponding to the second object type is equal to or less than a threshold, as a target image, and

determines that the first object and the second object come in contact with each other in the physical space, if a magnitude of acceleration, in the target image, of the first object is greater than or equal to a threshold, and an angle formed by a vector representing the acceleration and a vector representing velocity, in the target image, of the second object is less than a threshold.

10. An image processing apparatus, comprising:

an acquisition unit that acquires a target frame image which includes a first object and a second object;

a first calculation unit that calculates a distance between the first object and the second object on the acquired target frame image;

a second calculation unit that calculates acceleration of the first object and velocity of the second object on the acquired target frame image; and

a determination unit that determines, in a case where on the target frame image (A) the calculated distance is not more than a first threshold, (B) the calculated acceleration is not less than a second threshold, and (C) an angle formed by a vector representing the calculated acceleration and a vector representing the calculated velocity is less than a third threshold, that the first object comes in contact with the second object.

11. The apparatus according to claim 10, further comprising:

a first detection unit that detects a specific event from a plurality of frame images; and

an extraction unit that extracts, from the plurality of frame images, at least one frame image which includes the detected specific event,

wherein the acquisition unit acquires as the target frame image, a single frame image from the at least one frame images extracted by the extraction unit.

12. The apparatus according to claim 11, further comprising a second detection unit that detects, from the target frame image, the first object and the second object each of which is associated with the specific event detected by the first detection unit.

13. The apparatus according to claim 11, wherein the first detection unit detects, from a plurality of frame images obtained by capturing a soccer game, a sliding tackle as the specific event, and

the second detection unit detects a ball as the first object and detects a foot of a player as the second object.

14. An image processing method, comprising:

acquiring a target frame image which includes a first object and a second object;

calculating a distance between the first object and the second object on the acquired target frame image;

calculating acceleration of the first object and velocity of the second object on the acquired target frame image; and

determining, in a case where on the target frame image (A) the calculated distance is not more than a first threshold, (B) the calculated acceleration is not less than a second threshold, and (C) an angle formed by a vector representing the calculated acceleration and a vector representing the calculated velocity is less than a third threshold, that the first object comes in contact with the second object.

15. A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute an image processing method, the method comprising:

acquiring a target frame image which includes a first object and a second object;

calculating a distance between the first object and the second object on the acquired target frame image;

calculating acceleration of the first object and velocity of the second object on the acquired target frame image; and

determining, in a case where on the target frame image (A) the calculated distance is not more than a first threshold, (B) the calculated acceleration is not less than a second threshold, and (C) an angle formed by a vector representing the calculated acceleration and a vector representing the calculated velocity is less than a third threshold, that the first object comes in contact with the second object.