APPARATUS AND METHOD FOR RESOURCE-ADAPTIVE OBJECT DETECTION AND TRACKING

• US 20160163064A1
• Filed: 01/27/2016
• Published: 06/09/2016
• Est. Priority Date: 07/30/2013
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Ruf, Tobias, Ernst, Andreas, GARBAS, Jens-Uwe, Papst, Anton
• Granted Patent: US 9,786,059 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking

Abstract

An apparatus for providing object information based on an image sequence including a plurality of images is provided. The apparatus includes an object detector for conducting object detection on three or more images of the plurality of images of the image sequence to obtain the object information, wherein each image of the image sequence on which object detection is conducted, is an object-detected image of the image sequence, and wherein each image of the image sequence on which object detection is not conducted, is not an object-detected image of the image sequence. Moreover, the apparatus includes an object tracker for conducting object tracking on one or more images of the image sequence to obtain the object information.

Claims

1. An apparatus for providing object information based on an image sequence comprising a plurality of images, wherein the apparatus comprises:

an object detector for conducting object detection on three or more images of the plurality of images of the image sequence to acquire a result of the object detection as the object information, wherein each image of the image sequence on which object detection is conducted, is an object-detected image of the image sequence, and wherein each image of the image sequence on which object detection is not conducted, is not an object-detected image of the image sequence, and

an object tracker for conducting object tracking on one or more images of the image sequence to acquire a result of the object tracking as the object information,

wherein the object detector is adapted to conduct object detection and the object tracker is adapted to conduct object tracking such that an object detection distance is different for at least two pairs of two object-detected images of the image sequence, wherein the object detection distance is defined for each pair of two object-detected images of the image sequence, for which at least one image exists between said two object-detected images within the image sequence, and for which no object-detected image exists between said two object-detected images within the image sequence, as the number of images between said two object-detected images within the image sequence, which are not object-detected images,

wherein the apparatus furthermore comprises a receiving interface, a detection queue, a tracking queue and a queue controller,

wherein the receiving interface is configured to receive the plurality of images of the image sequence, and wherein the receiving interface is configured to store at least some of the images of the image sequence within the detection queue,

wherein the queue controller is configured to move one or more images, being stored in the detection queue, from the detection queue to the tracking queue depending on whether object detection on one of the images of the image sequence conducted by the object detector is completed, and

wherein the object tracker is configured to conduct object tracking on one of the images being stored in the tracking queue, depending on whether images are stored in the tracking queue.

2. An apparatus according to claim 1,

wherein the object detector is configured to conduct object detection on a first one of the plurality of images of the image sequence, and

wherein the object detector is configured to start conducting object detection on a second one of the images of the image sequence depending on when the conducting of the object detection on the first one of the plurality of images is completed.

3. An apparatus according to claim 1, wherein the object detector is configured to conduct object detection on one of the plurality of images of the image sequence in parallel to the object tracker conducting object tracking on another one of the plurality of images of the image sequence.

4. An apparatus according to claim 3, wherein the object detector is configured to employ at least two processes or at least two threads to conduct object detection on said one of the plurality of images of the image sequence in parallel.

5. An apparatus according to claim 3, wherein the object tracker is configured to employ at least two processes or at least two threads to conduct object tracking on said another one of the plurality of images of the image sequence in parallel.

6. An apparatus according to claim 1,

wherein the object detector is configured to select as a selected image, when two or more images of the image sequence are stored in the image queue, an image of the two or more images of the image sequence being stored in the image queue, which has most recently been received by the receiving interface among all images of the two or more images of the image sequence being stored in the image queue, and

wherein the object detector is configured to conduct object detection on the selected image.

7. An apparatus according to claim 1, wherein the receiving interface is configured to not store an image of the image sequence received by the receiving interface within the image queue, when the number of images that are stored in the image queue is equal to a maximum number.

8. An apparatus according to claim 1,

wherein the object detector is configured to acquire a detection result by conducting object detection on one of the plurality of images of the image sequence,

wherein the object tracker is configured to receive the detection result,

wherein the object tracker is configured to retrieve at least one of the images of the image sequence from the image queue when the object tracker has received the detection result, and

wherein the object tracker is configured to conduct object tracking on said at least one of the images retrieved from the image queue based on the detection result.

9. An apparatus according to claim 1, wherein the queue controller is configured to move all images being stored in the detection queue from the detection queue to the tracking queue, depending on whether the object detection on said one of the images of the image sequence conducted by the object detector is completed.

10. An apparatus according to claim 1, wherein the queue controller is configured to move all but one image being stored in the detection queue from the detection queue to the tracking queue, depending on whether the object detection on said one of the images of the image sequence conducted by the object detector is completed, wherein said one image being stored in the detection queue which is not moved from the detection queue to the tracking queue is that image which is last received by the receiving interface among all images being stored in the detection queue.

11. An apparatus according to claim 1, wherein the object detector is configured to conduct object detection without skipping any portion of an image on which object detection is conducted.

12. An apparatus according to claim 1,

wherein the object detector is configured to conduct object detection on the plurality of images of the image sequence by conducting face detection, and

wherein the object tracker is configured to conduct object tracking on the one or more images of the image sequence by conducting face tracking.

13. A method for providing object information based on an image sequence comprising a plurality of images, wherein the method comprises:

conducting object detection on three or more images of the plurality of images of the image sequence to acquire a result of the object detection as the object information, wherein each image of the image sequence on which object detection is conducted, is an object-detected image of the image sequence, and wherein each image of the image sequence on which object detection is not conducted, is not an object-detected image of the image sequence, and

conducting object tracking on one or more images of the image sequence to acquire a result of the object tracking as the object information,

wherein object detection and object tracking is conducted such that an object detection distance is different for at least two pairs of two object-detected images of the image sequence, wherein the object detection distance is defined for each pair of two object-detected images of the image sequence, for which at least one image exists between said two object-detected images within the image sequence, and for which no object-detected image exists between said two object-detected images within the image sequence, as the number of images between said two object-detected images within the image sequence, which are not object-detected images,

wherein the method further comprises;

receiving the plurality of images of the image sequence,

storing at least some of the images of the image sequence within a detection queue,

moving one or more images, being stored in the detection queue, from the detection queue to a tracking queue depending on whether object detection on one of the images of the image sequence conducted by the object detector is completed, and

conducting object tracking on one of the images being stored in the tracking queue, depending on whether images are stored in the tracking queue.

14. Non-transitory digital storage medium having computer-readable code stored thereon to perform the method of claim 13 when being executed on a computer or signal processor.