Image processing method, image processing apparatus, and recording medium for reducing variation in quality of training data items

• US 10,282,634 B2
• Filed: 03/06/2017
• Issued: 05/07/2019
• Est. Priority Date: 03/11/2016
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Inventors: Kozuka, Kazuki, Tanigawa, Toru, Saito, Masahiko
• Primary Examiner(s): Liu, Li
• Application Number:
  US15/450,383
• Publication Number:
  US 20170262728A1
• Time in Patent Office: 792 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An image processing method includes acquiring consecutive time-series images captured by an onboard camera of a vehicle, having a first annotation indicating two or more first regions, and at least including one or more images in which the two or more first regions are on a path of the vehicle and a distance therebetween is smaller than or equal to a threshold; determining, in reverse chronological order from an image of the last time point, positions of the two or more regions in each consecutive time-series image; identifying, from among the consecutive time-series images, the first image of a first time point in which none of the two or more first regions are located on the path, and setting, as a second region, a region between the two or more first regions in the identified first image; and attaching a second annotation to the first image corresponding to the first time point, the second annotation indicating the second region.

Claims

1. A computer-implemented image processing-method, the method, utilizing one or more processors, comprising:

acquiring, by the one or more processors, a plurality of consecutive time-series images that have been captured by an onboard camera mounted on a vehicle and to which a first annotation indicating two or more first regions is attached, the two or more first regions including at least one person region, the plurality of consecutive time-series images at least including one or more images in which the two or more first regions are located on a path of the vehicle and a distance between the two or more first regions is smaller than or equal to a threshold;

determining, by the one or more processors, in reverse chronological order from an image corresponding to the last time point in the time series, a position of each of the two or more regions in each of the plurality of consecutive time-series images acquired in the acquiring;

identifying, by the one or more processors, from among the plurality of consecutive time-series images, a first image corresponding to a first time point for which it has been determined for the first time in the determining that none of the two or more first regions are located on the path, and setting, as a second region, a region between the two or more first regions in the identified first image; and

attaching, by the one or more processors, a second annotation to the first image corresponding to the first time point, the second annotation indicating the second region set in the setting.

2. The image processing method according to claim 1, wherein in the identifying and setting,

a second image is identified in which the two or more first regions are located on the path of the vehicle and the distance between the two or more first regions is smaller than or equal to the threshold, and

a region between the two or more first regions in each of a plurality of consecutive time-series images from the identified first image to the identified second image is set as the second region.

3. The image processing method according to claim 1, further comprising:

performing a first extracting process of selecting, from among all of consecutive time-series images that have been captured by the onboard camera mounted on the vehicle and that are associated with information representing braking force or acceleration of the vehicle, first extracted images that are a plurality of images up to a time point preceding, by a predetermined period, a time point at which the braking force or acceleration of the vehicle is larger than a threshold; and

performing a second extracting process of selecting the plurality of consecutive time-series images from among the first extracted images selected through the first extracting process.

4. The image processing method according to claim 3, further comprising:

causing crowdsourcing workers to attach, to each of the all of the consecutive time-series images, the first annotation indicating the first region existing in the image prior to the first extracting process.

5. The image processing method according to claim 4, further comprising:

causing crowdsourcing workers to attach, to each of the first extracted images selected through the first extracting process, the first annotation indicating the first region existing in the first extracted image prior to the second extracting process.

6. The image processing method according to claim 1, wherein each of the two or more first regions is a person region representing a person.

7. The image processing method according to claim 1, wherein the two or more first regions include a person region representing a person and a vehicle region representing a stationary vehicle.

8. The image processing method according to claim 1,

wherein the second region is a hazard region in which, when objects represented by the two or more first regions approach, at least one of the objects crosses in front of the vehicle and possibly collides with the vehicle, and

wherein the image processing method further comprises:

adding, to the second annotation attached in the attaching, a hazard level that increases as an area of the second region decreases.

9. The image processing method according to claim 1,

wherein the second region is a hazard region in which, when objects represented by the two or more first regions approach, at least one of the objects crosses in front of the vehicle and possibly collides with the vehicle, and

wherein the image processing method further comprises:

adding, to the second annotation attached in the attaching, different hazard levels for a first sub-region and a second sub-region that constitute the second region, the hazard level for one of the first sub-region and the second sub-region that is closer to one of the two first regions having the second region interposed therebetween and involving a larger amount of movement being higher than the hazard level for the other of the first sub-region and the second sub-region.

10. The image processing method according to claim 1, wherein the determining includes

performing a first determining process of determining, in reverse chronological order from the image corresponding to the last time point in the time series, the first image corresponding to a third time point to which the first annotation is not attached from among the plurality of consecutive time-series images acquired in the acquiring, and

performing a second determining process of determining, through image processing, whether the first region exists at a position in each of the images that is obtained by shifting the first region in an image corresponding to the next time point after the third time point of the first image that has been determined through the first determining process in a direction perpendicular to a direction in which the vehicle moves in reverse chronological order from the image corresponding to the third time point.

13. The method according to claim 1, further comprising:

autonomously attaching, by the one or more processors, to the second annotation attached in the attaching, another annotation that requires a higher level of recognition compared to the first annotation.

14. The method according to claim 1, further comprising:

autonomously attaching, by the one or more processors, the first region that does not require a higher level of recognition to the plurality of time-series images that have been captured by the onboard video camera; and

autonomously attaching, by the one or more processors, the second region that requires a higher level of recognition to the plurality of time-series images that have been captured by the onboard video camera.

11. An image processing apparatus comprising one or more processors configured to:

acquire a plurality of consecutive time-series images that have been captured by an onboard camera mounted on a vehicle and to which a first annotation indicating two or more first regions is attached, the two or more first regions including at least one person region, the plurality of consecutive time-series images at least including one or more images in which the two or more first regions are located on a path of the vehicle and a distance between the two or more first regions is smaller than or equal to a threshold;

determine, in reverse chronological order from an image corresponding to the last time point in the time series, a position of each of the two or more regions in each of the plurality of acquired consecutive time-series images;

identify, from among the plurality of consecutive time-series images, a first image corresponding to a first time point for which it has been determined for the first time that none of the two or more first regions are located on the path, and sets, as a second region, a region between the two or more first regions in the identified first image; and

attach a second annotation to the first image corresponding to the first time point, the second annotation indicating the set second region set.

12. A non-transitory computer-readable recording medium storing a program thereon, the program causing a computer to execute:

acquiring a plurality of consecutive time-series images that have been captured by an onboard camera mounted on a vehicle and to which a first annotation indicating two or more first regions is attached, the two or more first regions including at least one person region, the plurality of consecutive time-series images at least including one or more images in which the two or more first regions are located on a path of the vehicle and a distance between the two or more first regions is smaller than or equal to a threshold;

determining, in reverse chronological order from an image corresponding to the last time point in the time series, a position of each of the two or more regions in each of the plurality of consecutive time-series images acquired in the acquiring;

identifying, from among the plurality of consecutive time-series images, a first image corresponding to a first time point for which it has been determined for the first time in the determining that none of the two or more first regions are located on the path, and setting, as a second region, a region between the two or more first regions in the identified first image; and

attaching a second annotation to the first image corresponding to the first time point, the second annotation indicating the second region set in the setting.