Image processing device, imaging device, equipment control system, equipment, image processing method, and recording medium storing program

• US 10,643,077 B2
• Filed: 08/01/2017
• Issued: 05/05/2020
• Est. Priority Date: 08/25/2016
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Nomura, Yasuhiro
• Primary Examiner(s): Ustaris, Joseph G
• Assistant Examiner(s): Kim, Matthew David
• Application Number:
  US15/665,869
• Publication Number:
  US 20180060671A1
• Time in Patent Office: 1,008 Days
• Field of Search: 348143
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 7/12 for automatic initiation; for initiation not subject to will of driver or passenger limiting speed of vehicles other than rail vehicles B60K31/00
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01S 13/00 Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/168 Driving aids for parking, e.g. acoustic or visual feedback on parking space
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof

Abstract

An image processing device, an imaging device, an equipment control system, equipment, an image processing method, and a recording medium storing a program. The image processing device and the image processing method includes capturing an object to obtain a captured image using at least one capturing unit, generating a correction parameter based on the captured image to update a correction parameter stored in a memory, correcting displacement in coordinate caused to the captured image, using the updated correction parameter, and updating a referring status of a specific thing included in the captured image to a not-yet-referred status when the correction parameter is updated. The imaging device includes the image processing device, the at least one capturing unit captures an image including the specific thing, and the memory stores the correction parameter.

Claims

1. An image processing device, comprising:

circuitry configured to

recognize and perform tracking of a specific object present in a first captured image captured by at least one capturing unit, and set a recognition status of the specific object to a first status indicating that the specific object has been recognized in the first captured image and will be tracked in a second captured image captured subsequent to the first captured image;

correct a displacement in coordinates between the first captured image and the second captured image, using a correction parameter stored in a memory,

generate a new correction parameter, based on the second captured image, and update the correction parameter stored in the memory to the generated new correction parameter,

determine whether the correction parameter used to correct the displacement has been newly updated in the memory;

when determining that the correction parameter has been newly updated, update the recognition status of the specific object included in the second captured image to a second status, wherein the second status of the specific object indicates a state in which the specific object has not been recognized and is not being tracked, so as to cause the circuitry to newly perform a recognition process to recognize the specific object using the second captured image; and

when determining that the correction parameter has not been newly updated, not update the recognition status of the specific object to the second status.

2. The image processing device according to claim 1, wherein when the specific object is at a distance equal to or longer than a prescribed distance, the circuitry is further configured to update the recognition status of the specific object to the second status.

3. The image processing device according to claim 1, wherein

when the recognition status of the specific object is updated to the second status, the circuitry is further configured to perform the recognition process to recognize the specific object using a corrected image obtained by correcting the second captured image based on the updated correction parameter, and

when the recognition status of the specific object is maintained, the circuitry is further configured to perform the recognition process using a result of the recognition process at a previous time.

4. The image processing device of claim 3, wherein when the recognition status of the specific object is updated to the second status, the circuitry is further configured to perform the recognition process.

5. The image processing device according to claim 1, wherein

the first status indicates a state in which the specific object has been recognized by the circuitry or a state in which the specific object is being tracked by the circuitry.

6. An imaging device, comprising:

the image processing device according to claim 1;

the at least one capturing unit that captures an image including the specific object; and

the memory that stores the correction parameter.

7. The imaging device according to claim 6, wherein

the at least one capturing unit comprises two capturing units, and

the imaging device further comprises a disparity calculator configured to calculate a disparity of the specific object based on a corrected image obtained by correcting, using the correction parameter, the captured image including the specific object, the captured image being captured by each of the two capturing units.

8. A control system to control a device, comprising:

the imaging device according to claim 6; and

a controller configured to control operation of at least one subsystem of the device based on information obtained using the imaging device.

9. A device, comprising:

the at least one subsystem of the device; and

the control system according to claim 8 to control the at least one subsystem of the device.

10. The imaging device of claim 6, wherein the at least one capturing unit includes a first camera and a second camera, and the circuitry is further configured to generate the new correction parameter and correct the displacement between the first captured image and the second captured image, which is due to the displacement of the first and second cameras over time.

11. The image processing device of claim 1, wherein the circuitry is further configured to generate the new correction parameter at predetermined time intervals.

12. The image processing device of claim 1, wherein the circuitry is further configured to generate the correction parameter, which is used in a correction formula by the circuitry to correct the displacement, the correction parameter including a plurality of coefficients.

13. The image processing device of claim 1, wherein the circuitry is further configured to correct the displacement in the coordinates in horizontal and vertical directions using the correction parameter.

14. A method of processing an image, the method comprising:

recognizing and performing tracking of a specific object present in a first captured image captured by at least one capturing unit, and setting a recognition status of the specific object to a first status indicating that the specific object has been recognized in the first captured image and will be tracked in a second captured image captured subsequent to the first captured image;

generating a correction parameter, based on the second captured image, and update an existing correction parameter stored in a memory to the generated correction parameter;

correcting displacement in coordinates of the captured image, using the updated correction parameter;

determining whether the correction parameter used to correct the displacement has been newly updated in the memory;

when determining that the correction parameter has been newly updated, updating the recognition status of the specific object included in the second captured image to a second status in response to updating the correction parameter in the memory, wherein the second status of the specific object indicates a state in which the specific object has not been recognized and is not being tracked, so as to cause the circuitry to newly perform a recognition process to recognize the specific object using the second captured image; and

when determining that the correction parameter has not been newly updated, not updating the recognition status of the specific object to the second status.

15. A computer-readable non-transitory recording medium storing a program for causing a computer to execute the method of claim 14.