Image capturing apparatus, image capturing method, storage medium, and device control system for controlling vehicle-mounted devices

• US 9,426,377 B2
• Filed: 09/18/2015
• Issued: 08/23/2016
• Est. Priority Date: 10/03/2014
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Kida, Shintaroh, Sekiguchi, Hiroyoshi, Oshikiri, Kohji, Itoh, Izumi
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US14/857,932
• Publication Number:
  US 20160100088A1
• Time in Patent Office: 340 Days
• US Class Current: 1/1
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: B60W 2050/0043 Signal treatments, identification of variables or parameters, parameter estimation or state estimation
• CPC Class Codes: B60W 50/00 Details of control systems for road vehicle drive control not related to the control of a particular sub-unit , e.g. process diagnostic or vehicle driver interfaces
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An image capturing apparatus includes a light source to emit first light to a light translucent member, an image sensor to capture an adhering detection frame for acquiring an adhering detection image to detect a substance adhering on the light translucent member by using the first light emitted from the light source, and to capture object recognition frames for acquiring object recognition images for recognizing objects other than the substance by receiving second light, and circuitry to set a frame time of the adhering detection frame shorter than a frame time of each of the object recognition frames, and to set an exposure time of an object recognition frame right after the adhering detection frame to a value that prevents overlapping of an exposure period of the adhering detection frame and an exposure period of the object recognition frame right after the adhering detection frame.

Claims

1. An image capturing apparatus comprising:

a light source to emit first light to a light translucent member;

an image sensor to capture an adhering detection frame for acquiring an adhering detection image to detect a substance adhering on the light translucent member by using the first light emitted from the light source, and to capture one or more object recognition frames for acquiring one or more object recognition images for recognizing one or more objects other than the substance by receiving second light, different from the first light emitted from the light source, by performing an image capturing operation of the adhering detection frame and the object recognition frames repeatedly; and

circuitry to set a frame time of the adhering detection frame shorter than a frame time of each of the object recognition frames, and to set an exposure time of an object recognition frame right after the adhering detection frame to a value that prevents overlapping of an exposure period of the adhering detection frame and an exposure period of the object recognition frame right after the adhering detection frame.

2. The image capturing apparatus of claim 1, wherein the object recognition frames include an automatic exposure frame capturable by the image sensor under an automatic exposure control that changes an exposure time automatically, and a locked-exposure frame capturable by the image sensor with a locked exposure time,

wherein the object recognition frame right after the adhering frame is captured as the locked-exposure frame exposed with the locked exposure time having a value that prevents overlapping of the exposure period of the adhering detection frame and the exposure period of the object recognition frame captured as the locked-exposure frame right after the adhering frame.

3. The image capturing apparatus of claim 2, further comprising an image converter to convert a first object recognition image acquired by setting the object recognition frame right after the adhering frame as the locked-exposure frame to a second object recognition image based on an exposure time set for capturing the automatic exposure frame.

4. The image capturing apparatus of claim 1, wherein the object recognition frames include an automatic exposure frame capturable by the image sensor under an automatic exposure control that changes an exposure time automatically,

wherein the object recognition frame right after the adhering detection frame is captured as the automatic exposure frame,

wherein the circuitry sets a upper limit of an exposure time of the object recognition frame captured as the automatic exposure frame right after the adhering detection frame, the upper limit changeable by the automatic exposure control, to a value that prevents overlapping of the exposure period of the adhering detection frame and the exposure period of the object recognition frame captured as the automatic exposure frame right after the adhering detection frame.

5. The image capturing apparatus of claim 4, further comprising an image converter, wherein when an exposure time of an automatic exposure frame not right after the adhering detection frame is set greater than the upper limit, the image converter converts a third object recognition image acquired by capturing the object recognition frame as the automatic exposure frame right after the adhering frame to a fourth object recognition image based on the exposure time set greater than the upper limit.

6. The image capturing apparatus of claim 1, wherein the image sensor includes first light receiving elements used for acquiring the adhering detection image of the adhering detection frame, and second light receiving elements used for acquiring the one or more object recognition images of the object recognition frames, and the number of the first light receiving elements is smaller than the number of the second light receiving elements.

7. The image capturing apparatus of claim 6, wherein the image capturing operation employs a rolling shutter method, and the image sensor starts outputting of data from the first light receiving elements used for acquiring the adhering detection image of the adhering detection frame.

8. The image capturing apparatus of claim 1, further comprising:

a light source controller to perform a light emission control of the light source,

wherein the image capturing operation employs the rolling shutter method,

wherein the light source controller performs the light emission control of the light source for the adhering detection frame by setting at least one time period emitting the first light from the light source, and another one time period not emitting the first light from the light source during the exposure period of the adhering detection frame,

wherein the circuitry sets the exposure time of the object recognition frame right after the adhering detection frame to a value that prevents overlapping of the exposure period of the adhering detection frame irradiated by the first light emitted from the light source and the exposure period of the object recognition frame right after the adhering detection frame.

11. A device control system to control vehicle-mounted devices comprising:

the image capturing apparatus of claim 1 to capture the adhering detection image and the object recognition images; and

one or more device controllers to control one or more devices mounted in a vehicle based on the adhering detection image and the object recognition images captured by the image capturing apparatus.

9. A method of capturing images by using an image capturing apparatus, comprising the steps of:

emitting first light from a light source to a light translucent member;

capturing an adhering detection frame for acquiring an adhering detection image to detect a substance adhering on the light translucent member by using the first light emitted from the light source and an image sensor;

capturing one or more object recognition frames for acquiring one or more object recognition images for recognizing one or more objects other than the substance by receiving second light, different from the first light emitted from the light source, by using the image sensor by repeatedly performing an image capturing operation of the adhering detection frame and the object recognition frames;

setting a frame time of the adhering detection frame shorter than a frame time of each of the object recognition frames; and

setting an exposure time of an object recognition frame right after the adhering detection frame (R4) to a value that prevents overlapping of an exposure period of the adhering detection frame and an exposure period of the object recognition frame right after the adhering detection frame.

10. A non-transitory computer-readable storage medium storing a program that, when executed by a computer, causes the computer to execute a method of capturing images by using an image capturing apparatus, the method comprising the steps of:

emitting first light from a light source to a light translucent member;

capturing an adhering detection frame for acquiring an adhering detection image to detect a substance adhering on the light translucent member by using the first light emitted from the light source and an image sensor;

capturing one or more object recognition frames for acquiring one or more object recognition images for recognizing one or more objects other than the substance by receiving second light, different from the first light emitted from the light source, by using the image sensor by repeatedly performing an image capturing operation of the adhering detection frame and the object recognition frames;

setting a frame time of the adhering detection frame shorter than a frame time of each of the object recognition frames; and

setting an exposure time of an object recognition frame right after the adhering detection frame (R4) to a value that prevents overlapping of an exposure period of the adhering detection frame and an exposure period of the object recognition frame right after the adhering detection frame.